Benchmark summary
BBH-based reasoning benchmark ported to the SC# evaluation runtime. It tests multi-step reasoning across challenging tasks involving logic, deduction, symbolic manipulation, planning, and problem solving.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Reasoning - BBH boolean expressions (set 1)
32 passed |
100.0% |
IN30 848 OUT8 190 | |
| ✓ | BbhBooleanExpressionsBD1not True or ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT318 |
| ✓ | BbhBooleanExpressionsBD10( True and not True and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT155 |
| ✓ | BbhBooleanExpressionsBD11False and False and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT394 |
| ✓ | BbhBooleanExpressionsBD12False or ( False and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT270 |
| ✓ | BbhBooleanExpressionsBD13True and False or ( not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT176 |
| ✓ | BbhBooleanExpressionsBD14not not ( True ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT255 |
| ✓ | BbhBooleanExpressionsBD15not False or ( True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT245 |
| ✓ | BbhBooleanExpressionsBD16not ( True ) or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT185 |
| ✓ | BbhBooleanExpressionsBD17( True and not not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT270 |
| ✓ | BbhBooleanExpressionsBD18( False or not False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT187 |
| ✓ | BbhBooleanExpressionsBD19False and False or True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT212 |
| ✓ | BbhBooleanExpressionsBD2not True or ( True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT227 |
| ✓ | BbhBooleanExpressionsBD20not not False or not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT358 |
| ✓ | BbhBooleanExpressionsBD21True and not True or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT232 |
| ✓ | BbhBooleanExpressionsBD22not False or True and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT296 |
| ✓ | BbhBooleanExpressionsBD23False and ( not False ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT269 |
| ✓ | BbhBooleanExpressionsBD24not not not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT233 |
| ✓ | BbhBooleanExpressionsBD25( True ) and True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT224 |
| ✓ | BbhBooleanExpressionsBD26not True or False and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT262 |
| ✓ | BbhBooleanExpressionsBD27not not not not not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT344 |
| ✓ | BbhBooleanExpressionsBD28not True or not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT245 |
| ✓ | BbhBooleanExpressionsBD29False or True or False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT279 |
| ✓ | BbhBooleanExpressionsBD3False or ( False ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT215 |
| ✓ | BbhBooleanExpressionsBD30not ( True or True ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT257 |
| ✓ | BbhBooleanExpressionsBD31not not not not True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT296 |
| ✓ | BbhBooleanExpressionsBD32( False ) or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT163 |
| ✓ | BbhBooleanExpressionsBD4not False or True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT362 |
| ✓ | BbhBooleanExpressionsBD5not True or False or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT177 |
| ✓ | BbhBooleanExpressionsBD6True and True and False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT186 |
| ✓ | BbhBooleanExpressionsBD7not not True and not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT324 |
| ✓ | BbhBooleanExpressionsBD8not not not ( True and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT251 |
| ✓ | BbhBooleanExpressionsBD9not not False and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT323 |
Reasoning - BBH boolean expressions (set 2)
31 passed1 failed |
96.9% |
IN30 848 OUT7 953 | |
| ✓ | BbhBooleanExpressionsBD33( False and True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT255 |
| ✓ | BbhBooleanExpressionsBD34not not not ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT181 |
| ✓ | BbhBooleanExpressionsBD35not not True and not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT375 |
| ✓ | BbhBooleanExpressionsBD36not False or True and not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT274 |
| ✓ | BbhBooleanExpressionsBD37True or True or not False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT168 |
| ✓ | BbhBooleanExpressionsBD38( not False and False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT184 |
| ✓ | BbhBooleanExpressionsBD39( ( not True ) or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT87 |
| ✓ | BbhBooleanExpressionsBD40not not ( not False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT429 |
| ✓ | BbhBooleanExpressionsBD41not True or False or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT205 |
| ✓ | BbhBooleanExpressionsBD42False or ( not False ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT216 |
| ✓ | BbhBooleanExpressionsBD43not False or False or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT234 |
| ✓ | BbhBooleanExpressionsBD44not not False or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT326 |
| ✓ | BbhBooleanExpressionsBD45True or ( ( not True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT208 |
| ✓ | BbhBooleanExpressionsBD46not False and not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT167 |
| ✓ | BbhBooleanExpressionsBD47( False ) and not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT280 |
| ✓ | BbhBooleanExpressionsBD48( True ) and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT201 |
| ✓ | BbhBooleanExpressionsBD49( not False and ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT126 |
| ✓ | BbhBooleanExpressionsBD50False or not ( False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT206 |
| ✓ | BbhBooleanExpressionsBD51True and not False and not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT197 |
| ✓ | BbhBooleanExpressionsBD52not True and True and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT292 |
| ✓ | BbhBooleanExpressionsBD53not not ( False and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT309 |
| × | BbhBooleanExpressionsBD54True or not ( ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("True")
Context: BbhBooleanExpressionsBD54: expected True, got [false]
Values:
got = "false" |
Failed | IN964 OUT750 |
| ✓ | BbhBooleanExpressionsBD55False or not True or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT172 |
| ✓ | BbhBooleanExpressionsBD56False or not not True or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT262 |
| ✓ | BbhBooleanExpressionsBD57not ( True or True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT176 |
| ✓ | BbhBooleanExpressionsBD58False or not not not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT312 |
| ✓ | BbhBooleanExpressionsBD59not ( False ) or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT220 |
| ✓ | BbhBooleanExpressionsBD60True and not False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT292 |
| ✓ | BbhBooleanExpressionsBD61True or ( not ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT124 |
| ✓ | BbhBooleanExpressionsBD62( False or True and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT164 |
| ✓ | BbhBooleanExpressionsBD63False or not True and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT297 |
| ✓ | BbhBooleanExpressionsBD64not not not not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT264 |
Reasoning - BBH boolean expressions (set 3)
32 passed |
100.0% |
IN30 848 OUT7 438 | |
| ✓ | BbhBooleanExpressionsBD65( not False ) and False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT233 |
| ✓ | BbhBooleanExpressionsBD66False and False or not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT248 |
| ✓ | BbhBooleanExpressionsBD67( not False or True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT301 |
| ✓ | BbhBooleanExpressionsBD68not ( False ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT122 |
| ✓ | BbhBooleanExpressionsBD69False or True and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT276 |
| ✓ | BbhBooleanExpressionsBD70( True ) and True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT201 |
| ✓ | BbhBooleanExpressionsBD71( not True ) and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT227 |
| ✓ | BbhBooleanExpressionsBD72( not not True and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT267 |
| ✓ | BbhBooleanExpressionsBD73True and ( False ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT226 |
| ✓ | BbhBooleanExpressionsBD74not not not False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT298 |
| ✓ | BbhBooleanExpressionsBD75not not not False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT270 |
| ✓ | BbhBooleanExpressionsBD76True or ( True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT209 |
| ✓ | BbhBooleanExpressionsBD77not False or ( ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT207 |
| ✓ | BbhBooleanExpressionsBD78not not ( False and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT260 |
| ✓ | BbhBooleanExpressionsBD79not not not not True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT254 |
| ✓ | BbhBooleanExpressionsBD80False and not ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT256 |
| ✓ | BbhBooleanExpressionsBD81True and True and True or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT230 |
| ✓ | BbhBooleanExpressionsBD82False or not False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT258 |
| ✓ | BbhBooleanExpressionsBD83not False and False and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT223 |
| ✓ | BbhBooleanExpressionsBD84( False ) or not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT228 |
| ✓ | BbhBooleanExpressionsBD85( not False ) or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT154 |
| ✓ | BbhBooleanExpressionsBD86not not ( False or not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT322 |
| ✓ | BbhBooleanExpressionsBD87False or not False or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT240 |
| ✓ | BbhBooleanExpressionsBD88False or not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT184 |
| ✓ | BbhBooleanExpressionsBD89False or ( False ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT227 |
| ✓ | BbhBooleanExpressionsBD90not True or False and True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT254 |
| ✓ | BbhBooleanExpressionsBD91True or ( True or not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT186 |
| ✓ | BbhBooleanExpressionsBD92True and not False and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT235 |
| ✓ | BbhBooleanExpressionsBD93False or not False or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT263 |
| ✓ | BbhBooleanExpressionsBD94( True or not False ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT156 |
| ✓ | BbhBooleanExpressionsBD95False or not ( True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT244 |
| ✓ | BbhBooleanExpressionsBD96not False and not True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT179 |
Reasoning - BBH boolean expressions (set 4)
31 passed1 failed |
96.9% |
IN30 848 OUT7 043 | |
| ✓ | BbhBooleanExpressionsBD100( True or not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT233 |
| ✓ | BbhBooleanExpressionsBD101False and ( False or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT216 |
| ✓ | BbhBooleanExpressionsBD102False or not True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT210 |
| ✓ | BbhBooleanExpressionsBD103True and not True or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT208 |
| ✓ | BbhBooleanExpressionsBD104( not False or ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT140 |
| ✓ | BbhBooleanExpressionsBD105( ( not ( True ) ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT155 |
| ✓ | BbhBooleanExpressionsBD106False and False and not ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT252 |
| ✓ | BbhBooleanExpressionsBD107not True and not True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT164 |
| ✓ | BbhBooleanExpressionsBD108not ( True or False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT202 |
| ✓ | BbhBooleanExpressionsBD109False or ( True ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT237 |
| ✓ | BbhBooleanExpressionsBD110False or False and False or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT258 |
| × | BbhBooleanExpressionsBD111True or False or not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("True")
Context: BbhBooleanExpressionsBD111: expected True, got [false]
Values:
got = "false" |
Failed | IN964 OUT161 |
| ✓ | BbhBooleanExpressionsBD112not True and ( True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT192 |
| ✓ | BbhBooleanExpressionsBD113not True and True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT195 |
| ✓ | BbhBooleanExpressionsBD114not False or ( True or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT209 |
| ✓ | BbhBooleanExpressionsBD115not ( False or ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT202 |
| ✓ | BbhBooleanExpressionsBD116not ( True or True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT165 |
| ✓ | BbhBooleanExpressionsBD117False or not False or True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT208 |
| ✓ | BbhBooleanExpressionsBD118False or False and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT340 |
| ✓ | BbhBooleanExpressionsBD119not ( True or False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT251 |
| ✓ | BbhBooleanExpressionsBD120( False ) or not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT186 |
| ✓ | BbhBooleanExpressionsBD121True and True and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT259 |
| ✓ | BbhBooleanExpressionsBD122( True ) or True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT315 |
| ✓ | BbhBooleanExpressionsBD123False and False and not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT192 |
| ✓ | BbhBooleanExpressionsBD124( True ) and not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT218 |
| ✓ | BbhBooleanExpressionsBD125not not not True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT267 |
| ✓ | BbhBooleanExpressionsBD126False and False and not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT319 |
| ✓ | BbhBooleanExpressionsBD127False and False or False or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT213 |
| ✓ | BbhBooleanExpressionsBD128False or ( not ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT213 |
| ✓ | BbhBooleanExpressionsBD97False and ( True and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT216 |
| ✓ | BbhBooleanExpressionsBD98not False or ( False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT237 |
| ✓ | BbhBooleanExpressionsBD99False and not ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT210 |
Reasoning - BBH boolean expressions (set 5)
30 passed |
100.0% |
IN28 920 OUT6 840 | |
| ✓ | BbhBooleanExpressionsBD129not True and True and not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT411 |
| ✓ | BbhBooleanExpressionsBD130( False ) and ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT194 |
| ✓ | BbhBooleanExpressionsBD131( not True ) and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT261 |
| ✓ | BbhBooleanExpressionsBD132False and True and ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT247 |
| ✓ | BbhBooleanExpressionsBD133( ( True ) ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT181 |
| ✓ | BbhBooleanExpressionsBD134not True and False or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT242 |
| ✓ | BbhBooleanExpressionsBD135True and not True and True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT183 |
| ✓ | BbhBooleanExpressionsBD136( True or True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT189 |
| ✓ | BbhBooleanExpressionsBD137True or not False and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT158 |
| ✓ | BbhBooleanExpressionsBD138( not ( False ) or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT130 |
| ✓ | BbhBooleanExpressionsBD139True and not ( True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT181 |
| ✓ | BbhBooleanExpressionsBD140not ( False and True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT197 |
| ✓ | BbhBooleanExpressionsBD141not ( True and True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT224 |
| ✓ | BbhBooleanExpressionsBD142False or not True and not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT254 |
| ✓ | BbhBooleanExpressionsBD143not True and False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT179 |
| ✓ | BbhBooleanExpressionsBD144not True or True and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT228 |
| ✓ | BbhBooleanExpressionsBD145( not ( False ) and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT203 |
| ✓ | BbhBooleanExpressionsBD146False or ( True and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT190 |
| ✓ | BbhBooleanExpressionsBD147not False and not True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT237 |
| ✓ | BbhBooleanExpressionsBD148( False or not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT232 |
| ✓ | BbhBooleanExpressionsBD149( not False ) or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT322 |
| ✓ | BbhBooleanExpressionsBD150( not not True ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT244 |
| ✓ | BbhBooleanExpressionsBD151True or not ( ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT238 |
| ✓ | BbhBooleanExpressionsBD152( False ) and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT233 |
| ✓ | BbhBooleanExpressionsBD153not False and True or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT230 |
| ✓ | BbhBooleanExpressionsBD154True or not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT255 |
| ✓ | BbhBooleanExpressionsBD155False and not True and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT338 |
| ✓ | BbhBooleanExpressionsBD156not not True and True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT215 |
| ✓ | BbhBooleanExpressionsBD157not False or not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT219 |
| ✓ | BbhBooleanExpressionsBD158not not ( True ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT225 |
Reasoning - BBH boolean expressions (set 6)
32 passed |
100.0% |
IN30 848 OUT7 546 | |
| ✓ | BbhBooleanExpressionsBe166( True and not True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT181 |
| ✓ | BbhBooleanExpressionsBe167True or False or not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT135 |
| ✓ | BbhBooleanExpressionsBe168not True or True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT377 |
| ✓ | BbhBooleanExpressionsBe169True and not not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT259 |
| ✓ | BbhBooleanExpressionsBe170not True and ( False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT205 |
| ✓ | BbhBooleanExpressionsBe171False and False or not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT186 |
| ✓ | BbhBooleanExpressionsBe172True or False and not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT390 |
| ✓ | BbhBooleanExpressionsBe173not ( False or False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT179 |
| ✓ | BbhBooleanExpressionsBe174not False or True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT277 |
| ✓ | BbhBooleanExpressionsBe175not False and False or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT319 |
| ✓ | BbhBooleanExpressionsBe176( not True ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT232 |
| ✓ | BbhBooleanExpressionsBe177True and not not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT220 |
| ✓ | BbhBooleanExpressionsBe178not ( not not False ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT175 |
| ✓ | BbhBooleanExpressionsBe179not ( not False ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT242 |
| ✓ | BbhBooleanExpressionsBe180False or True and not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT221 |
| ✓ | BbhBooleanExpressionsBe181( not not not True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT274 |
| ✓ | BbhBooleanExpressionsBe182False and not ( True ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT231 |
| ✓ | BbhBooleanExpressionsBe183( not True and False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT166 |
| ✓ | BbhBooleanExpressionsBe184not ( not not True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT168 |
| ✓ | BbhBooleanExpressionsBe185( not False ) or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT141 |
| ✓ | BbhBooleanExpressionsBe186not not not False and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT330 |
| ✓ | BbhBooleanExpressionsBe187True or not True or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT224 |
| ✓ | BbhBooleanExpressionsBe188not False or not not False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT333 |
| ✓ | BbhBooleanExpressionsBe189not False and ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT302 |
| ✓ | BbhBooleanExpressionsBe190False and True or ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT233 |
| ✓ | BbhBooleanExpressionsBe191False and not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT228 |
| ✓ | BbhBooleanExpressionsBe192not ( ( False and True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT179 |
| ✓ | BbhBooleanExpressionsBe193False or ( not False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT219 |
| ✓ | BbhBooleanExpressionsBe194False and not False or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT226 |
| ✓ | BbhBooleanExpressionsBe195not False and not not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT265 |
| ✓ | BbhBooleanExpressionsBe196True and True or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT218 |
| ✓ | BbhBooleanExpressionsBe197not True or False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT211 |
Reasoning - BBH boolean expressions (set 7)
31 passed1 failed |
96.9% |
IN30 848 OUT6 926 | |
| × | BbhBooleanExpressionsBe198True or False or not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("True")
Context: BbhBooleanExpressionsBe198: expected True, got [False] |
Failed | IN964 OUT137 |
| ✓ | BbhBooleanExpressionsBe199False or False or not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT171 |
| ✓ | BbhBooleanExpressionsBe200False or False or True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT242 |
| ✓ | BbhBooleanExpressionsBe201( not ( True or False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT181 |
| ✓ | BbhBooleanExpressionsBe202not False and False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT210 |
| ✓ | BbhBooleanExpressionsBe203True and False or ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT235 |
| ✓ | BbhBooleanExpressionsBe204not False or not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT276 |
| ✓ | BbhBooleanExpressionsBe205True and ( not ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT120 |
| ✓ | BbhBooleanExpressionsBe206True and False and not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT219 |
| ✓ | BbhBooleanExpressionsBe207( ( not False ) ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT155 |
| ✓ | BbhBooleanExpressionsBe208not ( True ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT227 |
| ✓ | BbhBooleanExpressionsBe209True or not False and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT168 |
| ✓ | BbhBooleanExpressionsBe210True and not ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT236 |
| ✓ | BbhBooleanExpressionsBe211not True and False or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT272 |
| ✓ | BbhBooleanExpressionsBe212not True or not False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT229 |
| ✓ | BbhBooleanExpressionsBe213not True or False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT210 |
| ✓ | BbhBooleanExpressionsBe214not True and True and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT390 |
| ✓ | BbhBooleanExpressionsBe215False and False or not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT200 |
| ✓ | BbhBooleanExpressionsBe216( not False or False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT220 |
| ✓ | BbhBooleanExpressionsBe217( True or False ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT216 |
| ✓ | BbhBooleanExpressionsBe218not not not ( False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT256 |
| ✓ | BbhBooleanExpressionsBe219not False and False and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT187 |
| ✓ | BbhBooleanExpressionsBe220False and not True and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT249 |
| ✓ | BbhBooleanExpressionsBe221( False or not False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT228 |
| ✓ | BbhBooleanExpressionsBe222False or ( False or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT191 |
| ✓ | BbhBooleanExpressionsBe223not False or True or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT232 |
| ✓ | BbhBooleanExpressionsBe224True or ( not False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT137 |
| ✓ | BbhBooleanExpressionsBe225not False and False or True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT278 |
| ✓ | BbhBooleanExpressionsBe226False and not ( not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT213 |
| ✓ | BbhBooleanExpressionsBe227not not ( not True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT174 |
| ✓ | BbhBooleanExpressionsBe228False and not True and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT273 |
| ✓ | BbhBooleanExpressionsBe229True or False and not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT194 |
Reasoning - BBH boolean expressions (set 8)
29 passed3 failed |
90.6% |
IN33 175 OUT13 566 | |
| ✓ | BbhBooleanExpressionsBe230not ( False ) and ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT180 |
| ✓ | BbhBooleanExpressionsBe231False and not ( ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT215 |
| ✓ | BbhBooleanExpressionsBe232not False or ( not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT283 |
| ✓ | BbhBooleanExpressionsBe233( ( not False ) and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT123 |
| ✓ | BbhBooleanExpressionsBe234True or False and not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT406 |
| ✓ | BbhBooleanExpressionsBe235( ( not True and True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT163 |
| ✓ | BbhBooleanExpressionsBe236not ( False ) or not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT205 |
| ✓ | BbhBooleanExpressionsBe237not ( False or False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT234 |
| ✓ | BbhBooleanExpressionsBe238False and not ( not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT223 |
| ✓ | BbhBooleanExpressionsBe239False and False and not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT295 |
| ✓ | BbhBooleanExpressionsBe240False or ( not False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT307 |
| ✓ | BbhBooleanExpressionsBe241False or ( not not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT212 |
| ✓ | BbhBooleanExpressionsBe242( not False ) or not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT274 |
| ✓ | BbhBooleanExpressionsBe243not False and False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT278 |
| ✓ | BbhBooleanExpressionsBe244True or not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT198 |
| ✓ | BbhBooleanExpressionsBe245( not ( False ) ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT129 |
| ✓ | BbhBooleanExpressionsBe246True and not True or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT199 |
| ✓ | BbhBooleanExpressionsBe247not ( not not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT196 |
| ✓ | BbhBooleanExpressionsBe248True and ( not True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT182 |
| ✓ | BbhBooleanExpressionsBe249( not not not False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT231 |
| × | BbhCausalJudgementBe10judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe10: expected Yes, got [No] |
Failed | IN1 113 OUT534 |
| ✓ | BbhCausalJudgementBe11judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 286 |
| ✓ | BbhCausalJudgementBe12judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT527 |
| ✓ | BbhCausalJudgementBe13judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT843 |
| × | BbhCausalJudgementBe14judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe14: expected Yes, got [No] |
Failed | IN1 147 OUT557 |
| ✓ | BbhCausalJudgementBe15judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 182 OUT513 |
| ✓ | BbhCausalJudgementBe16judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT398 |
| × | BbhCausalJudgementBe17judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("No")
Context: BbhCausalJudgementBe17: expected No, got [Yes] |
Failed | IN1 172 OUT941 |
| ✓ | BbhCausalJudgementBe18judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 353 OUT2 173 |
| ✓ | BbhCausalJudgementBe19judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT297 |
| ✓ | BbhCausalJudgementBe8judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT379 |
| ✓ | BbhCausalJudgementBe9judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT585 |
Reasoning - BBH Dyck languages - close the brackets (set 1)
25 passed4 failed1 broken |
83.3% |
IN29 771 OUT95 225 | |
| ✓ | BbhDyckLanguagesBi13correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT384 |
| ✓ | BbhDyckLanguagesBi14correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT3 345 |
| ✓ | BbhDyckLanguagesBi15correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT845 |
| × | BbhDyckLanguagesBi16correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") }")
Context: BbhDyckLanguagesBi16: expected ) }, got [{ ( [ [ ] ( ) ] ) }] |
Failed | IN981 OUT336 |
| ✓ | BbhDyckLanguagesBi17correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT316 |
| ✓ | BbhDyckLanguagesBi18correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT2 111 |
| ✓ | BbhDyckLanguagesBi19correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT489 |
| ! | BbhDyckLanguagesBi20correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 064 OUT62 144 |
| ✓ | BbhDyckLanguagesBi21correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT613 |
| × | BbhDyckLanguagesBi22correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ] >")
Context: BbhDyckLanguagesBi22: expected } ] >, got [}]] |
Failed | IN1 026 OUT1 881 |
| × | BbhDyckLanguagesBi23correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] > )")
Context: BbhDyckLanguagesBi23: expected ] > ), got [( < [ ( ) ] > )] |
Failed | IN978 OUT622 |
| ✓ | BbhDyckLanguagesBi24correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT659 |
| ✓ | BbhDyckLanguagesBi25correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT367 |
| ✓ | BbhDyckLanguagesBi26correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT801 |
| ✓ | BbhDyckLanguagesBi27correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT958 |
| × | BbhDyckLanguagesBi28correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBi28: expected ) ), got [( ( { } ) )] |
Failed | IN977 OUT422 |
| ✓ | BbhDyckLanguagesBi29correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 556 |
| ✓ | BbhDyckLanguagesBi30correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT829 |
| ✓ | BbhDyckLanguagesBi31correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 013 OUT1 390 |
| ✓ | BbhDyckLanguagesBi32correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT828 |
| ✓ | BbhDyckLanguagesBi33correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 022 OUT1 613 |
| ✓ | BbhDyckLanguagesBi34correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT150 |
| ✓ | BbhDyckLanguagesBi35correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT172 |
| ✓ | BbhDyckLanguagesBi36correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT908 |
| ✓ | BbhDyckLanguagesBi37correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT2 501 |
| ✓ | BbhDyckLanguagesBi38correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT405 |
| ✓ | BbhDyckLanguagesBi39correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 056 OUT7 023 |
| ✓ | BbhDyckLanguagesBi40correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT441 |
| ✓ | BbhDyckLanguagesBi41correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT340 |
| ✓ | BbhDyckLanguagesBi42correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT776 |
Reasoning - BBH Dyck languages - close the brackets (set 2)
30 passed2 failed |
93.8% |
IN31 990 OUT61 561 | |
| ✓ | BbhDyckLanguagesBj51correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 036 |
| ✓ | BbhDyckLanguagesBj52correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT311 |
| ✓ | BbhDyckLanguagesBj53correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT361 |
| ✓ | BbhDyckLanguagesBj54correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 066 OUT12 167 |
| ✓ | BbhDyckLanguagesBj55correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT358 |
| ✓ | BbhDyckLanguagesBj56correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT354 |
| ✓ | BbhDyckLanguagesBj57correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT112 |
| ✓ | BbhDyckLanguagesBj58correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT577 |
| ✓ | BbhDyckLanguagesBj59correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT290 |
| × | BbhDyckLanguagesBj60correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("}")
Context: BbhDyckLanguagesBj60: expected }, got [{ [ [ [ [ ( ) ] ] ] ] }] |
Failed | IN984 OUT512 |
| ✓ | BbhDyckLanguagesBj61correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 010 OUT2 668 |
| ✓ | BbhDyckLanguagesBj62correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT882 |
| ✓ | BbhDyckLanguagesBj63correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT427 |
| ✓ | BbhDyckLanguagesBj64correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 162 |
| ✓ | BbhDyckLanguagesBj65correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT712 |
| ✓ | BbhDyckLanguagesBj66correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 028 OUT4 283 |
| ✓ | BbhDyckLanguagesBj67correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT3 836 |
| × | BbhDyckLanguagesBj68correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] )")
Context: BbhDyckLanguagesBj68: expected ] ), got [( [ ( ) ] )] |
Failed | IN977 OUT386 |
| ✓ | BbhDyckLanguagesBj69correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT2 388 |
| ✓ | BbhDyckLanguagesBj70correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT1 139 |
| ✓ | BbhDyckLanguagesBj71correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 029 OUT6 358 |
| ✓ | BbhDyckLanguagesBj72correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT470 |
| ✓ | BbhDyckLanguagesBj73correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 037 OUT6 235 |
| ✓ | BbhDyckLanguagesBj74correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 072 OUT2 819 |
| ✓ | BbhDyckLanguagesBj75correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 030 OUT2 456 |
| ✓ | BbhDyckLanguagesBj76correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT544 |
| ✓ | BbhDyckLanguagesBj77correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT427 |
| ✓ | BbhDyckLanguagesBj78correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 022 OUT3 130 |
| ✓ | BbhDyckLanguagesBj79correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT2 143 |
| ✓ | BbhDyckLanguagesBj80correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT837 |
| ✓ | BbhDyckLanguagesBj81correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 664 |
| ✓ | BbhDyckLanguagesBj82correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT517 |
Reasoning - BBH Dyck languages - close the brackets (set 3)
29 passed3 failed |
90.6% |
IN31 966 OUT39 703 | |
| ✓ | BbhDyckLanguagesBj100correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 258 |
| ✓ | BbhDyckLanguagesBj101correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT456 |
| ✓ | BbhDyckLanguagesBj102correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 047 OUT2 544 |
| ✓ | BbhDyckLanguagesBj103correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT242 |
| ✓ | BbhDyckLanguagesBj104correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT559 |
| ✓ | BbhDyckLanguagesBj105correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT380 |
| ✓ | BbhDyckLanguagesBj106correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 010 OUT2 993 |
| ✓ | BbhDyckLanguagesBj107correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT696 |
| × | BbhDyckLanguagesBj108correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> } >")
Context: BbhDyckLanguagesBj108: expected > } >, got [< { < ( ) > } >] |
Failed | IN978 OUT326 |
| ✓ | BbhDyckLanguagesBj109correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 066 OUT2 638 |
| ✓ | BbhDyckLanguagesBj110correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT279 |
| ✓ | BbhDyckLanguagesBj111correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 012 OUT2 603 |
| ✓ | BbhDyckLanguagesBj112correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 065 OUT2 529 |
| × | BbhDyckLanguagesBj113correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("]")
Context: BbhDyckLanguagesBj113: expected ], got [[ ( { } ) ]] |
Failed | IN978 OUT204 |
| ✓ | BbhDyckLanguagesBj114correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 334 |
| ✓ | BbhDyckLanguagesBj83correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT137 |
| ✓ | BbhDyckLanguagesBj84correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT499 |
| ✓ | BbhDyckLanguagesBj85correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT421 |
| × | BbhDyckLanguagesBj86correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj86: expected ), got [( ( ( ) ) )] |
Failed | IN978 OUT289 |
| ✓ | BbhDyckLanguagesBj87correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 031 OUT2 274 |
| ✓ | BbhDyckLanguagesBj88correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 059 OUT5 492 |
| ✓ | BbhDyckLanguagesBj89correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT435 |
| ✓ | BbhDyckLanguagesBj90correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT942 |
| ✓ | BbhDyckLanguagesBj91correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 604 |
| ✓ | BbhDyckLanguagesBj92correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT337 |
| ✓ | BbhDyckLanguagesBj93correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT757 |
| ✓ | BbhDyckLanguagesBj94correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 017 OUT1 497 |
| ✓ | BbhDyckLanguagesBj95correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 059 OUT3 925 |
| ✓ | BbhDyckLanguagesBj96correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT505 |
| ✓ | BbhDyckLanguagesBj97correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT185 |
| ✓ | BbhDyckLanguagesBj98correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT517 |
| ✓ | BbhDyckLanguagesBj99correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT846 |
Reasoning - BBH Dyck languages - close the brackets (set 4)
22 passed10 failed |
68.8% |
IN31 909 OUT53 509 | |
| × | BbhDyckLanguagesBj115correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] }")
Context: BbhDyckLanguagesBj115: expected ] }, got [{ [ [ ] [ ] ] }] |
Failed | IN979 OUT342 |
| ✓ | BbhDyckLanguagesBj116correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 005 OUT2 488 |
| ✓ | BbhDyckLanguagesBj117correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT2 566 |
| ✓ | BbhDyckLanguagesBj118correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT518 |
| ✓ | BbhDyckLanguagesBj119correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT447 |
| ✓ | BbhDyckLanguagesBj120correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 019 OUT3 121 |
| ✓ | BbhDyckLanguagesBj121correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 383 |
| ✓ | BbhDyckLanguagesBj122correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 042 OUT5 586 |
| × | BbhDyckLanguagesBj123correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ]")
Context: BbhDyckLanguagesBj123: expected > ], got [[ < [ ] > ]] |
Failed | IN977 OUT328 |
| ✓ | BbhDyckLanguagesBj124correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT915 |
| ✓ | BbhDyckLanguagesBj125correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 197 |
| ✓ | BbhDyckLanguagesBj126correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT658 |
| ✓ | BbhDyckLanguagesBj127correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 017 OUT1 246 |
| × | BbhDyckLanguagesBj128correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj128: expected >, got [< ( ) >] |
Failed | IN976 OUT111 |
| × | BbhDyckLanguagesBj129correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> } ]")
Context: BbhDyckLanguagesBj129: expected > } ], got [[ { < ( ) > } ]] |
Failed | IN978 OUT254 |
| ✓ | BbhDyckLanguagesBj130correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT560 |
| ✓ | BbhDyckLanguagesBj131correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT1 474 |
| × | BbhDyckLanguagesBj132correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBj132: expected ) >, got [< < > ( [ ] ) >] |
Failed | IN979 OUT319 |
| ✓ | BbhDyckLanguagesBj133correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 057 |
| ✓ | BbhDyckLanguagesBj134correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT702 |
| ✓ | BbhDyckLanguagesBj135correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 024 OUT7 188 |
| × | BbhDyckLanguagesBj136correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj136: expected ) ), got [( ( ( [ { } ] ) ) )] |
Failed | IN981 OUT509 |
| ✓ | BbhDyckLanguagesBj137correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT596 |
| ✓ | BbhDyckLanguagesBj138correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 713 |
| × | BbhDyckLanguagesBj139correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj139: expected ), got [""] |
Failed | IN1 050 OUT8 039 |
| ✓ | BbhDyckLanguagesBj140correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 010 OUT1 808 |
| × | BbhDyckLanguagesBj141correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj141: expected ) ), got [( ) ( ())] |
Failed | IN977 OUT375 |
| ✓ | BbhDyckLanguagesBj142correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT653 |
| ✓ | BbhDyckLanguagesBj143correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT535 |
| × | BbhDyckLanguagesBj144correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ) }")
Context: BbhDyckLanguagesBj144: expected ) ) }, got [{ ( ( ) ( { ( < ( ) > ) } ))}] |
Failed | IN986 OUT1 308 |
| × | BbhDyckLanguagesBj145correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") > ]")
Context: BbhDyckLanguagesBj145: expected ) > ], got [>)]] |
Failed | IN1 048 OUT5 021 |
| ✓ | BbhDyckLanguagesBj146correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT492 |
Reasoning - BBH Dyck languages - close the brackets (set 5)
25 passed7 failed |
78.1% |
IN31 812 OUT43 713 | |
| × | BbhDyckLanguagesBj147correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj147: expected >, got [< < > >] |
Failed | IN976 OUT221 |
| ✓ | BbhDyckLanguagesBj148correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 018 OUT2 892 |
| ✓ | BbhDyckLanguagesBj149correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT291 |
| ✓ | BbhDyckLanguagesBj150correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT352 |
| ✓ | BbhDyckLanguagesBj151correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT657 |
| × | BbhDyckLanguagesBj152correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("}")
Context: BbhDyckLanguagesBj152: expected }, got [< > }] |
Failed | IN982 OUT639 |
| ✓ | BbhDyckLanguagesBj153correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT2 068 |
| ✓ | BbhDyckLanguagesBj154correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT2 600 |
| ✓ | BbhDyckLanguagesBj155correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 011 OUT2 875 |
| ✓ | BbhDyckLanguagesBj156correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT494 |
| ✓ | BbhDyckLanguagesBj157correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT854 |
| ✓ | BbhDyckLanguagesBj158correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT393 |
| ✓ | BbhDyckLanguagesBj159correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT540 |
| ✓ | BbhDyckLanguagesBj160correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 020 OUT6 904 |
| × | BbhDyckLanguagesBj161correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ) }")
Context: BbhDyckLanguagesBj161: expected ) ) }, got [{(())}] |
Failed | IN976 OUT1 353 |
| ✓ | BbhDyckLanguagesBj162correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT651 |
| ✓ | BbhDyckLanguagesBj163correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT294 |
| × | BbhDyckLanguagesBj164correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> }")
Context: BbhDyckLanguagesBj164: expected > }, got [( ) { < > }] |
Failed | IN977 OUT232 |
| ✓ | BbhDyckLanguagesBj165correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 411 |
| × | BbhDyckLanguagesBj166correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> > }")
Context: BbhDyckLanguagesBj166: expected > > }, got [>>>}] |
Failed | IN982 OUT500 |
| ✓ | BbhDyckLanguagesBj167correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT2 002 |
| ✓ | BbhDyckLanguagesBj168correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT204 |
| × | BbhDyckLanguagesBj169correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ) }")
Context: BbhDyckLanguagesBj169: expected ] ) }, got [) }] |
Failed | IN988 OUT670 |
| ✓ | BbhDyckLanguagesBj170correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 023 OUT2 531 |
| ✓ | BbhDyckLanguagesBj171correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT2 530 |
| ✓ | BbhDyckLanguagesBj172correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT514 |
| ✓ | BbhDyckLanguagesBj173correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT1 002 |
| ✓ | BbhDyckLanguagesBj174correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 064 OUT3 979 |
| ✓ | BbhDyckLanguagesBj175correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT1 428 |
| ✓ | BbhDyckLanguagesBj176correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 305 |
| ✓ | BbhDyckLanguagesBj177correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT239 |
| × | BbhDyckLanguagesBj178correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> } ]")
Context: BbhDyckLanguagesBj178: expected > } ], got [>]}] |
Failed | IN998 OUT1 088 |
Reasoning - BBH Dyck languages - close the brackets (set 6)
24 passed6 failed |
80.0% |
IN29 940 OUT66 246 | |
| ✓ | BbhDyckLanguagesBj179correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT263 |
| ✓ | BbhDyckLanguagesBj180correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT481 |
| ✓ | BbhDyckLanguagesBj181correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT347 |
| ✓ | BbhDyckLanguagesBj182correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT209 |
| ✓ | BbhDyckLanguagesBj183correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT596 |
| × | BbhDyckLanguagesBj184correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj184: expected >, got [""] |
Failed | IN1 068 OUT12 411 |
| ✓ | BbhDyckLanguagesBj185correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT565 |
| ✓ | BbhDyckLanguagesBj186correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 036 OUT6 417 |
| × | BbhDyckLanguagesBj187correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ]")
Context: BbhDyckLanguagesBj187: expected ) ], got [[ () ]] |
Failed | IN975 OUT1 150 |
| ✓ | BbhDyckLanguagesBj188correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT151 |
| × | BbhDyckLanguagesBj189correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ]")
Context: BbhDyckLanguagesBj189: expected ) ], got [}]] |
Failed | IN1 051 OUT6 243 |
| ✓ | BbhDyckLanguagesBj190correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT2 205 |
| ✓ | BbhDyckLanguagesBj191correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT453 |
| ✓ | BbhDyckLanguagesBj192correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT406 |
| ✓ | BbhDyckLanguagesBj193correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT206 |
| ✓ | BbhDyckLanguagesBj194correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT1 083 |
| ✓ | BbhDyckLanguagesBj195correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 037 OUT6 507 |
| × | BbhDyckLanguagesBj196correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) >")
Context: BbhDyckLanguagesBj196: expected } ) >, got [< ( { } ) >] |
Failed | IN976 OUT128 |
| × | BbhDyckLanguagesBj197correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ) ]")
Context: BbhDyckLanguagesBj197: expected ] ) ], got [[ ( [] ) ]] |
Failed | IN976 OUT274 |
| ✓ | BbhDyckLanguagesBj198correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 046 OUT4 463 |
| ✓ | BbhDyckLanguagesBj199correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 021 OUT1 566 |
| × | BbhDyckLanguagesBj200correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj200: expected ), got [>>>}}]])] |
Failed | IN1 066 OUT10 874 |
| ✓ | BbhDyckLanguagesBj201correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT2 466 |
| ✓ | BbhDyckLanguagesBj202correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT227 |
| ✓ | BbhDyckLanguagesBj203correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT256 |
| ✓ | BbhDyckLanguagesBj204correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT2 275 |
| ✓ | BbhDyckLanguagesBj205correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 173 |
| ✓ | BbhDyckLanguagesBj206correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT807 |
| ✓ | BbhDyckLanguagesBj207correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 088 |
| ✓ | BbhDyckLanguagesBj208correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT956 |
Reasoning - BBH Dyck languages - close the brackets (set 7)
19 passed13 failed |
59.4% |
IN32 026 OUT64 637 | |
| × | BbhDyckLanguagesBk209correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} } )")
Context: BbhDyckLanguagesBk209: expected } } ), got [}] |
Failed | IN1 028 OUT6 256 |
| ✓ | BbhDyckLanguagesBk210correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT674 |
| × | BbhDyckLanguagesBk211correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] } )")
Context: BbhDyckLanguagesBk211: expected ] } ), got [{ [ { } ] }] |
Failed | IN978 OUT303 |
| × | BbhDyckLanguagesBk212correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBk212: expected } ), got [}] |
Failed | IN1 069 OUT9 750 |
| × | BbhDyckLanguagesBk213correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBk213: expected ) >, got [>] |
Failed | IN989 OUT709 |
| ✓ | BbhDyckLanguagesBk214correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT724 |
| × | BbhDyckLanguagesBk215correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") > }")
Context: BbhDyckLanguagesBk215: expected ) > }, got [>}] |
Failed | IN1 030 OUT3 362 |
| ✓ | BbhDyckLanguagesBk216correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT351 |
| × | BbhDyckLanguagesBk217correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ] ]")
Context: BbhDyckLanguagesBk217: expected > ] ], got [[ [ <> ] ]] |
Failed | IN976 OUT719 |
| × | BbhDyckLanguagesBk218correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBk218: expected ), got [] |
Failed | IN976 OUT116 |
| ✓ | BbhDyckLanguagesBk219correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 691 |
| ✓ | BbhDyckLanguagesBk220correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT606 |
| × | BbhDyckLanguagesBk221correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ] ]")
Context: BbhDyckLanguagesBk221: expected ) ] ], got []] |
Failed | IN1 004 OUT1 270 |
| ✓ | BbhDyckLanguagesBk222correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT993 |
| × | BbhDyckLanguagesBk223correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> > }")
Context: BbhDyckLanguagesBk223: expected > > }, got [{<<>>}] |
Failed | IN978 OUT516 |
| ✓ | BbhDyckLanguagesBk224correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 262 |
| ✓ | BbhDyckLanguagesBk225correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 066 OUT4 286 |
| ✓ | BbhDyckLanguagesBk226correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT2 299 |
| × | BbhDyckLanguagesBk227correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} } )")
Context: BbhDyckLanguagesBk227: expected } } ), got [}] |
Failed | IN1 070 OUT8 125 |
| × | BbhDyckLanguagesBk228correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBk228: expected ) >, got [< >] |
Failed | IN975 OUT666 |
| ✓ | BbhDyckLanguagesBk229correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 192 |
| ✓ | BbhDyckLanguagesBk230correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 401 |
| ✓ | BbhDyckLanguagesBk231correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT550 |
| ✓ | BbhDyckLanguagesBk232correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT658 |
| × | BbhDyckLanguagesBk233correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("}")
Context: BbhDyckLanguagesBk233: expected }, got [}}] |
Failed | IN992 OUT635 |
| ✓ | BbhDyckLanguagesBk234correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT990 |
| ✓ | BbhDyckLanguagesBk235correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 034 OUT8 990 |
| × | BbhDyckLanguagesBk236correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") } >")
Context: BbhDyckLanguagesBk236: expected ) } >, got [}>] |
Failed | IN990 OUT675 |
| ✓ | BbhDyckLanguagesBk237correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 070 OUT2 966 |
| ✓ | BbhDyckLanguagesBk238correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT298 |
| ✓ | BbhDyckLanguagesBk239correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT598 |
| ✓ | BbhDyckLanguagesBk240correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 006 |
Reasoning - BBH formal fallacies (set 1)
26 passed6 failed |
81.3% |
IN33 806 OUT31 075 | |
| × | BbhDyckLanguagesBk241correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBk241: expected ), got [] |
Failed | IN982 OUT809 |
| × | BbhDyckLanguagesBk242correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") }")
Context: BbhDyckLanguagesBk242: expected ) }, got [}] |
Failed | IN983 OUT596 |
| × | BbhDyckLanguagesBk243correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBk243: expected } ), got [}] |
Failed | IN999 OUT1 428 |
| × | BbhDyckLanguagesBk244correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) }")
Context: BbhDyckLanguagesBk244: expected > ) }, got [> }] |
Failed | IN976 OUT126 |
| × | BbhDyckLanguagesBk245correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) }")
Context: BbhDyckLanguagesBk245: expected } ) }, got [}}] |
Failed | IN998 OUT2 274 |
| ✓ | BbhDyckLanguagesBk246correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 019 OUT1 729 |
| × | BbhDyckLanguagesBk247correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) )")
Context: BbhDyckLanguagesBk247: expected > ) ), got [>] |
Failed | IN992 OUT1 519 |
| ✓ | BbhDyckLanguagesBk248correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT299 |
| ✓ | BbhDyckLanguagesBk249correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT532 |
| ✓ | BbhFormalFallaciesBk10decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT516 |
| ✓ | BbhFormalFallaciesBk11decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT741 |
| ✓ | BbhFormalFallaciesBk12decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT555 |
| ✓ | BbhFormalFallaciesBk13decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 109 |
| ✓ | BbhFormalFallaciesBk14decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 425 |
| ✓ | BbhFormalFallaciesBk15decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT931 |
| ✓ | BbhFormalFallaciesBk16decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT731 |
| ✓ | BbhFormalFallaciesBk17decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT518 |
| ✓ | BbhFormalFallaciesBk18decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 180 |
| ✓ | BbhFormalFallaciesBk19decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT709 |
| ✓ | BbhFormalFallaciesBk20decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 394 |
| ✓ | BbhFormalFallaciesBk21decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT631 |
| ✓ | BbhFormalFallaciesBk22decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT933 |
| ✓ | BbhFormalFallaciesBk23decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT867 |
| ✓ | BbhFormalFallaciesBk24decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 955 |
| ✓ | BbhFormalFallaciesBk25decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT727 |
| ✓ | BbhFormalFallaciesBk26decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT473 |
| ✓ | BbhFormalFallaciesBk27decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT869 |
| ✓ | BbhFormalFallaciesBk28decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT971 |
| ✓ | BbhFormalFallaciesBk29decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT744 |
| ✓ | BbhFormalFallaciesBk30decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT2 741 |
| ✓ | BbhFormalFallaciesBk8decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT397 |
| ✓ | BbhFormalFallaciesBk9decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT646 |
Reasoning - BBH formal fallacies (set 2)
32 passed |
100.0% |
IN34 415 OUT28 784 | |
| ✓ | BbhFormalFallaciesBk31decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT935 |
| ✓ | BbhFormalFallaciesBk32decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT1 022 |
| ✓ | BbhFormalFallaciesBk33decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT798 |
| ✓ | BbhFormalFallaciesBk34decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 490 |
| ✓ | BbhFormalFallaciesBk35decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT541 |
| ✓ | BbhFormalFallaciesBk36decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 558 |
| ✓ | BbhFormalFallaciesBk37decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 289 |
| ✓ | BbhFormalFallaciesBk38decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT733 |
| ✓ | BbhFormalFallaciesBk39decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 194 |
| ✓ | BbhFormalFallaciesBk40decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 363 |
| ✓ | BbhFormalFallaciesBk41decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT531 |
| ✓ | BbhFormalFallaciesBk42decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT806 |
| ✓ | BbhFormalFallaciesBk43decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT832 |
| ✓ | BbhFormalFallaciesBk44decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT748 |
| ✓ | BbhFormalFallaciesBk45decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT557 |
| ✓ | BbhFormalFallaciesBk46decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 188 |
| ✓ | BbhFormalFallaciesBk47decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT752 |
| ✓ | BbhFormalFallaciesBk48decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT530 |
| ✓ | BbhFormalFallaciesBk49decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 086 |
| ✓ | BbhFormalFallaciesBk50decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT948 |
| ✓ | BbhFormalFallaciesBk51decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT976 |
| ✓ | BbhFormalFallaciesBk52decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT484 |
| ✓ | BbhFormalFallaciesBk53decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT776 |
| ✓ | BbhFormalFallaciesBk54decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT539 |
| ✓ | BbhFormalFallaciesBk55decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT614 |
| ✓ | BbhFormalFallaciesBk56decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT546 |
| ✓ | BbhFormalFallaciesBk57decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT474 |
| ✓ | BbhFormalFallaciesBk58decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT2 087 |
| ✓ | BbhFormalFallaciesBk59decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT897 |
| ✓ | BbhFormalFallaciesBk60decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 010 |
| ✓ | BbhFormalFallaciesBk61decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT778 |
| ✓ | BbhFormalFallaciesBk62decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT702 |
Reasoning - BBH formal fallacies (set 3)
32 passed |
100.0% |
IN34 624 OUT32 033 | |
| ✓ | BbhFormalFallaciesBk63decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT683 |
| ✓ | BbhFormalFallaciesBk64decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT628 |
| ✓ | BbhFormalFallaciesBk65decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 264 |
| ✓ | BbhFormalFallaciesBk66decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT543 |
| ✓ | BbhFormalFallaciesBk67decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT728 |
| ✓ | BbhFormalFallaciesBk68decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 183 |
| ✓ | BbhFormalFallaciesBk69decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT3 524 |
| ✓ | BbhFormalFallaciesBk70decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT470 |
| ✓ | BbhFormalFallaciesBk71decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT656 |
| ✓ | BbhFormalFallaciesBk72decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT631 |
| ✓ | BbhFormalFallaciesBk73decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT619 |
| ✓ | BbhFormalFallaciesBk74decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT832 |
| ✓ | BbhFormalFallaciesBk75decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT984 |
| ✓ | BbhFormalFallaciesBk76decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 669 |
| ✓ | BbhFormalFallaciesBk77decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 134 |
| ✓ | BbhFormalFallaciesBk78decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 301 |
| ✓ | BbhFormalFallaciesBk79decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT576 |
| ✓ | BbhFormalFallaciesBk80decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT858 |
| ✓ | BbhFormalFallaciesBk81decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT2 068 |
| ✓ | BbhFormalFallaciesBk82decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT696 |
| ✓ | BbhFormalFallaciesBk83decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 067 |
| ✓ | BbhFormalFallaciesBk84decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT521 |
| ✓ | BbhFormalFallaciesBk85decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT774 |
| ✓ | BbhFormalFallaciesBk86decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 109 |
| ✓ | BbhFormalFallaciesBk87decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 417 |
| ✓ | BbhFormalFallaciesBk88decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT831 |
| ✓ | BbhFormalFallaciesBk89decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT710 |
| ✓ | BbhFormalFallaciesBk90decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT1 464 |
| ✓ | BbhFormalFallaciesBk91decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT982 |
| ✓ | BbhFormalFallaciesBk92decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT790 |
| ✓ | BbhFormalFallaciesBk93decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT605 |
| ✓ | BbhFormalFallaciesBk94decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT716 |
Reasoning - BBH formal fallacies (set 4)
30 passed |
100.0% |
IN32 188 OUT26 884 | |
| ✓ | BbhFormalFallaciesBk100decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT758 |
| ✓ | BbhFormalFallaciesBk101decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 436 |
| ✓ | BbhFormalFallaciesBk102decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT302 |
| ✓ | BbhFormalFallaciesBk103decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT514 |
| ✓ | BbhFormalFallaciesBk104decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT758 |
| ✓ | BbhFormalFallaciesBk105decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 088 |
| ✓ | BbhFormalFallaciesBk106decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT500 |
| ✓ | BbhFormalFallaciesBk107decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT898 |
| ✓ | BbhFormalFallaciesBk108decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 082 |
| ✓ | BbhFormalFallaciesBk109decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT447 |
| ✓ | BbhFormalFallaciesBk110decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 512 |
| ✓ | BbhFormalFallaciesBk111decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 172 |
| ✓ | BbhFormalFallaciesBk112decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT512 |
| ✓ | BbhFormalFallaciesBk113decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT986 |
| ✓ | BbhFormalFallaciesBk114decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT638 |
| ✓ | BbhFormalFallaciesBk115decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT737 |
| ✓ | BbhFormalFallaciesBk116decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT748 |
| ✓ | BbhFormalFallaciesBk117decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT456 |
| ✓ | BbhFormalFallaciesBk118decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT485 |
| ✓ | BbhFormalFallaciesBk119decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT613 |
| ✓ | BbhFormalFallaciesBk120decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT427 |
| ✓ | BbhFormalFallaciesBk121decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT518 |
| ✓ | BbhFormalFallaciesBk122decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT990 |
| ✓ | BbhFormalFallaciesBk123decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 029 |
| ✓ | BbhFormalFallaciesBk124decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT644 |
| ✓ | BbhFormalFallaciesBk95decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 407 |
| ✓ | BbhFormalFallaciesBk96decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT781 |
| ✓ | BbhFormalFallaciesBk97decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT3 536 |
| ✓ | BbhFormalFallaciesBk98decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT637 |
| ✓ | BbhFormalFallaciesBk99decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 273 |
Reasoning - BBH formal fallacies (set 5)
32 passed |
100.0% |
IN34 494 OUT27 262 | |
| ✓ | BbhFormalFallacies125BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT720 |
| ✓ | BbhFormalFallacies126BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT728 |
| ✓ | BbhFormalFallacies127BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT634 |
| ✓ | BbhFormalFallacies128BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT610 |
| ✓ | BbhFormalFallacies129BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT567 |
| ✓ | BbhFormalFallacies130BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT551 |
| ✓ | BbhFormalFallacies131BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT1 001 |
| ✓ | BbhFormalFallacies132BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT552 |
| ✓ | BbhFormalFallacies133BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT390 |
| ✓ | BbhFormalFallacies134BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT620 |
| ✓ | BbhFormalFallacies135BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT824 |
| ✓ | BbhFormalFallacies136BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT2 275 |
| ✓ | BbhFormalFallacies137BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT703 |
| ✓ | BbhFormalFallacies138BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT856 |
| ✓ | BbhFormalFallacies139BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT682 |
| ✓ | BbhFormalFallacies140BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT629 |
| ✓ | BbhFormalFallacies141BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 038 |
| ✓ | BbhFormalFallacies142BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT789 |
| ✓ | BbhFormalFallacies143BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT878 |
| ✓ | BbhFormalFallacies144BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT938 |
| ✓ | BbhFormalFallacies145BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT752 |
| ✓ | BbhFormalFallacies146BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT534 |
| ✓ | BbhFormalFallacies147BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT595 |
| ✓ | BbhFormalFallacies148BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 292 |
| ✓ | BbhFormalFallacies149BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 299 |
| ✓ | BbhFormalFallacies150BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 075 |
| ✓ | BbhFormalFallacies151BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT740 |
| ✓ | BbhFormalFallacies152BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT778 |
| ✓ | BbhFormalFallacies153BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT1 140 |
| ✓ | BbhFormalFallacies154BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT653 |
| ✓ | BbhFormalFallacies155BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT898 |
| ✓ | BbhFormalFallacies156BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 521 |
Reasoning - BBH formal fallacies (set 6)
32 passed |
100.0% |
IN34 489 OUT30 600 | |
| ✓ | BbhFormalFallacies157BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT685 |
| ✓ | BbhFormalFallacies158BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT1 079 |
| ✓ | BbhFormalFallacies159BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT1 388 |
| ✓ | BbhFormalFallacies160BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 345 |
| ✓ | BbhFormalFallacies161BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT635 |
| ✓ | BbhFormalFallacies162BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT837 |
| ✓ | BbhFormalFallacies163BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT821 |
| ✓ | BbhFormalFallacies164BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 741 |
| ✓ | BbhFormalFallacies165BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 008 |
| ✓ | BbhFormalFallacies166BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT904 |
| ✓ | BbhFormalFallacies167BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT732 |
| ✓ | BbhFormalFallacies168BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT617 |
| ✓ | BbhFormalFallacies169BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 265 |
| ✓ | BbhFormalFallacies170BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT533 |
| ✓ | BbhFormalFallacies171BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT1 524 |
| ✓ | BbhFormalFallacies172BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 169 |
| ✓ | BbhFormalFallacies173BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT551 |
| ✓ | BbhFormalFallacies174BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT573 |
| ✓ | BbhFormalFallacies175BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT491 |
| ✓ | BbhFormalFallacies176BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT724 |
| ✓ | BbhFormalFallacies177BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 036 |
| ✓ | BbhFormalFallacies178BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 137 |
| ✓ | BbhFormalFallacies179BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 321 |
| ✓ | BbhFormalFallacies180BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT476 |
| ✓ | BbhFormalFallacies181BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT841 |
| ✓ | BbhFormalFallacies182BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 240 |
| ✓ | BbhFormalFallacies183BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT973 |
| ✓ | BbhFormalFallacies184BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT548 |
| ✓ | BbhFormalFallacies185BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT501 |
| ✓ | BbhFormalFallacies186BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT457 |
| ✓ | BbhFormalFallacies187BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 229 |
| ✓ | BbhFormalFallacies188BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 219 |
Reasoning - BBH formal fallacies (set 7)
32 passed |
100.0% |
IN34 363 OUT28 327 | |
| ✓ | BbhFormalFallacies189BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT358 |
| ✓ | BbhFormalFallacies190BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT772 |
| ✓ | BbhFormalFallacies191BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT566 |
| ✓ | BbhFormalFallacies192BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT785 |
| ✓ | BbhFormalFallacies193BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT493 |
| ✓ | BbhFormalFallacies194BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT386 |
| ✓ | BbhFormalFallacies195BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT569 |
| ✓ | BbhFormalFallacies196BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 412 |
| ✓ | BbhFormalFallacies197BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT622 |
| ✓ | BbhFormalFallacies198BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT813 |
| ✓ | BbhFormalFallacies199BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 852 |
| ✓ | BbhFormalFallacies200BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT791 |
| ✓ | BbhFormalFallacies201BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 136 |
| ✓ | BbhFormalFallacies202BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 583 |
| ✓ | BbhFormalFallacies203BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT792 |
| ✓ | BbhFormalFallacies204BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT577 |
| ✓ | BbhFormalFallacies205BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT768 |
| ✓ | BbhFormalFallacies206BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 636 |
| ✓ | BbhFormalFallacies207BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT806 |
| ✓ | BbhFormalFallacies208BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT916 |
| ✓ | BbhFormalFallacies209BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT667 |
| ✓ | BbhFormalFallacies210BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT484 |
| ✓ | BbhFormalFallacies211BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 165 |
| ✓ | BbhFormalFallacies212BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT471 |
| ✓ | BbhFormalFallacies213BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 798 |
| ✓ | BbhFormalFallacies214BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT721 |
| ✓ | BbhFormalFallacies215BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT574 |
| ✓ | BbhFormalFallacies216BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 052 |
| ✓ | BbhFormalFallacies217BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 560 |
| ✓ | BbhFormalFallacies218BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 163 |
| ✓ | BbhFormalFallacies219BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT655 |
| ✓ | BbhFormalFallacies220BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT384 |
Reasoning - BBH formal fallacies (set 8)
28 passed1 failed |
96.6% |
IN31 240 OUT26 567 | |
| ✓ | BbhFormalFallacies221BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT593 |
| ✓ | BbhFormalFallacies222BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT923 |
| ✓ | BbhFormalFallacies223BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT1 111 |
| × | BbhFormalFallacies224BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("invalid")
Context: BbhFormalFallacies224BL: expected invalid, got [ANWSER: invalid] |
Failed | IN1 080 OUT714 |
| ✓ | BbhFormalFallacies225BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT607 |
| ✓ | BbhFormalFallacies226BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT1 225 |
| ✓ | BbhFormalFallacies227BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT922 |
| ✓ | BbhFormalFallacies228BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 582 |
| ✓ | BbhFormalFallacies229BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 006 |
| ✓ | BbhFormalFallacies230BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 629 |
| ✓ | BbhFormalFallacies231BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT932 |
| ✓ | BbhFormalFallacies232BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT563 |
| ✓ | BbhFormalFallacies233BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 005 |
| ✓ | BbhFormalFallacies234BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT1 543 |
| ✓ | BbhFormalFallacies235BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT375 |
| ✓ | BbhFormalFallacies236BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT577 |
| ✓ | BbhFormalFallacies237BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT735 |
| ✓ | BbhFormalFallacies238BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 024 |
| ✓ | BbhFormalFallacies239BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 230 |
| ✓ | BbhFormalFallacies240BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 157 |
| ✓ | BbhFormalFallacies241BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT400 |
| ✓ | BbhFormalFallacies242BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT997 |
| ✓ | BbhFormalFallacies243BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT990 |
| ✓ | BbhFormalFallacies244BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT705 |
| ✓ | BbhFormalFallacies245BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT461 |
| ✓ | BbhFormalFallacies246BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 996 |
| ✓ | BbhFormalFallacies247BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT472 |
| ✓ | BbhFormalFallacies248BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT589 |
| ✓ | BbhFormalFallacies249BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT504 |
Reasoning - BBH geometric shapes (SVG paths) (set 1)
26 passed7 failed |
78.8% |
IN36 670 OUT38 273 | |
| ✓ | BbhGeometricShapes10BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT813 |
| ✓ | BbhGeometricShapes11BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT742 |
| × | BbhGeometricShapes12BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes12BL: expected (K), got [(A)] |
Failed | IN1 103 OUT1 672 |
| × | BbhGeometricShapes13BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes13BL: expected (K), got [(A)] |
Failed | IN1 099 OUT1 033 |
| × | BbhGeometricShapes14BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes14BL: expected (K), got [(H)] |
Failed | IN1 096 OUT1 548 |
| ✓ | BbhGeometricShapes15BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 380 |
| ✓ | BbhGeometricShapes16BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 037 |
| ✓ | BbhGeometricShapes17BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 196 OUT1 503 |
| × | BbhGeometricShapes18BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes18BL: expected (K), got [(H)] |
Failed | IN1 112 OUT2 528 |
| ✓ | BbhGeometricShapes19BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 272 |
| ✓ | BbhGeometricShapes20BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT609 |
| ✓ | BbhGeometricShapes21BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT2 183 |
| ✓ | BbhGeometricShapes22BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 539 |
| ✓ | BbhGeometricShapes23BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 166 OUT873 |
| ✓ | BbhGeometricShapes24BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 311 |
| ✓ | BbhGeometricShapes25BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 918 |
| ✓ | BbhGeometricShapes26BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT534 |
| ✓ | BbhGeometricShapes27BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT725 |
| ✓ | BbhGeometricShapes28BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 068 |
| × | BbhGeometricShapes29BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes29BL: expected (K), got [(H)] |
Failed | IN1 084 OUT988 |
| ✓ | BbhGeometricShapes30BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT801 |
| ✓ | BbhGeometricShapes31BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT847 |
| ✓ | BbhGeometricShapes32BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 040 |
| × | BbhGeometricShapes33BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes33BL: expected (K), got [(A)] |
Failed | IN1 103 OUT2 131 |
| ✓ | BbhGeometricShapes34BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT951 |
| ✓ | BbhGeometricShapes35BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 224 |
| ✓ | BbhGeometricShapes36BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT730 |
| ✓ | BbhGeometricShapes37BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT583 |
| ✓ | BbhGeometricShapes38BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 125 |
| ✓ | BbhGeometricShapes39BLThis SVG path element <path d="M 2.27,59.23 L 52.45,67.18"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT334 |
| ✓ | BbhGeometricShapes40BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT650 |
| × | BbhGeometricShapes8BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes8BL: expected (K), got [(H)] |
Failed | IN1 084 OUT1 990 |
| ✓ | BbhGeometricShapes9BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT591 |
Reasoning - BBH geometric shapes (SVG paths) (set 2)
25 passed7 failed |
78.1% |
IN35 097 OUT36 429 | |
| ✓ | BbhGeometricShapes41Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 458 |
| ✓ | BbhGeometricShapes42Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 372 |
| ✓ | BbhGeometricShapes43BmThis SVG path element <path d="M 98.96,92.35 L 44.27,28.27"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT395 |
| ✓ | BbhGeometricShapes44Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT396 |
| × | BbhGeometricShapes45Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes45Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 392 |
| × | BbhGeometricShapes46Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes46Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 915 |
| ✓ | BbhGeometricShapes47Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 388 |
| ✓ | BbhGeometricShapes48Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT905 |
| ✓ | BbhGeometricShapes49Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 587 |
| ✓ | BbhGeometricShapes50Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 167 OUT1 342 |
| ✓ | BbhGeometricShapes51Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 028 |
| ✓ | BbhGeometricShapes52Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT833 |
| ✓ | BbhGeometricShapes53Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 106 |
| ✓ | BbhGeometricShapes54Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 487 |
| ✓ | BbhGeometricShapes55BmThis SVG path element <path d="M 32.95,20.92 L 31.65,35.07"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT189 |
| ✓ | BbhGeometricShapes56Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 291 |
| ✓ | BbhGeometricShapes57Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 446 |
| ✓ | BbhGeometricShapes58Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 908 |
| × | BbhGeometricShapes59Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes59Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT992 |
| ✓ | BbhGeometricShapes60Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 710 |
| ✓ | BbhGeometricShapes61Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT615 |
| ✓ | BbhGeometricShapes62Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 258 |
| ✓ | BbhGeometricShapes63Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT789 |
| ✓ | BbhGeometricShapes64Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT592 |
| × | BbhGeometricShapes65Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes65Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 470 |
| × | BbhGeometricShapes66Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes66Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 097 OUT2 009 |
| ✓ | BbhGeometricShapes67Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT541 |
| ✓ | BbhGeometricShapes68BmThis SVG path element <path d="M 11.79,16.93 L 56.17,80.16"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT296 |
| ✓ | BbhGeometricShapes69Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 345 |
| ✓ | BbhGeometricShapes70Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT822 |
| × | BbhGeometricShapes71Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes71Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 792 |
| × | BbhGeometricShapes72Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes72Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT760 |
Reasoning - BBH geometric shapes (SVG paths) (set 3)
25 passed7 failed |
78.1% |
IN35 038 OUT35 745 | |
| × | BbhGeometricShapes100Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(F)")
Context: BbhGeometricShapes100Bm: expected (F), got [b]
Values:
got = "b" |
Failed | IN1 131 OUT528 |
| ✓ | BbhGeometricShapes101Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT723 |
| ✓ | BbhGeometricShapes102Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 436 |
| ✓ | BbhGeometricShapes103Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 669 |
| × | BbhGeometricShapes104Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes104Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 098 OUT1 760 |
| ✓ | BbhGeometricShapes73BmThis SVG path element <path d="M 31.97,19.98 L 38.55,43.43"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT312 |
| ✓ | BbhGeometricShapes74BmThis SVG path element <path d="M 81.33,89.86 L 65.20,41.90"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT308 |
| ✓ | BbhGeometricShapes75BmThis SVG path element <path d="M 27.49,66.87 L 19.65,56.41"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT326 |
| ✓ | BbhGeometricShapes76Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 547 |
| ✓ | BbhGeometricShapes77Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 492 |
| ✓ | BbhGeometricShapes78Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 491 |
| ✓ | BbhGeometricShapes79BmThis SVG path element <path d="M 14.31,45.36 L 17.22,37.11"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT344 |
| × | BbhGeometricShapes80Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes80Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 097 OUT1 779 |
| ✓ | BbhGeometricShapes81Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT2 574 |
| ✓ | BbhGeometricShapes82Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 183 |
| ✓ | BbhGeometricShapes83Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT674 |
| ✓ | BbhGeometricShapes84Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT638 |
| ✓ | BbhGeometricShapes85BmThis SVG path element <path d="M 0.58,63.64 L 34.42,42.20"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT354 |
| ✓ | BbhGeometricShapes86Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 945 |
| × | BbhGeometricShapes87Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes87Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT2 015 |
| ✓ | BbhGeometricShapes88Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 600 |
| ✓ | BbhGeometricShapes89Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT694 |
| ✓ | BbhGeometricShapes90Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT964 |
| ✓ | BbhGeometricShapes91Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT2 028 |
| ✓ | BbhGeometricShapes92Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT583 |
| × | BbhGeometricShapes93Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes93Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT2 167 |
| ✓ | BbhGeometricShapes94Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT758 |
| × | BbhGeometricShapes95Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes95Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 173 |
| ✓ | BbhGeometricShapes96Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT702 |
| ✓ | BbhGeometricShapes97Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT478 |
| × | BbhGeometricShapes98Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes98Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT869 |
| ✓ | BbhGeometricShapes99Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT631 |
Reasoning - BBH geometric shapes (SVG paths) (set 4)
24 passed8 failed |
75.0% |
IN35 330 OUT32 606 | |
| ✓ | BbhGeometricShapes105Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 166 OUT806 |
| ✓ | BbhGeometricShapes106Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 584 |
| ✓ | BbhGeometricShapes107BmThis SVG path element <path d="M 22.00,62.00 L 30.00,74.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT393 |
| ✓ | BbhGeometricShapes108Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT900 |
| × | BbhGeometricShapes109Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes109Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 331 |
| × | BbhGeometricShapes110Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes110Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 303 |
| × | BbhGeometricShapes111Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes111Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT1 150 |
| ✓ | BbhGeometricShapes112Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 126 |
| ✓ | BbhGeometricShapes113Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 013 |
| ✓ | BbhGeometricShapes114Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT994 |
| ✓ | BbhGeometricShapes115Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT666 |
| ✓ | BbhGeometricShapes116Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT610 |
| ✓ | BbhGeometricShapes117Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 183 OUT1 276 |
| × | BbhGeometricShapes118Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes118Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 342 |
| ✓ | BbhGeometricShapes119BmThis SVG path element <path d="M 17.11,37.49 L 43.48,13.53"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT388 |
| × | BbhGeometricShapes120Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes120Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 112 OUT2 191 |
| ✓ | BbhGeometricShapes121Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT932 |
| ✓ | BbhGeometricShapes122Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT681 |
| ✓ | BbhGeometricShapes123BmThis SVG path element <path d="M 51.00,18.00 L 5.00,62.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT311 |
| × | BbhGeometricShapes124Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes124Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT1 161 |
| ✓ | BbhGeometricShapes125Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 191 |
| ✓ | BbhGeometricShapes126Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 088 |
| ✓ | BbhGeometricShapes127Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 503 |
| × | BbhGeometricShapes128Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes128Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 070 |
| ✓ | BbhGeometricShapes129Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 164 OUT1 140 |
| ✓ | BbhGeometricShapes130Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT974 |
| ✓ | BbhGeometricShapes131Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 209 OUT860 |
| ✓ | BbhGeometricShapes132BmThis SVG path element <path d="M 26.00,23.00 L 19.00,16.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT381 |
| ✓ | BbhGeometricShapes133Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT857 |
| × | BbhGeometricShapes134Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes134Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT1 919 |
| ✓ | BbhGeometricShapes135Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 086 |
| ✓ | BbhGeometricShapes136BmThis SVG path element <path d="M 46.92,78.61 L 46.94,45.83"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT379 |
Reasoning - BBH geometric shapes (SVG paths) (set 5)
26 passed5 failed |
83.9% |
IN34 220 OUT32 928 | |
| ✓ | BbhGeometricShapes137Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT901 |
| ✓ | BbhGeometricShapes138Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 687 |
| ✓ | BbhGeometricShapes139BmThis SVG path element <path d="M 58.33,52.18 L 34.28,31.10"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT359 |
| ✓ | BbhGeometricShapes140Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 304 |
| ✓ | BbhGeometricShapes141BmThis SVG path element <path d="M 38.35,49.41 L 31.18,9.15"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT310 |
| × | BbhGeometricShapes142Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes142Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 109 |
| ✓ | BbhGeometricShapes143Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT611 |
| ✓ | BbhGeometricShapes144Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT875 |
| × | BbhGeometricShapes145Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes145Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT1 375 |
| ✓ | BbhGeometricShapes146Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT592 |
| ✓ | BbhGeometricShapes147Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 368 |
| ✓ | BbhGeometricShapes148Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT453 |
| ✓ | BbhGeometricShapes149Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT969 |
| ✓ | BbhGeometricShapes150BmThis SVG path element <path d="M 90.41,82.38 L 45.53,97.84"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT315 |
| ✓ | BbhGeometricShapes151Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 199 |
| ✓ | BbhGeometricShapes152Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 177 |
| ✓ | BbhGeometricShapes153Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT566 |
| ✓ | BbhGeometricShapes154Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT700 |
| ✓ | BbhGeometricShapes155Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 264 |
| ✓ | BbhGeometricShapes156Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 087 |
| ✓ | BbhGeometricShapes157Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 625 |
| ✓ | BbhGeometricShapes158Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT741 |
| ✓ | BbhGeometricShapes159Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 255 |
| ✓ | BbhGeometricShapes160Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT792 |
| × | BbhGeometricShapes161Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes161Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 167 |
| ✓ | BbhGeometricShapes162Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 183 OUT1 362 |
| × | BbhGeometricShapes163Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes163Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 125 OUT2 177 |
| × | BbhGeometricShapes164Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes164Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 795 |
| ✓ | BbhGeometricShapes165Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT713 |
| ✓ | BbhGeometricShapes166Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT778 |
| ✓ | BbhGeometricShapes167Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 302 |
Reasoning - BBH geometric shapes (SVG paths) (set 6)
29 passed2 failed |
93.5% |
IN34 334 OUT40 318 | |
| ✓ | BbhGeometricShapes168Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT437 |
| ✓ | BbhGeometricShapes169Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 310 |
| ✓ | BbhGeometricShapes170Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 442 |
| ✓ | BbhGeometricShapes171Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT812 |
| ✓ | BbhGeometricShapes172Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT960 |
| ✓ | BbhGeometricShapes173Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT2 868 |
| ✓ | BbhGeometricShapes174Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 161 |
| ✓ | BbhGeometricShapes175Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 611 |
| ✓ | BbhGeometricShapes176Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 326 |
| ✓ | BbhGeometricShapes177Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 585 |
| ✓ | BbhGeometricShapes178Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT1 377 |
| ✓ | BbhGeometricShapes179Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT757 |
| ✓ | BbhGeometricShapes180Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 527 |
| × | BbhGeometricShapes181Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes181Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT1 722 |
| ✓ | BbhGeometricShapes182Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT3 143 |
| ✓ | BbhGeometricShapes183Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 692 |
| ✓ | BbhGeometricShapes184BmThis SVG path element <path d="M 25.00,38.00 L 89.00,58.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT309 |
| ✓ | BbhGeometricShapes185Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT673 |
| ✓ | BbhGeometricShapes186Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT562 |
| ✓ | BbhGeometricShapes187BmThis SVG path element <path d="M 51.78,63.32 L 1.41,97.49"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT182 |
| ✓ | BbhGeometricShapes188Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 524 |
| ✓ | BbhGeometricShapes189Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT662 |
| ✓ | BbhGeometricShapes190Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT2 085 |
| ✓ | BbhGeometricShapes191Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 650 |
| ✓ | BbhGeometricShapes192Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT975 |
| ✓ | BbhGeometricShapes193BmThis SVG path element <path d="M 41.63,86.23 L 86.15,83.17"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT220 |
| × | BbhGeometricShapes194Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes194Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 095 OUT2 432 |
| ✓ | BbhGeometricShapes195Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 781 |
| ✓ | BbhGeometricShapes196Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 499 |
| ✓ | BbhGeometricShapes197Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT617 |
| ✓ | BbhGeometricShapes198Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 417 |
Reasoning - BBH geometric shapes (SVG paths) (set 7)
23 passed9 failed |
71.9% |
IN35 172 OUT38 041 | |
| × | BbhGeometricShapes199BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes199BN: expected (K), got [(H)] |
Failed | IN1 086 OUT2 037 |
| × | BbhGeometricShapes200BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes200BN: expected (K), got [(A)] |
Failed | IN1 101 OUT1 578 |
| ✓ | BbhGeometricShapes201BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT964 |
| ✓ | BbhGeometricShapes202BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT632 |
| × | BbhGeometricShapes203BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes203BN: expected (K), got [(A)] |
Failed | IN1 099 OUT790 |
| ✓ | BbhGeometricShapes204BNThis SVG path element <path d="M 68.00,36.00 L 61.00,80.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT318 |
| ✓ | BbhGeometricShapes205BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 141 OUT1 239 |
| ✓ | BbhGeometricShapes206BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT505 |
| ✓ | BbhGeometricShapes207BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 491 |
| ✓ | BbhGeometricShapes208BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 475 |
| ✓ | BbhGeometricShapes209BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT738 |
| ✓ | BbhGeometricShapes210BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 341 |
| × | BbhGeometricShapes211BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes211BN: expected (K), got [(H)] |
Failed | IN1 125 OUT1 966 |
| ✓ | BbhGeometricShapes212BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 646 |
| ✓ | BbhGeometricShapes213BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 388 |
| ✓ | BbhGeometricShapes214BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT547 |
| ✓ | BbhGeometricShapes215BNThis SVG path element <path d="M 37.71,30.46 L 40.10,45.62"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT298 |
| ✓ | BbhGeometricShapes216BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 571 |
| ✓ | BbhGeometricShapes217BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 208 |
| ✓ | BbhGeometricShapes218BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 160 |
| ✓ | BbhGeometricShapes219BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT563 |
| × | BbhGeometricShapes220BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes220BN: expected (K), got [(A)] |
Failed | IN1 099 OUT1 043 |
| × | BbhGeometricShapes221BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes221BN: expected (K), got [(A)] |
Failed | IN1 103 OUT1 117 |
| ✓ | BbhGeometricShapes222BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 827 |
| × | BbhGeometricShapes223BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes223BN: expected (K), got [(H)] |
Failed | IN1 099 OUT1 912 |
| × | BbhGeometricShapes224BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes224BN: expected (K), got [(H)] |
Failed | IN1 086 OUT857 |
| ✓ | BbhGeometricShapes225BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 509 |
| × | BbhGeometricShapes226BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes226BN: expected (K), got [(H)] |
Failed | IN1 086 OUT2 327 |
| ✓ | BbhGeometricShapes227BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT772 |
| ✓ | BbhGeometricShapes228BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 199 |
| ✓ | BbhGeometricShapes229BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT504 |
| ✓ | BbhGeometricShapes230BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 519 |
Reasoning - BBH geometric shapes (SVG paths) (set 8)
29 passed3 failed |
90.6% |
IN33 751 OUT28 233 | |
| ✓ | BbhGeometricShapes231BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT604 |
| ✓ | BbhGeometricShapes232BNThis SVG path element <path d="M 93.43,69.32 L 68.06,88.58"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT371 |
| ✓ | BbhGeometricShapes233BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 438 |
| ✓ | BbhGeometricShapes234BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT744 |
| ✓ | BbhGeometricShapes235BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 065 |
| ✓ | BbhGeometricShapes236BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT2 471 |
| ✓ | BbhGeometricShapes237BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT460 |
| × | BbhGeometricShapes238BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes238BN: expected (K), got [(H)] |
Failed | IN1 086 OUT916 |
| ✓ | BbhGeometricShapes239BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT476 |
| ✓ | BbhGeometricShapes240BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT796 |
| ✓ | BbhGeometricShapes241BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 026 |
| × | BbhGeometricShapes242BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes242BN: expected (K), got [(A)] |
Failed | IN1 103 OUT1 923 |
| ✓ | BbhGeometricShapes243BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 589 |
| ✓ | BbhGeometricShapes244BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT618 |
| ✓ | BbhGeometricShapes245BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 274 |
| ✓ | BbhGeometricShapes246BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 293 |
| × | BbhGeometricShapes247BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes247BN: expected (K), got [(H)] |
Failed | IN1 095 OUT2 133 |
| ✓ | BbhGeometricShapes248BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 447 |
| ✓ | BbhGeometricShapes249BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 402 |
| ✓ | BbhHyperbaton10BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT708 |
| ✓ | BbhHyperbaton11BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT390 |
| ✓ | BbhHyperbaton12BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT546 |
| ✓ | BbhHyperbaton13BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT444 |
| ✓ | BbhHyperbaton14BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT222 |
| ✓ | BbhHyperbaton15BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT417 |
| ✓ | BbhHyperbaton16BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT322 |
| ✓ | BbhHyperbaton17BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT403 |
| ✓ | BbhHyperbaton18BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT386 |
| ✓ | BbhHyperbaton19BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT481 |
| ✓ | BbhHyperbaton20BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT985 |
| ✓ | BbhHyperbaton8BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT648 |
| ✓ | BbhHyperbaton9BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT235 |
Reasoning - BBH hyperbaton - adjective ordering (set 1)
32 passed |
100.0% |
IN31 506 OUT18 212 | |
| ✓ | BbhHyperbaton21BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT557 |
| ✓ | BbhHyperbaton22BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT454 |
| ✓ | BbhHyperbaton23BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT384 |
| ✓ | BbhHyperbaton24BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT627 |
| ✓ | BbhHyperbaton25BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT389 |
| ✓ | BbhHyperbaton26BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT372 |
| ✓ | BbhHyperbaton27BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT601 |
| ✓ | BbhHyperbaton28BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT725 |
| ✓ | BbhHyperbaton29BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT645 |
| ✓ | BbhHyperbaton30BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT407 |
| ✓ | BbhHyperbaton31BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT379 |
| ✓ | BbhHyperbaton32BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT227 |
| ✓ | BbhHyperbaton33BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT435 |
| ✓ | BbhHyperbaton34BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT510 |
| ✓ | BbhHyperbaton35BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT478 |
| ✓ | BbhHyperbaton36BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT603 |
| ✓ | BbhHyperbaton37BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT360 |
| ✓ | BbhHyperbaton38BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 012 |
| ✓ | BbhHyperbaton39BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT317 |
| ✓ | BbhHyperbaton40BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 046 |
| ✓ | BbhHyperbaton41BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT450 |
| ✓ | BbhHyperbaton42BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT335 |
| ✓ | BbhHyperbaton43BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT723 |
| ✓ | BbhHyperbaton44BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT382 |
| ✓ | BbhHyperbaton45BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT468 |
| ✓ | BbhHyperbaton46BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT353 |
| ✓ | BbhHyperbaton47BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 504 |
| ✓ | BbhHyperbaton48BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT433 |
| ✓ | BbhHyperbaton49BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 524 |
| ✓ | BbhHyperbaton50BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT463 |
| ✓ | BbhHyperbaton51BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT198 |
| ✓ | BbhHyperbaton52BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT851 |
Reasoning - BBH hyperbaton - adjective ordering (set 2)
32 passed |
100.0% |
IN31 524 OUT21 288 | |
| ✓ | BbhHyperbaton53BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT383 |
| ✓ | BbhHyperbaton54BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT915 |
| ✓ | BbhHyperbaton55BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT929 |
| ✓ | BbhHyperbaton56BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT416 |
| ✓ | BbhHyperbaton57BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT420 |
| ✓ | BbhHyperbaton58BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT982 |
| ✓ | BbhHyperbaton59BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT374 |
| ✓ | BbhHyperbaton60BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT432 |
| ✓ | BbhHyperbaton61BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT390 |
| ✓ | BbhHyperbaton62BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT309 |
| ✓ | BbhHyperbaton63BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT419 |
| ✓ | BbhHyperbaton64BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT370 |
| ✓ | BbhHyperbaton65BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT646 |
| ✓ | BbhHyperbaton66BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT865 |
| ✓ | BbhHyperbaton67BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 047 |
| ✓ | BbhHyperbaton68BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT515 |
| ✓ | BbhHyperbaton69BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT499 |
| ✓ | BbhHyperbaton70BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT296 |
| ✓ | BbhHyperbaton71BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 029 |
| ✓ | BbhHyperbaton72BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 279 |
| ✓ | BbhHyperbaton73BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT903 |
| ✓ | BbhHyperbaton74BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT623 |
| ✓ | BbhHyperbaton75BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT291 |
| ✓ | BbhHyperbaton76BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT357 |
| ✓ | BbhHyperbaton77BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT595 |
| ✓ | BbhHyperbaton78BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT742 |
| ✓ | BbhHyperbaton79BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 693 |
| ✓ | BbhHyperbaton80BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT422 |
| ✓ | BbhHyperbaton81BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT633 |
| ✓ | BbhHyperbaton82BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 141 |
| ✓ | BbhHyperbaton83BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT491 |
| ✓ | BbhHyperbaton84BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT882 |
Reasoning - BBH hyperbaton - adjective ordering (set 3)
30 passed |
100.0% |
IN29 554 OUT17 449 | |
| ✓ | BbhHyperbaton100BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT774 |
| ✓ | BbhHyperbaton101BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT955 |
| ✓ | BbhHyperbaton102BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 288 |
| ✓ | BbhHyperbaton103BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT843 |
| ✓ | BbhHyperbaton104BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT468 |
| ✓ | BbhHyperbaton105BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT383 |
| ✓ | BbhHyperbaton106BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 105 |
| ✓ | BbhHyperbaton107BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT413 |
| ✓ | BbhHyperbaton108BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT369 |
| ✓ | BbhHyperbaton109BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT384 |
| ✓ | BbhHyperbaton110BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT363 |
| ✓ | BbhHyperbaton111BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT571 |
| ✓ | BbhHyperbaton112BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT359 |
| ✓ | BbhHyperbaton113BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT384 |
| ✓ | BbhHyperbaton114BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT521 |
| ✓ | BbhHyperbaton85BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT593 |
| ✓ | BbhHyperbaton86BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT671 |
| ✓ | BbhHyperbaton87BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT522 |
| ✓ | BbhHyperbaton88BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 444 |
| ✓ | BbhHyperbaton89BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT367 |
| ✓ | BbhHyperbaton90BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT483 |
| ✓ | BbhHyperbaton91BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT635 |
| ✓ | BbhHyperbaton92BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT413 |
| ✓ | BbhHyperbaton93BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT438 |
| ✓ | BbhHyperbaton94BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT371 |
| ✓ | BbhHyperbaton95BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT421 |
| ✓ | BbhHyperbaton96BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT678 |
| ✓ | BbhHyperbaton97BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT541 |
| ✓ | BbhHyperbaton98BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT313 |
| ✓ | BbhHyperbaton99BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT379 |
Reasoning - BBH hyperbaton - adjective ordering (set 4)
32 passed |
100.0% |
IN31 490 OUT21 012 | |
| ✓ | BbhHyperbaton115BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 433 |
| ✓ | BbhHyperbaton116BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT422 |
| ✓ | BbhHyperbaton117BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT439 |
| ✓ | BbhHyperbaton118BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT411 |
| ✓ | BbhHyperbaton119BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT406 |
| ✓ | BbhHyperbaton120BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT201 |
| ✓ | BbhHyperbaton121BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT390 |
| ✓ | BbhHyperbaton122BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT415 |
| ✓ | BbhHyperbaton123BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT602 |
| ✓ | BbhHyperbaton124BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT729 |
| ✓ | BbhHyperbaton125BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT241 |
| ✓ | BbhHyperbaton126BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT703 |
| ✓ | BbhHyperbaton127BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT594 |
| ✓ | BbhHyperbaton128BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT225 |
| ✓ | BbhHyperbaton129BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT428 |
| ✓ | BbhHyperbaton130BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 591 |
| ✓ | BbhHyperbaton131BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT834 |
| ✓ | BbhHyperbaton132BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 187 |
| ✓ | BbhHyperbaton133BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT377 |
| ✓ | BbhHyperbaton134BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT577 |
| ✓ | BbhHyperbaton135BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT306 |
| ✓ | BbhHyperbaton136BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT399 |
| ✓ | BbhHyperbaton137BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT713 |
| ✓ | BbhHyperbaton138BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT418 |
| ✓ | BbhHyperbaton139BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 696 |
| ✓ | BbhHyperbaton140BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT384 |
| ✓ | BbhHyperbaton141BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 111 |
| ✓ | BbhHyperbaton142BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT626 |
| ✓ | BbhHyperbaton143BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT503 |
| ✓ | BbhHyperbaton144BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 198 |
| ✓ | BbhHyperbaton145BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT951 |
| ✓ | BbhHyperbaton146BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT502 |
Reasoning - BBH hyperbaton - adjective ordering (set 5)
32 passed |
100.0% |
IN31 512 OUT15 008 | |
| ✓ | BbhHyperbaton147BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT579 |
| ✓ | BbhHyperbaton148BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT358 |
| ✓ | BbhHyperbaton149BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT755 |
| ✓ | BbhHyperbaton150BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT322 |
| ✓ | BbhHyperbaton151BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT365 |
| ✓ | BbhHyperbaton152BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT557 |
| ✓ | BbhHyperbaton153BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT390 |
| ✓ | BbhHyperbaton154BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT421 |
| ✓ | BbhHyperbaton155BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 142 |
| ✓ | BbhHyperbaton156BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT361 |
| ✓ | BbhHyperbaton157BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT363 |
| ✓ | BbhHyperbaton158BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT574 |
| ✓ | BbhHyperbaton159BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT352 |
| ✓ | BbhHyperbaton160BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT195 |
| ✓ | BbhHyperbaton161BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT410 |
| ✓ | BbhHyperbaton162BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT783 |
| ✓ | BbhHyperbaton163BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT274 |
| ✓ | BbhHyperbaton164BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT665 |
| ✓ | BbhHyperbaton165BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT209 |
| ✓ | BbhHyperbaton166BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT465 |
| ✓ | BbhHyperbaton167BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT449 |
| ✓ | BbhHyperbaton168BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT332 |
| ✓ | BbhHyperbaton169BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT366 |
| ✓ | BbhHyperbaton170BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT383 |
| ✓ | BbhHyperbaton171BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT349 |
| ✓ | BbhHyperbaton172BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 030 |
| ✓ | BbhHyperbaton173BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT446 |
| ✓ | BbhHyperbaton174BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT374 |
| ✓ | BbhHyperbaton175BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT503 |
| ✓ | BbhHyperbaton176BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT354 |
| ✓ | BbhHyperbaton177BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT524 |
| ✓ | BbhHyperbaton178BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT358 |
Reasoning - BBH hyperbaton - adjective ordering (set 6)
32 passed |
100.0% |
IN31 500 OUT16 527 | |
| ✓ | BbhHyperbaton179BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 089 |
| ✓ | BbhHyperbaton180BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT958 |
| ✓ | BbhHyperbaton181BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT780 |
| ✓ | BbhHyperbaton182BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT431 |
| ✓ | BbhHyperbaton183BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT500 |
| ✓ | BbhHyperbaton184BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT567 |
| ✓ | BbhHyperbaton185BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT353 |
| ✓ | BbhHyperbaton186BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 427 |
| ✓ | BbhHyperbaton187BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT612 |
| ✓ | BbhHyperbaton188BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT370 |
| ✓ | BbhHyperbaton189BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT413 |
| ✓ | BbhHyperbaton190BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT271 |
| ✓ | BbhHyperbaton191BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT175 |
| ✓ | BbhHyperbaton192BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT329 |
| ✓ | BbhHyperbaton193BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT362 |
| ✓ | BbhHyperbaton194BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT334 |
| ✓ | BbhHyperbaton195BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT396 |
| ✓ | BbhHyperbaton196BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT421 |
| ✓ | BbhHyperbaton197BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT456 |
| ✓ | BbhHyperbaton198BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT363 |
| ✓ | BbhHyperbaton199BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT411 |
| ✓ | BbhHyperbaton200BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT384 |
| ✓ | BbhHyperbaton201BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT257 |
| ✓ | BbhHyperbaton202BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT414 |
| ✓ | BbhHyperbaton203BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT915 |
| ✓ | BbhHyperbaton204BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT670 |
| ✓ | BbhHyperbaton205BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT379 |
| ✓ | BbhHyperbaton206BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 195 |
| ✓ | BbhHyperbaton207BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT420 |
| ✓ | BbhHyperbaton208BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT349 |
| ✓ | BbhHyperbaton209BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT231 |
| ✓ | BbhHyperbaton210BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT295 |
Reasoning - BBH hyperbaton - adjective ordering (set 7)
32 passed |
100.0% |
IN31 486 OUT18 265 | |
| ✓ | BbhHyperbaton211BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT483 |
| ✓ | BbhHyperbaton212BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT773 |
| ✓ | BbhHyperbaton213BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT475 |
| ✓ | BbhHyperbaton214BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT447 |
| ✓ | BbhHyperbaton215BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 493 |
| ✓ | BbhHyperbaton216BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT397 |
| ✓ | BbhHyperbaton217BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 095 |
| ✓ | BbhHyperbaton218BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT481 |
| ✓ | BbhHyperbaton219BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 214 |
| ✓ | BbhHyperbaton220BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT252 |
| ✓ | BbhHyperbaton221BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT238 |
| ✓ | BbhHyperbaton222BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT451 |
| ✓ | BbhHyperbaton223BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 093 |
| ✓ | BbhHyperbaton224BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT861 |
| ✓ | BbhHyperbaton225BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT414 |
| ✓ | BbhHyperbaton226BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT772 |
| ✓ | BbhHyperbaton227BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT324 |
| ✓ | BbhHyperbaton228BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT400 |
| ✓ | BbhHyperbaton229BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 254 |
| ✓ | BbhHyperbaton230BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT484 |
| ✓ | BbhHyperbaton231BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT394 |
| ✓ | BbhHyperbaton232BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT351 |
| ✓ | BbhHyperbaton233BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT321 |
| ✓ | BbhHyperbaton234BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT377 |
| ✓ | BbhHyperbaton235BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT338 |
| ✓ | BbhHyperbaton236BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT153 |
| ✓ | BbhHyperbaton237BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT518 |
| ✓ | BbhHyperbaton238BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT337 |
| ✓ | BbhHyperbaton239BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT551 |
| ✓ | BbhHyperbaton240BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT508 |
| ✓ | BbhHyperbaton241BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT379 |
| ✓ | BbhHyperbaton242BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT637 |
Reasoning - BBH logic: boolean expressions, causal judgement, dates
21 passed3 failed |
87.5% |
IN25 895 OUT13 866 | |
| ✓ | BbhBooleanExpressionsBa0not ( True ) and ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT199 |
| ✓ | BbhBooleanExpressionsBa1True and not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT188 |
| ✓ | BbhBooleanExpressionsBa2not True or False or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT194 |
| ✓ | BbhBooleanExpressionsBa3False or not ( True ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT192 |
| × | BbhBooleanExpressionsBa4True or not False and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("True")
Context: BbhBooleanExpressionsBa4: expected True, got [False] |
Failed | IN964 OUT227 |
| ✓ | BbhBooleanExpressionsBa5False or not not not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT290 |
| ✓ | BbhBooleanExpressionsBa6not True and ( False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT268 |
| ✓ | BbhBooleanExpressionsBa7True and not False or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT219 |
| ✓ | BbhCausalJudgementBa0judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT368 |
| ✓ | BbhCausalJudgementBa1judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 350 OUT1 112 |
| ✓ | BbhCausalJudgementBa2judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 352 OUT767 |
| ✓ | BbhCausalJudgementBa3judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 345 OUT1 764 |
| ✓ | BbhCausalJudgementBa4judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT562 |
| ✓ | BbhCausalJudgementBa5judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT974 |
| ✓ | BbhCausalJudgementBa6judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT283 |
| ✓ | BbhCausalJudgementBa7judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT382 |
| ✓ | BbhDateUnderstandingBa0date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT214 |
| ✓ | BbhDateUnderstandingBa1date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT480 |
| × | BbhDateUnderstandingBa2date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhDateUnderstandingBa2: expected (B), got [(C)] |
Failed | IN1 079 OUT598 |
| ✓ | BbhDateUnderstandingBa3date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT2 083 |
| ✓ | BbhDateUnderstandingBa4date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT411 |
| ✓ | BbhDateUnderstandingBa5date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT424 |
| × | BbhDateUnderstandingBa6date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhDateUnderstandingBa6: expected (A), got [(D)] |
Failed | IN1 087 OUT1 242 |
| ✓ | BbhDateUnderstandingBa7Tomorrow is 11/12/2019. What is the date yesterday in MM/DD/YYYY?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT425 |
Reasoning - BBH logic: disambiguation, dyck languages, formal fallacies
20 passed4 failed |
83.3% |
IN24 625 OUT18 744 | |
| ✓ | BbhDisambiguationQaBa0resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT325 |
| ✓ | BbhDisambiguationQaBa1resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT717 |
| ✓ | BbhDisambiguationQaBa2resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT631 |
| × | BbhDisambiguationQaBa3resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa3: expected (C), got [(A)] |
Failed | IN1 028 OUT431 |
| ✓ | BbhDisambiguationQaBa4resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT600 |
| × | BbhDisambiguationQaBa5resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa5: expected (C), got [(A)] |
Failed | IN1 023 OUT886 |
| ✓ | BbhDisambiguationQaBa6resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT1 515 |
| ✓ | BbhDisambiguationQaBa7resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT787 |
| × | BbhDyckLanguagesBa0correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ]")
Context: BbhDyckLanguagesBa0: expected ] ], got [[[ ]]] |
Failed | IN975 OUT232 |
| ✓ | BbhDyckLanguagesBa1correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT190 |
| ✓ | BbhDyckLanguagesBa2correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 038 OUT2 099 |
| ✓ | BbhDyckLanguagesBa3correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT533 |
| ✓ | BbhDyckLanguagesBa4correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT742 |
| ✓ | BbhDyckLanguagesBa5correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT755 |
| ✓ | BbhDyckLanguagesBa6correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 127 |
| × | BbhDyckLanguagesBa7correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ) >")
Context: BbhDyckLanguagesBa7: expected ) ) >, got [)))>] |
Failed | IN982 OUT489 |
| ✓ | BbhFormalFallaciesBa0decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT727 |
| ✓ | BbhFormalFallaciesBa1decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT741 |
| ✓ | BbhFormalFallaciesBa2decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 310 |
| ✓ | BbhFormalFallaciesBa3decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT792 |
| ✓ | BbhFormalFallaciesBa4decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT843 |
| ✓ | BbhFormalFallaciesBa5decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT1 193 |
| ✓ | BbhFormalFallaciesBa6decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT453 |
| ✓ | BbhFormalFallaciesBa7decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT626 |
Reasoning - BBH logic: geometric shapes, hyperbaton, logical deduction
22 passed2 failed |
91.7% |
IN25 466 OUT22 136 | |
| ✓ | BbhGeometricShapesBa0identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT966 |
| ✓ | BbhGeometricShapesBa1identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT697 |
| × | BbhGeometricShapesBa2identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapesBa2: expected (K), got [(H)] |
Failed | IN1 085 OUT3 374 |
| ✓ | BbhGeometricShapesBa3This SVG path element <path d="M 12.64,60.32 L 16.36,27.35"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT303 |
| × | BbhGeometricShapesBa4identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapesBa4: expected (K), got [(H)] |
Failed | IN1 112 OUT1 961 |
| ✓ | BbhGeometricShapesBa5identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 175 |
| ✓ | BbhGeometricShapesBa6identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT606 |
| ✓ | BbhGeometricShapesBa7identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT1 462 |
| ✓ | BbhHyperbatonBa0Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT409 |
| ✓ | BbhHyperbatonBa1Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT451 |
| ✓ | BbhHyperbatonBa2Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT545 |
| ✓ | BbhHyperbatonBa3Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT692 |
| ✓ | BbhHyperbatonBa4Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT431 |
| ✓ | BbhHyperbatonBa5Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT877 |
| ✓ | BbhHyperbatonBa6Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT263 |
| ✓ | BbhHyperbatonBa7Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT370 |
| ✓ | BbhLogicalDeductionFiveBa0order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT553 |
| ✓ | BbhLogicalDeductionFiveBa1order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 825 |
| ✓ | BbhLogicalDeductionFiveBa2order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT613 |
| ✓ | BbhLogicalDeductionFiveBa3order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT620 |
| ✓ | BbhLogicalDeductionFiveBa4order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT824 |
| ✓ | BbhLogicalDeductionFiveBa5order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 163 |
| ✓ | BbhLogicalDeductionFiveBa6order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT943 |
| ✓ | BbhLogicalDeductionFiveBa7order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 013 |
Reasoning - BBH logic: logical deduction, movie recommendation
24 passed |
100.0% |
IN25 511 OUT16 782 | |
| ✓ | BbhLogicalDeductionSeven0order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT835 |
| ✓ | BbhLogicalDeductionSeven1order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 358 |
| ✓ | BbhLogicalDeductionSeven2order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT883 |
| ✓ | BbhLogicalDeductionSeven3order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 185 |
| ✓ | BbhLogicalDeductionSeven4order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT807 |
| ✓ | BbhLogicalDeductionSeven5order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT1 092 |
| ✓ | BbhLogicalDeductionSeven6order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT730 |
| ✓ | BbhLogicalDeductionSeven7order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 142 OUT820 |
| ✓ | BbhLogicalDeductionThree0order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT340 |
| ✓ | BbhLogicalDeductionThree1order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT220 |
| ✓ | BbhLogicalDeductionThree2order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT253 |
| ✓ | BbhLogicalDeductionThree3order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT327 |
| ✓ | BbhLogicalDeductionThree4order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT290 |
| ✓ | BbhLogicalDeductionThree5order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT267 |
| ✓ | BbhLogicalDeductionThree6order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT328 |
| ✓ | BbhLogicalDeductionThree7order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT181 |
| ✓ | BbhMovieRec0Find a movie similar to Batman, The Mask, The Fugitive, Pretty Woman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 411 |
| ✓ | BbhMovieRec1pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT686 |
| ✓ | BbhMovieRec2pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT655 |
| ✓ | BbhMovieRec3pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT730 |
| ✓ | BbhMovieRec4Find a movie similar to Braveheart, The Mask, The Fugitive, Batman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 694 |
| ✓ | BbhMovieRec5pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT585 |
| ✓ | BbhMovieRec6pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT494 |
| ✓ | BbhMovieRec7pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT611 |
Reasoning - BBH logic: multistep arithmetic, navigation, object counting
24 passed |
100.0% |
IN23 808 OUT6 424 | |
| ✓ | BbhMultistepArith0((-1 + 2 + 9 * 5) - (-2 + -4 + -4 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT367 |
| ✓ | BbhMultistepArith1((-9 * -5 - 6 + -2) - (-8 - -6 * -3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT445 |
| ✓ | BbhMultistepArith2((3 * -3 * 6 + -5) - (-2 + -7 - 7 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT354 |
| ✓ | BbhMultistepArith3((6 * -6 * 8 * 1) * (-1 * 7 * -6 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT366 |
| ✓ | BbhMultistepArith4((-6 - -4 + 9 + 0) + (1 + -4 - -9 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT447 |
| ✓ | BbhMultistepArith5((-6 - 4 * 2 - 6) + (1 + -2 * 1 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT370 |
| ✓ | BbhMultistepArith6((1 - 0 + 1 - 4) - (-3 * 1 - -6 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT352 |
| ✓ | BbhMultistepArith7((1 + 7 * -9 + -5) + (3 + -5 * 2 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT420 |
| ✓ | BbhNavigate0does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT498 |
| ✓ | BbhNavigate1does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT280 |
| ✓ | BbhNavigate2does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT435 |
| ✓ | BbhNavigate3does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT123 |
| ✓ | BbhNavigate4does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT445 |
| ✓ | BbhNavigate5does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT159 |
| ✓ | BbhNavigate6does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT431 |
| ✓ | BbhNavigate7does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT155 |
| ✓ | BbhObjectCount0count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT167 |
| ✓ | BbhObjectCount1count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT142 |
| ✓ | BbhObjectCount2I have a fridge, a chair, and a microwave. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT41 |
| ✓ | BbhObjectCount3count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT140 |
| ✓ | BbhObjectCount4count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT75 |
| ✓ | BbhObjectCount5I have a fish, a bear, and a frog. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT41 |
| ✓ | BbhObjectCount6I have a car, and a toaster. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT33 |
| ✓ | BbhObjectCount7count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT138 |
Reasoning - BBH logic: object tracking, web of lies, word sorting
24 passed |
100.0% |
IN24 430 OUT9 025 | |
| ✓ | BbhTrackThree0track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT310 |
| ✓ | BbhTrackThree1track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT352 |
| ✓ | BbhTrackThree2track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT318 |
| ✓ | BbhTrackThree3track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT289 |
| ✓ | BbhTrackThree4track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT299 |
| ✓ | BbhTrackThree5track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT323 |
| ✓ | BbhTrackThree6track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT366 |
| ✓ | BbhTrackThree7track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT311 |
| ✓ | BbhWebLies0evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT301 |
| ✓ | BbhWebLies1evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT294 |
| ✓ | BbhWebLies2evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT290 |
| ✓ | BbhWebLies3evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT208 |
| ✓ | BbhWebLies4evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT218 |
| ✓ | BbhWebLies5evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT291 |
| ✓ | BbhWebLies6evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT300 |
| ✓ | BbhWebLies7evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT225 |
| ✓ | BbhWordSort0Sort the following words alphabetically: List: syndrome therefrom1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT93 |
| ✓ | BbhWordSort1sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 270 |
| ✓ | BbhWordSort2sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT383 |
| ✓ | BbhWordSort3sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT344 |
| ✓ | BbhWordSort4sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT113 |
| ✓ | BbhWordSort5sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT931 |
| ✓ | BbhWordSort6sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT806 |
| ✓ | BbhWordSort7sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT390 |
Reasoning - BBH logic: penguins table, colored objects, ruin names
23 passed1 failed |
95.8% |
IN26 008 OUT8 563 | |
| ✓ | BbhColoredObjects0reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT268 |
| ✓ | BbhColoredObjects1reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT109 |
| ✓ | BbhColoredObjects2reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT176 |
| ✓ | BbhColoredObjects3reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT334 |
| ✓ | BbhColoredObjects4reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT157 |
| ✓ | BbhColoredObjects5reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT339 |
| ✓ | BbhColoredObjects6reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT95 |
| ✓ | BbhColoredObjects7reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT58 |
| ✓ | BbhPenguins0answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 217 OUT206 |
| ✓ | BbhPenguins1answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT177 |
| ✓ | BbhPenguins2answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT236 |
| ✓ | BbhPenguins3answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT309 |
| ✓ | BbhPenguins4answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT250 |
| ✓ | BbhPenguins5answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 207 OUT123 |
| ✓ | BbhPenguins6answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 226 OUT250 |
| ✓ | BbhPenguins7answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 202 OUT142 |
| ✓ | BbhRuinNames0pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT561 |
| ✓ | BbhRuinNames1pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT317 |
| ✓ | BbhRuinNames2pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT261 |
| ✓ | BbhRuinNames3pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT787 |
| × | BbhRuinNames4pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames4: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT1 983 |
| ✓ | BbhRuinNames5pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT710 |
| ✓ | BbhRuinNames6pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT298 |
| ✓ | BbhRuinNames7pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT417 |
Reasoning - BBH logic: temporal sequences, object tracking
24 passed |
100.0% |
IN27 227 OUT14 929 | |
| ✓ | BbhTemporal0find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT539 |
| ✓ | BbhTemporal1find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT249 |
| ✓ | BbhTemporal2find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT465 |
| ✓ | BbhTemporal3find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT357 |
| ✓ | BbhTemporal4find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT516 |
| ✓ | BbhTemporal5find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT618 |
| ✓ | BbhTemporal6find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT405 |
| ✓ | BbhTemporal7find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT603 |
| ✓ | BbhTrackFive0track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT588 |
| ✓ | BbhTrackFive1track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT484 |
| ✓ | BbhTrackFive2track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT843 |
| ✓ | BbhTrackFive3track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT654 |
| ✓ | BbhTrackFive4track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT565 |
| ✓ | BbhTrackFive5track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT561 |
| ✓ | BbhTrackFive6track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT751 |
| ✓ | BbhTrackFive7track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT462 |
| ✓ | BbhTrackSeven0track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT902 |
| ✓ | BbhTrackSeven1track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT595 |
| ✓ | BbhTrackSeven2track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT753 |
| ✓ | BbhTrackSeven3track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT595 |
| ✓ | BbhTrackSeven4track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT972 |
| ✓ | BbhTrackSeven5track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT962 |
| ✓ | BbhTrackSeven6track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT755 |
| ✓ | BbhTrackSeven7track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT735 |
Reasoning - BBH logic: translation errors, snarks, sports
20 passed4 failed |
83.3% |
IN25 261 OUT12 793 | |
| ✓ | BbhSalientTransErr0identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 188 OUT566 |
| × | BbhSalientTransErr1identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhSalientTransErr1: expected (A), got [(D)] |
Failed | IN1 188 OUT511 |
| × | BbhSalientTransErr2identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(F)")
Context: BbhSalientTransErr2: expected (F), got [(D)] |
Failed | IN1 185 OUT1 101 |
| × | BbhSalientTransErr3identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(F)")
Context: BbhSalientTransErr3: expected (F), got [(D)] |
Failed | IN1 182 OUT676 |
| ✓ | BbhSalientTransErr4identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 179 OUT1 184 |
| ✓ | BbhSalientTransErr5identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 191 OUT991 |
| ✓ | BbhSalientTransErr6identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT638 |
| ✓ | BbhSalientTransErr7identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 166 OUT1 195 |
| ✓ | BbhSnarks0Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT555 |
| ✓ | BbhSnarks1Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT374 |
| ✓ | BbhSnarks2Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT235 |
| ✓ | BbhSnarks3Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT785 |
| ✓ | BbhSnarks4Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT539 |
| ✓ | BbhSnarks5Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT288 |
| ✓ | BbhSnarks6Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT323 |
| ✓ | BbhSnarks7Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT412 |
| × | BbhSports0Is the following sentence plausible? "Elias Lindholm beat the buzzer."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("no")
Context: BbhSports0: expected no, got [Yes] |
Failed | IN971 OUT302 |
| ✓ | BbhSports1judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT184 |
| ✓ | BbhSports2judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN974 OUT284 |
| ✓ | BbhSports3judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN978 OUT369 |
| ✓ | BbhSports4judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT564 |
| ✓ | BbhSports5judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN978 OUT258 |
| ✓ | BbhSports6Is the following sentence plausible? "David Silva took a throw in."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT196 |
| ✓ | BbhSports7judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT263 |
Reasoning - BBH logical deduction - order 3 objects (set 1)
32 passed |
100.0% |
IN33 651 OUT9 562 | |
| ✓ | BbhLogicalDeductionThree21BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT326 |
| ✓ | BbhLogicalDeductionThree22BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT203 |
| ✓ | BbhLogicalDeductionThree23BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT267 |
| ✓ | BbhLogicalDeductionThree24BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT217 |
| ✓ | BbhLogicalDeductionThree25BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT312 |
| ✓ | BbhLogicalDeductionThree26BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT325 |
| ✓ | BbhLogicalDeductionThree27BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT683 |
| ✓ | BbhLogicalDeductionThree28BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT360 |
| ✓ | BbhLogicalDeductionThree29BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT227 |
| ✓ | BbhLogicalDeductionThree30BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT230 |
| ✓ | BbhLogicalDeductionThree31BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT195 |
| ✓ | BbhLogicalDeductionThree32BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT503 |
| ✓ | BbhLogicalDeductionThree33BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT332 |
| ✓ | BbhLogicalDeductionThree34BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT374 |
| ✓ | BbhLogicalDeductionThree35BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT291 |
| ✓ | BbhLogicalDeductionThree36BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT203 |
| ✓ | BbhLogicalDeductionThree37BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT256 |
| ✓ | BbhLogicalDeductionThree38BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT337 |
| ✓ | BbhLogicalDeductionThree39BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT347 |
| ✓ | BbhLogicalDeductionThree40BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT273 |
| ✓ | BbhLogicalDeductionThree41BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT210 |
| ✓ | BbhLogicalDeductionThree42BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT282 |
| ✓ | BbhLogicalDeductionThree43BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT274 |
| ✓ | BbhLogicalDeductionThree44BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT244 |
| ✓ | BbhLogicalDeductionThree45BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT222 |
| ✓ | BbhLogicalDeductionThree46BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT515 |
| ✓ | BbhLogicalDeductionThree47BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT175 |
| ✓ | BbhLogicalDeductionThree48BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT359 |
| ✓ | BbhLogicalDeductionThree49BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT300 |
| ✓ | BbhLogicalDeductionThree50BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT235 |
| ✓ | BbhLogicalDeductionThree51BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT274 |
| ✓ | BbhLogicalDeductionThree52BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT211 |
Reasoning - BBH logical deduction - order 3 objects (set 2)
32 passed |
100.0% |
IN33 500 OUT9 482 | |
| ✓ | BbhLogicalDeductionThree53BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT784 |
| ✓ | BbhLogicalDeductionThree54BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT300 |
| ✓ | BbhLogicalDeductionThree55BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT253 |
| ✓ | BbhLogicalDeductionThree56BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT192 |
| ✓ | BbhLogicalDeductionThree57BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT282 |
| ✓ | BbhLogicalDeductionThree58BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT312 |
| ✓ | BbhLogicalDeductionThree59BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT462 |
| ✓ | BbhLogicalDeductionThree60BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT374 |
| ✓ | BbhLogicalDeductionThree61BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT266 |
| ✓ | BbhLogicalDeductionThree62BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT295 |
| ✓ | BbhLogicalDeductionThree63BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT339 |
| ✓ | BbhLogicalDeductionThree64BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT283 |
| ✓ | BbhLogicalDeductionThree65BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT598 |
| ✓ | BbhLogicalDeductionThree66BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT258 |
| ✓ | BbhLogicalDeductionThree67BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT197 |
| ✓ | BbhLogicalDeductionThree68BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT271 |
| ✓ | BbhLogicalDeductionThree69BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT249 |
| ✓ | BbhLogicalDeductionThree70BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT273 |
| ✓ | BbhLogicalDeductionThree71BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT184 |
| ✓ | BbhLogicalDeductionThree72BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT285 |
| ✓ | BbhLogicalDeductionThree73BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT278 |
| ✓ | BbhLogicalDeductionThree74BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT316 |
| ✓ | BbhLogicalDeductionThree75BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT224 |
| ✓ | BbhLogicalDeductionThree76BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT208 |
| ✓ | BbhLogicalDeductionThree77BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT289 |
| ✓ | BbhLogicalDeductionThree78BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT274 |
| ✓ | BbhLogicalDeductionThree79BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT197 |
| ✓ | BbhLogicalDeductionThree80BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT243 |
| ✓ | BbhLogicalDeductionThree81BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT322 |
| ✓ | BbhLogicalDeductionThree82BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT292 |
| ✓ | BbhLogicalDeductionThree83BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT222 |
| ✓ | BbhLogicalDeductionThree84BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT160 |
Reasoning - BBH logical deduction - order 3 objects (set 3)
32 passed |
100.0% |
IN33 574 OUT9 582 | |
| ✓ | BbhLogicalDeductionThree100BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT270 |
| ✓ | BbhLogicalDeductionThree101BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT313 |
| ✓ | BbhLogicalDeductionThree102BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT254 |
| ✓ | BbhLogicalDeductionThree103BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT262 |
| ✓ | BbhLogicalDeductionThree104BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT283 |
| ✓ | BbhLogicalDeductionThree105BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT195 |
| ✓ | BbhLogicalDeductionThree106BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT302 |
| ✓ | BbhLogicalDeductionThree107BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT310 |
| ✓ | BbhLogicalDeductionThree108BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT283 |
| ✓ | BbhLogicalDeductionThree109BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT275 |
| ✓ | BbhLogicalDeductionThree110BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT363 |
| ✓ | BbhLogicalDeductionThree111BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT239 |
| ✓ | BbhLogicalDeductionThree112BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT265 |
| ✓ | BbhLogicalDeductionThree113BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT276 |
| ✓ | BbhLogicalDeductionThree114BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT185 |
| ✓ | BbhLogicalDeductionThree115BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT221 |
| ✓ | BbhLogicalDeductionThree116BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT546 |
| ✓ | BbhLogicalDeductionThree85BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT225 |
| ✓ | BbhLogicalDeductionThree86BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT286 |
| ✓ | BbhLogicalDeductionThree87BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT172 |
| ✓ | BbhLogicalDeductionThree88BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT733 |
| ✓ | BbhLogicalDeductionThree89BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT242 |
| ✓ | BbhLogicalDeductionThree90BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT207 |
| ✓ | BbhLogicalDeductionThree91BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT258 |
| ✓ | BbhLogicalDeductionThree92BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT200 |
| ✓ | BbhLogicalDeductionThree93BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT403 |
| ✓ | BbhLogicalDeductionThree94BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT380 |
| ✓ | BbhLogicalDeductionThree95BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT217 |
| ✓ | BbhLogicalDeductionThree96BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT597 |
| ✓ | BbhLogicalDeductionThree97BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT313 |
| ✓ | BbhLogicalDeductionThree98BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT323 |
| ✓ | BbhLogicalDeductionThree99BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT184 |
Reasoning - BBH logical deduction - order 3 objects (set 4)
32 passed |
100.0% |
IN33 622 OUT8 738 | |
| ✓ | BbhLogicalDeductionThree117BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT265 |
| ✓ | BbhLogicalDeductionThree118BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT310 |
| ✓ | BbhLogicalDeductionThree119BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT318 |
| ✓ | BbhLogicalDeductionThree120BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT215 |
| ✓ | BbhLogicalDeductionThree121BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT331 |
| ✓ | BbhLogicalDeductionThree122BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT298 |
| ✓ | BbhLogicalDeductionThree123BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT352 |
| ✓ | BbhLogicalDeductionThree124BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT275 |
| ✓ | BbhLogicalDeductionThree125BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT325 |
| ✓ | BbhLogicalDeductionThree126BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT288 |
| ✓ | BbhLogicalDeductionThree127BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT305 |
| ✓ | BbhLogicalDeductionThree128BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT250 |
| ✓ | BbhLogicalDeductionThree129BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT169 |
| ✓ | BbhLogicalDeductionThree130BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT305 |
| ✓ | BbhLogicalDeductionThree131BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT332 |
| ✓ | BbhLogicalDeductionThree132BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT238 |
| ✓ | BbhLogicalDeductionThree133BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT281 |
| ✓ | BbhLogicalDeductionThree134BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT258 |
| ✓ | BbhLogicalDeductionThree135BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT250 |
| ✓ | BbhLogicalDeductionThree136BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT349 |
| ✓ | BbhLogicalDeductionThree137BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT316 |
| ✓ | BbhLogicalDeductionThree138BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT192 |
| ✓ | BbhLogicalDeductionThree139BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT298 |
| ✓ | BbhLogicalDeductionThree140BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT271 |
| ✓ | BbhLogicalDeductionThree141BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT292 |
| ✓ | BbhLogicalDeductionThree142BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT224 |
| ✓ | BbhLogicalDeductionThree143BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT197 |
| ✓ | BbhLogicalDeductionThree144BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT265 |
| ✓ | BbhLogicalDeductionThree145BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT210 |
| ✓ | BbhLogicalDeductionThree146BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT296 |
| ✓ | BbhLogicalDeductionThree147BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT230 |
| ✓ | BbhLogicalDeductionThree148BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT233 |
Reasoning - BBH logical deduction - order 3 objects (set 5)
30 passed |
100.0% |
IN31 460 OUT9 622 | |
| ✓ | BbhLogicalDeductionThree149BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT210 |
| ✓ | BbhLogicalDeductionThree150BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT239 |
| ✓ | BbhLogicalDeductionThree151BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT235 |
| ✓ | BbhLogicalDeductionThree152BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT348 |
| ✓ | BbhLogicalDeductionThree153BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT261 |
| ✓ | BbhLogicalDeductionThree154BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT456 |
| ✓ | BbhLogicalDeductionThree155BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 063 |
| ✓ | BbhLogicalDeductionThree156BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT229 |
| ✓ | BbhLogicalDeductionThree157BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT216 |
| ✓ | BbhLogicalDeductionThree158BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT224 |
| ✓ | BbhLogicalDeductionThree159BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT338 |
| ✓ | BbhLogicalDeductionThree160BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT457 |
| ✓ | BbhLogicalDeductionThree161BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT202 |
| ✓ | BbhLogicalDeductionThree162BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT257 |
| ✓ | BbhLogicalDeductionThree163BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT308 |
| ✓ | BbhLogicalDeductionThree164BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT294 |
| ✓ | BbhLogicalDeductionThree165BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT245 |
| ✓ | BbhLogicalDeductionThree166BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT272 |
| ✓ | BbhLogicalDeductionThree167BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT189 |
| ✓ | BbhLogicalDeductionThree168BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT261 |
| ✓ | BbhLogicalDeductionThree169BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT236 |
| ✓ | BbhLogicalDeductionThree170BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT236 |
| ✓ | BbhLogicalDeductionThree171BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT327 |
| ✓ | BbhLogicalDeductionThree172BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT478 |
| ✓ | BbhLogicalDeductionThree173BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT445 |
| ✓ | BbhLogicalDeductionThree174BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT186 |
| ✓ | BbhLogicalDeductionThree175BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT229 |
| ✓ | BbhLogicalDeductionThree176BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT467 |
| ✓ | BbhLogicalDeductionThree177BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT401 |
| ✓ | BbhLogicalDeductionThree178BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT313 |
Reasoning - BBH logical deduction - order 3 objects (set 6)
32 passed |
100.0% |
IN33 519 OUT9 005 | |
| ✓ | BbhLogicalDeductionThreeObjects179BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT457 |
| ✓ | BbhLogicalDeductionThreeObjects180BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT234 |
| ✓ | BbhLogicalDeductionThreeObjects181BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT334 |
| ✓ | BbhLogicalDeductionThreeObjects182BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT213 |
| ✓ | BbhLogicalDeductionThreeObjects183BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT219 |
| ✓ | BbhLogicalDeductionThreeObjects184BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT234 |
| ✓ | BbhLogicalDeductionThreeObjects185BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT207 |
| ✓ | BbhLogicalDeductionThreeObjects186BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT356 |
| ✓ | BbhLogicalDeductionThreeObjects187BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT286 |
| ✓ | BbhLogicalDeductionThreeObjects188BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT306 |
| ✓ | BbhLogicalDeductionThreeObjects189BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT234 |
| ✓ | BbhLogicalDeductionThreeObjects190BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT260 |
| ✓ | BbhLogicalDeductionThreeObjects191BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT200 |
| ✓ | BbhLogicalDeductionThreeObjects192BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT209 |
| ✓ | BbhLogicalDeductionThreeObjects193BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT198 |
| ✓ | BbhLogicalDeductionThreeObjects194BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT340 |
| ✓ | BbhLogicalDeductionThreeObjects195BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT267 |
| ✓ | BbhLogicalDeductionThreeObjects196BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT564 |
| ✓ | BbhLogicalDeductionThreeObjects197BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT252 |
| ✓ | BbhLogicalDeductionThreeObjects198BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT217 |
| ✓ | BbhLogicalDeductionThreeObjects199BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT195 |
| ✓ | BbhLogicalDeductionThreeObjects200BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT309 |
| ✓ | BbhLogicalDeductionThreeObjects201BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT264 |
| ✓ | BbhLogicalDeductionThreeObjects202BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT349 |
| ✓ | BbhLogicalDeductionThreeObjects203BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT265 |
| ✓ | BbhLogicalDeductionThreeObjects204BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT246 |
| ✓ | BbhLogicalDeductionThreeObjects205BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT259 |
| ✓ | BbhLogicalDeductionThreeObjects206BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT284 |
| ✓ | BbhLogicalDeductionThreeObjects207BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT226 |
| ✓ | BbhLogicalDeductionThreeObjects208BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT339 |
| ✓ | BbhLogicalDeductionThreeObjects209BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT283 |
| ✓ | BbhLogicalDeductionThreeObjects210BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT399 |
Reasoning - BBH logical deduction - order 3 objects (set 7)
32 passed |
100.0% |
IN33 552 OUT8 992 | |
| ✓ | BbhLogicalDeductionThreeObjects211BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT251 |
| ✓ | BbhLogicalDeductionThreeObjects212BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT260 |
| ✓ | BbhLogicalDeductionThreeObjects213BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT320 |
| ✓ | BbhLogicalDeductionThreeObjects214BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT355 |
| ✓ | BbhLogicalDeductionThreeObjects215BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT218 |
| ✓ | BbhLogicalDeductionThreeObjects216BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT215 |
| ✓ | BbhLogicalDeductionThreeObjects217BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT324 |
| ✓ | BbhLogicalDeductionThreeObjects218BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT251 |
| ✓ | BbhLogicalDeductionThreeObjects219BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT267 |
| ✓ | BbhLogicalDeductionThreeObjects220BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT329 |
| ✓ | BbhLogicalDeductionThreeObjects221BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT177 |
| ✓ | BbhLogicalDeductionThreeObjects222BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT280 |
| ✓ | BbhLogicalDeductionThreeObjects223BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT193 |
| ✓ | BbhLogicalDeductionThreeObjects224BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT176 |
| ✓ | BbhLogicalDeductionThreeObjects225BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT426 |
| ✓ | BbhLogicalDeductionThreeObjects226BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT459 |
| ✓ | BbhLogicalDeductionThreeObjects227BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT259 |
| ✓ | BbhLogicalDeductionThreeObjects228BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT227 |
| ✓ | BbhLogicalDeductionThreeObjects229BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT297 |
| ✓ | BbhLogicalDeductionThreeObjects230BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT168 |
| ✓ | BbhLogicalDeductionThreeObjects231BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT298 |
| ✓ | BbhLogicalDeductionThreeObjects232BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT278 |
| ✓ | BbhLogicalDeductionThreeObjects233BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT347 |
| ✓ | BbhLogicalDeductionThreeObjects234BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT232 |
| ✓ | BbhLogicalDeductionThreeObjects235BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT295 |
| ✓ | BbhLogicalDeductionThreeObjects236BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT357 |
| ✓ | BbhLogicalDeductionThreeObjects237BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT190 |
| ✓ | BbhLogicalDeductionThreeObjects238BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT300 |
| ✓ | BbhLogicalDeductionThreeObjects239BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT241 |
| ✓ | BbhLogicalDeductionThreeObjects240BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT393 |
| ✓ | BbhLogicalDeductionThreeObjects241BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT318 |
| ✓ | BbhLogicalDeductionThreeObjects242BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT291 |
Reasoning - BBH logical deduction - order 5 objects (set 1)
30 passed |
100.0% |
IN31 910 OUT21 447 | |
| ✓ | BbhHyperbaton243BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT962 |
| ✓ | BbhHyperbaton244BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 258 |
| ✓ | BbhHyperbaton245BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT340 |
| ✓ | BbhHyperbaton246BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT517 |
| ✓ | BbhHyperbaton247BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT418 |
| ✓ | BbhHyperbaton248BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT350 |
| ✓ | BbhHyperbaton249BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 298 |
| ✓ | BbhLogicalDeductionFiveObjects10BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT436 |
| ✓ | BbhLogicalDeductionFiveObjects11BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT676 |
| ✓ | BbhLogicalDeductionFiveObjects12BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT634 |
| ✓ | BbhLogicalDeductionFiveObjects13BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 213 |
| ✓ | BbhLogicalDeductionFiveObjects14BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT373 |
| ✓ | BbhLogicalDeductionFiveObjects15BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT418 |
| ✓ | BbhLogicalDeductionFiveObjects16BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT925 |
| ✓ | BbhLogicalDeductionFiveObjects17BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT489 |
| ✓ | BbhLogicalDeductionFiveObjects18BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT238 |
| ✓ | BbhLogicalDeductionFiveObjects19BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT810 |
| ✓ | BbhLogicalDeductionFiveObjects20BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 465 |
| ✓ | BbhLogicalDeductionFiveObjects21BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 737 |
| ✓ | BbhLogicalDeductionFiveObjects22BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT451 |
| ✓ | BbhLogicalDeductionFiveObjects23BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT808 |
| ✓ | BbhLogicalDeductionFiveObjects24BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT669 |
| ✓ | BbhLogicalDeductionFiveObjects25BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT562 |
| ✓ | BbhLogicalDeductionFiveObjects26BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT391 |
| ✓ | BbhLogicalDeductionFiveObjects27BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT717 |
| ✓ | BbhLogicalDeductionFiveObjects28BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT404 |
| ✓ | BbhLogicalDeductionFiveObjects29BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT509 |
| ✓ | BbhLogicalDeductionFiveObjects30BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT383 |
| ✓ | BbhLogicalDeductionFiveObjects8BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 132 |
| ✓ | BbhLogicalDeductionFiveObjects9BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT864 |
Reasoning - BBH logical deduction - order 5 objects (set 2)
32 passed |
100.0% |
IN34 949 OUT25 175 | |
| ✓ | BbhLogicalDeductionFive189BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT540 |
| ✓ | BbhLogicalDeductionFive190BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT639 |
| ✓ | BbhLogicalDeductionFive191BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT623 |
| ✓ | BbhLogicalDeductionFive192BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT922 |
| ✓ | BbhLogicalDeductionFive193BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT706 |
| ✓ | BbhLogicalDeductionFive194BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT484 |
| ✓ | BbhLogicalDeductionFive195BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT312 |
| ✓ | BbhLogicalDeductionFive196BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT657 |
| ✓ | BbhLogicalDeductionFive197BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT560 |
| ✓ | BbhLogicalDeductionFive198BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT629 |
| ✓ | BbhLogicalDeductionFive199BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT2 174 |
| ✓ | BbhLogicalDeductionFive200BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT422 |
| ✓ | BbhLogicalDeductionFive201BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 350 |
| ✓ | BbhLogicalDeductionFive202BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 614 |
| ✓ | BbhLogicalDeductionFive203BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 216 |
| ✓ | BbhLogicalDeductionFive204BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT561 |
| ✓ | BbhLogicalDeductionFive205BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT775 |
| ✓ | BbhLogicalDeductionFive206BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT760 |
| ✓ | BbhLogicalDeductionFive207BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT588 |
| ✓ | BbhLogicalDeductionFive208BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT874 |
| ✓ | BbhLogicalDeductionFive209BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT655 |
| ✓ | BbhLogicalDeductionFive210BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT579 |
| ✓ | BbhLogicalDeductionFive211BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT931 |
| ✓ | BbhLogicalDeductionFive212BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT544 |
| ✓ | BbhLogicalDeductionFive213BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT881 |
| ✓ | BbhLogicalDeductionFive214BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT494 |
| ✓ | BbhLogicalDeductionFive215BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT2 041 |
| ✓ | BbhLogicalDeductionFive216BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT536 |
| ✓ | BbhLogicalDeductionFive217BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT759 |
| ✓ | BbhLogicalDeductionFive218BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT407 |
| ✓ | BbhLogicalDeductionFive219BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT602 |
| ✓ | BbhLogicalDeductionFive220BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT340 |
Reasoning - BBH logical deduction - order 5 objects (set 3)
29 passed |
100.0% |
IN31 763 OUT18 479 | |
| ✓ | BbhLogicalDeductionFive221BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT781 |
| ✓ | BbhLogicalDeductionFive222BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT576 |
| ✓ | BbhLogicalDeductionFive223BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT914 |
| ✓ | BbhLogicalDeductionFive224BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT649 |
| ✓ | BbhLogicalDeductionFive225BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT412 |
| ✓ | BbhLogicalDeductionFive226BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT619 |
| ✓ | BbhLogicalDeductionFive227BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 206 |
| ✓ | BbhLogicalDeductionFive228BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT742 |
| ✓ | BbhLogicalDeductionFive229BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT500 |
| ✓ | BbhLogicalDeductionFive230BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT387 |
| ✓ | BbhLogicalDeductionFive231BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT613 |
| ✓ | BbhLogicalDeductionFive232BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT287 |
| ✓ | BbhLogicalDeductionFive233BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 115 |
| ✓ | BbhLogicalDeductionFive234BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT525 |
| ✓ | BbhLogicalDeductionFive235BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT698 |
| ✓ | BbhLogicalDeductionFive236BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT674 |
| ✓ | BbhLogicalDeductionFive237BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT576 |
| ✓ | BbhLogicalDeductionFive238BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT548 |
| ✓ | BbhLogicalDeductionFive239BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT572 |
| ✓ | BbhLogicalDeductionFive240BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT459 |
| ✓ | BbhLogicalDeductionFive241BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT786 |
| ✓ | BbhLogicalDeductionFive242BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT528 |
| ✓ | BbhLogicalDeductionFive243BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT810 |
| ✓ | BbhLogicalDeductionFive244BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT405 |
| ✓ | BbhLogicalDeductionFive245BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT695 |
| ✓ | BbhLogicalDeductionFive246BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT1 038 |
| ✓ | BbhLogicalDeductionFive247BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT583 |
| ✓ | BbhLogicalDeductionFive248BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT389 |
| ✓ | BbhLogicalDeductionFive249BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT392 |
Reasoning - BBH logical deduction - order 5 objects (set 4)
32 passed |
100.0% |
IN35 110 OUT21 335 | |
| ✓ | BbhLogicalDeductionFiveObjects31BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT501 |
| ✓ | BbhLogicalDeductionFiveObjects32BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT386 |
| ✓ | BbhLogicalDeductionFiveObjects33BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT845 |
| ✓ | BbhLogicalDeductionFiveObjects34BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT792 |
| ✓ | BbhLogicalDeductionFiveObjects35BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT533 |
| ✓ | BbhLogicalDeductionFiveObjects36BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT413 |
| ✓ | BbhLogicalDeductionFiveObjects37BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT598 |
| ✓ | BbhLogicalDeductionFiveObjects38BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT651 |
| ✓ | BbhLogicalDeductionFiveObjects39BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT412 |
| ✓ | BbhLogicalDeductionFiveObjects40BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT887 |
| ✓ | BbhLogicalDeductionFiveObjects41BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 423 |
| ✓ | BbhLogicalDeductionFiveObjects42BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT424 |
| ✓ | BbhLogicalDeductionFiveObjects43BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT523 |
| ✓ | BbhLogicalDeductionFiveObjects44BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT937 |
| ✓ | BbhLogicalDeductionFiveObjects45BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT568 |
| ✓ | BbhLogicalDeductionFiveObjects46BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT614 |
| ✓ | BbhLogicalDeductionFiveObjects47BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 406 |
| ✓ | BbhLogicalDeductionFiveObjects48BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT362 |
| ✓ | BbhLogicalDeductionFiveObjects49BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT522 |
| ✓ | BbhLogicalDeductionFiveObjects50BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT731 |
| ✓ | BbhLogicalDeductionFiveObjects51BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT484 |
| ✓ | BbhLogicalDeductionFiveObjects52BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT380 |
| ✓ | BbhLogicalDeductionFiveObjects53BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 448 |
| ✓ | BbhLogicalDeductionFiveObjects54BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT693 |
| ✓ | BbhLogicalDeductionFiveObjects55BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT504 |
| ✓ | BbhLogicalDeductionFiveObjects56BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT528 |
| ✓ | BbhLogicalDeductionFiveObjects57BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT674 |
| ✓ | BbhLogicalDeductionFiveObjects58BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT523 |
| ✓ | BbhLogicalDeductionFiveObjects59BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT720 |
| ✓ | BbhLogicalDeductionFiveObjects60BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT463 |
| ✓ | BbhLogicalDeductionFiveObjects61BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT550 |
| ✓ | BbhLogicalDeductionFiveObjects62BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT840 |
Reasoning - BBH logical deduction - order 5 objects (set 5)
32 passed |
100.0% |
IN35 118 OUT25 405 | |
| ✓ | BbhLogicalDeductionFiveObjects63BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT536 |
| ✓ | BbhLogicalDeductionFiveObjects64BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 039 |
| ✓ | BbhLogicalDeductionFiveObjects65BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT714 |
| ✓ | BbhLogicalDeductionFiveObjects66BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT4 792 |
| ✓ | BbhLogicalDeductionFiveObjects67BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT748 |
| ✓ | BbhLogicalDeductionFiveObjects68BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT444 |
| ✓ | BbhLogicalDeductionFiveObjects69BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT345 |
| ✓ | BbhLogicalDeductionFiveObjects70BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT737 |
| ✓ | BbhLogicalDeductionFiveObjects71BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT587 |
| ✓ | BbhLogicalDeductionFiveObjects72BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT770 |
| ✓ | BbhLogicalDeductionFiveObjects73BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT483 |
| ✓ | BbhLogicalDeductionFiveObjects74BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT394 |
| ✓ | BbhLogicalDeductionFiveObjects75BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT604 |
| ✓ | BbhLogicalDeductionFiveObjects76BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT784 |
| ✓ | BbhLogicalDeductionFiveObjects77BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT941 |
| ✓ | BbhLogicalDeductionFiveObjects78BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT661 |
| ✓ | BbhLogicalDeductionFiveObjects79BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT389 |
| ✓ | BbhLogicalDeductionFiveObjects80BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT936 |
| ✓ | BbhLogicalDeductionFiveObjects81BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT514 |
| ✓ | BbhLogicalDeductionFiveObjects82BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 584 |
| ✓ | BbhLogicalDeductionFiveObjects83BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT334 |
| ✓ | BbhLogicalDeductionFiveObjects84BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT732 |
| ✓ | BbhLogicalDeductionFiveObjects85BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT717 |
| ✓ | BbhLogicalDeductionFiveObjects86BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT301 |
| ✓ | BbhLogicalDeductionFiveObjects87BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT604 |
| ✓ | BbhLogicalDeductionFiveObjects88BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 104 |
| ✓ | BbhLogicalDeductionFiveObjects89BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT634 |
| ✓ | BbhLogicalDeductionFiveObjects90BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT748 |
| ✓ | BbhLogicalDeductionFiveObjects91BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT555 |
| ✓ | BbhLogicalDeductionFiveObjects92BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT574 |
| ✓ | BbhLogicalDeductionFiveObjects93BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT550 |
| ✓ | BbhLogicalDeductionFiveObjects94BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT550 |
Reasoning - BBH logical deduction - order 5 objects (set 6)
32 passed |
100.0% |
IN35 052 OUT20 550 | |
| ✓ | BbhLogicalDeductionFiveObjects100BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT392 |
| ✓ | BbhLogicalDeductionFiveObjects101BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT452 |
| ✓ | BbhLogicalDeductionFiveObjects102BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT625 |
| ✓ | BbhLogicalDeductionFiveObjects103BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT793 |
| ✓ | BbhLogicalDeductionFiveObjects104BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT688 |
| ✓ | BbhLogicalDeductionFiveObjects105BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 332 |
| ✓ | BbhLogicalDeductionFiveObjects106BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT730 |
| ✓ | BbhLogicalDeductionFiveObjects107BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT314 |
| ✓ | BbhLogicalDeductionFiveObjects108BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT531 |
| ✓ | BbhLogicalDeductionFiveObjects109BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT829 |
| ✓ | BbhLogicalDeductionFiveObjects110BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 299 |
| ✓ | BbhLogicalDeductionFiveObjects111BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT562 |
| ✓ | BbhLogicalDeductionFiveObjects112BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT490 |
| ✓ | BbhLogicalDeductionFiveObjects113BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT607 |
| ✓ | BbhLogicalDeductionFiveObjects114BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT761 |
| ✓ | BbhLogicalDeductionFiveObjects115BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT500 |
| ✓ | BbhLogicalDeductionFiveObjects116BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT884 |
| ✓ | BbhLogicalDeductionFiveObjects117BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT584 |
| ✓ | BbhLogicalDeductionFiveObjects118BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT581 |
| ✓ | BbhLogicalDeductionFiveObjects119BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT554 |
| ✓ | BbhLogicalDeductionFiveObjects120BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT617 |
| ✓ | BbhLogicalDeductionFiveObjects121BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT459 |
| ✓ | BbhLogicalDeductionFiveObjects122BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT451 |
| ✓ | BbhLogicalDeductionFiveObjects123BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT589 |
| ✓ | BbhLogicalDeductionFiveObjects124BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 046 |
| ✓ | BbhLogicalDeductionFiveObjects125BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT470 |
| ✓ | BbhLogicalDeductionFiveObjects126BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT797 |
| ✓ | BbhLogicalDeductionFiveObjects95BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT557 |
| ✓ | BbhLogicalDeductionFiveObjects96BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT472 |
| ✓ | BbhLogicalDeductionFiveObjects97BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT793 |
| ✓ | BbhLogicalDeductionFiveObjects98BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT336 |
| ✓ | BbhLogicalDeductionFiveObjects99BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT455 |
Reasoning - BBH logical deduction - order 5 objects (set 7)
32 passed |
100.0% |
IN35 085 OUT24 060 | |
| ✓ | BbhLogicalDeductionFiveObjects127BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT504 |
| ✓ | BbhLogicalDeductionFiveObjects128BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT751 |
| ✓ | BbhLogicalDeductionFiveObjects129BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT560 |
| ✓ | BbhLogicalDeductionFiveObjects130BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 336 |
| ✓ | BbhLogicalDeductionFiveObjects131BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT848 |
| ✓ | BbhLogicalDeductionFiveObjects132BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT978 |
| ✓ | BbhLogicalDeductionFiveObjects133BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 354 |
| ✓ | BbhLogicalDeductionFiveObjects134BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT578 |
| ✓ | BbhLogicalDeductionFiveObjects135BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT600 |
| ✓ | BbhLogicalDeductionFiveObjects136BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 031 |
| ✓ | BbhLogicalDeductionFiveObjects137BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT895 |
| ✓ | BbhLogicalDeductionFiveObjects138BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 107 |
| ✓ | BbhLogicalDeductionFiveObjects139BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT506 |
| ✓ | BbhLogicalDeductionFiveObjects140BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT770 |
| ✓ | BbhLogicalDeductionFiveObjects141BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT550 |
| ✓ | BbhLogicalDeductionFiveObjects142BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT648 |
| ✓ | BbhLogicalDeductionFiveObjects143BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT421 |
| ✓ | BbhLogicalDeductionFiveObjects144BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT713 |
| ✓ | BbhLogicalDeductionFiveObjects145BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT552 |
| ✓ | BbhLogicalDeductionFiveObjects146BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT523 |
| ✓ | BbhLogicalDeductionFiveObjects147BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT811 |
| ✓ | BbhLogicalDeductionFiveObjects148BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT355 |
| ✓ | BbhLogicalDeductionFiveObjects149BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 173 |
| ✓ | BbhLogicalDeductionFiveObjects150BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 265 |
| ✓ | BbhLogicalDeductionFiveObjects151BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT439 |
| ✓ | BbhLogicalDeductionFiveObjects152BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT613 |
| ✓ | BbhLogicalDeductionFiveObjects153BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT611 |
| ✓ | BbhLogicalDeductionFiveObjects154BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT701 |
| ✓ | BbhLogicalDeductionFiveObjects155BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT597 |
| ✓ | BbhLogicalDeductionFiveObjects156BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT954 |
| ✓ | BbhLogicalDeductionFiveObjects157BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT730 |
| ✓ | BbhLogicalDeductionFiveObjects158BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT586 |
Reasoning - BBH logical deduction - order 5 objects (set 8)
29 passed1 failed |
96.7% |
IN32 775 OUT21 723 | |
| ✓ | BbhLogicalDeductionFiveObjects159BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT585 |
| ✓ | BbhLogicalDeductionFiveObjects160BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT553 |
| ✓ | BbhLogicalDeductionFiveObjects161BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT517 |
| ✓ | BbhLogicalDeductionFiveObjects162BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT749 |
| ✓ | BbhLogicalDeductionFiveObjects163BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT773 |
| ✓ | BbhLogicalDeductionFiveObjects164BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT999 |
| × | BbhLogicalDeductionFiveObjects165BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhLogicalDeductionFiveObjects165BP: expected (D), got [e]
Values:
got = "e" |
Failed | IN1 092 OUT1 388 |
| ✓ | BbhLogicalDeductionFiveObjects166BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT386 |
| ✓ | BbhLogicalDeductionFiveObjects167BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT361 |
| ✓ | BbhLogicalDeductionFiveObjects168BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 085 |
| ✓ | BbhLogicalDeductionFiveObjects169BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT732 |
| ✓ | BbhLogicalDeductionFiveObjects170BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT655 |
| ✓ | BbhLogicalDeductionFiveObjects171BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT591 |
| ✓ | BbhLogicalDeductionFiveObjects172BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT798 |
| ✓ | BbhLogicalDeductionFiveObjects173BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT587 |
| ✓ | BbhLogicalDeductionFiveObjects174BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT400 |
| ✓ | BbhLogicalDeductionFiveObjects175BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT848 |
| ✓ | BbhLogicalDeductionFiveObjects176BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT946 |
| ✓ | BbhLogicalDeductionFiveObjects177BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT929 |
| ✓ | BbhLogicalDeductionFiveObjects178BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT516 |
| ✓ | BbhLogicalDeductionFiveObjects179BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT629 |
| ✓ | BbhLogicalDeductionFiveObjects180BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 404 |
| ✓ | BbhLogicalDeductionFiveObjects181BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT522 |
| ✓ | BbhLogicalDeductionFiveObjects182BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT556 |
| ✓ | BbhLogicalDeductionFiveObjects183BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT964 |
| ✓ | BbhLogicalDeductionFiveObjects184BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT999 |
| ✓ | BbhLogicalDeductionFiveObjects185BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT795 |
| ✓ | BbhLogicalDeductionFiveObjects186BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT452 |
| ✓ | BbhLogicalDeductionFiveObjects187BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT452 |
| ✓ | BbhLogicalDeductionFiveObjects188BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT552 |
Reasoning - BBH logical deduction - order 7 objects (set 1)
33 passed |
100.0% |
IN37 746 OUT34 934 | |
| ✓ | BbhLogicalDeductionSeven10BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT858 |
| ✓ | BbhLogicalDeductionSeven11BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 421 |
| ✓ | BbhLogicalDeductionSeven12BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT745 |
| ✓ | BbhLogicalDeductionSeven13BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT812 |
| ✓ | BbhLogicalDeductionSeven14BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 069 |
| ✓ | BbhLogicalDeductionSeven15BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT926 |
| ✓ | BbhLogicalDeductionSeven16BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 248 |
| ✓ | BbhLogicalDeductionSeven17BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT932 |
| ✓ | BbhLogicalDeductionSeven18BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 465 |
| ✓ | BbhLogicalDeductionSeven19BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT933 |
| ✓ | BbhLogicalDeductionSeven20BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT899 |
| ✓ | BbhLogicalDeductionSeven21BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT857 |
| ✓ | BbhLogicalDeductionSeven22BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 073 |
| ✓ | BbhLogicalDeductionSeven23BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT849 |
| ✓ | BbhLogicalDeductionSeven24BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT964 |
| ✓ | BbhLogicalDeductionSeven25BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT975 |
| ✓ | BbhLogicalDeductionSeven26BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 907 |
| ✓ | BbhLogicalDeductionSeven27BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT808 |
| ✓ | BbhLogicalDeductionSeven28BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT611 |
| ✓ | BbhLogicalDeductionSeven29BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 233 |
| ✓ | BbhLogicalDeductionSeven30BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 281 |
| ✓ | BbhLogicalDeductionSeven31BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 084 OUT1 614 |
| ✓ | BbhLogicalDeductionSeven32BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 384 |
| ✓ | BbhLogicalDeductionSeven33BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT842 |
| ✓ | BbhLogicalDeductionSeven34BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT942 |
| ✓ | BbhLogicalDeductionSeven35BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT871 |
| ✓ | BbhLogicalDeductionSeven36BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT918 |
| ✓ | BbhLogicalDeductionSeven37BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT798 |
| ✓ | BbhLogicalDeductionSeven38BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT1 756 |
| ✓ | BbhLogicalDeductionSeven39BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 100 OUT1 054 |
| ✓ | BbhLogicalDeductionSeven40BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 037 |
| ✓ | BbhLogicalDeductionSeven8BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 136 OUT1 217 |
| ✓ | BbhLogicalDeductionSeven9BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT635 |
Reasoning - BBH logical deduction - order 7 objects (set 2)
33 passed |
100.0% |
IN37 433 OUT37 952 | |
| ✓ | BbhLogicalDeductionSeven41BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT628 |
| ✓ | BbhLogicalDeductionSeven42BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 135 OUT1 782 |
| ✓ | BbhLogicalDeductionSeven43BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT731 |
| ✓ | BbhLogicalDeductionSeven44BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 133 OUT2 623 |
| ✓ | BbhLogicalDeductionSeven45BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT578 |
| ✓ | BbhLogicalDeductionSeven46BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT1 139 |
| ✓ | BbhLogicalDeductionSeven47BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT862 |
| ✓ | BbhLogicalDeductionSeven48BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT581 |
| ✓ | BbhLogicalDeductionSeven49BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT1 589 |
| ✓ | BbhLogicalDeductionSeven50BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 458 |
| ✓ | BbhLogicalDeductionSeven51BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT874 |
| ✓ | BbhLogicalDeductionSeven52BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 093 OUT819 |
| ✓ | BbhLogicalDeductionSeven53BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT700 |
| ✓ | BbhLogicalDeductionSeven54BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT887 |
| ✓ | BbhLogicalDeductionSeven55BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT682 |
| ✓ | BbhLogicalDeductionSeven56BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 720 |
| ✓ | BbhLogicalDeductionSeven57BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 121 |
| ✓ | BbhLogicalDeductionSeven58BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 037 |
| ✓ | BbhLogicalDeductionSeven59BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 413 |
| ✓ | BbhLogicalDeductionSeven60BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 093 OUT1 089 |
| ✓ | BbhLogicalDeductionSeven61BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT978 |
| ✓ | BbhLogicalDeductionSeven62BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 123 |
| ✓ | BbhLogicalDeductionSeven63BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 363 |
| ✓ | BbhLogicalDeductionSeven64BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 084 OUT1 591 |
| ✓ | BbhLogicalDeductionSeven65BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 103 |
| ✓ | BbhLogicalDeductionSeven66BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT904 |
| ✓ | BbhLogicalDeductionSeven67BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT844 |
| ✓ | BbhLogicalDeductionSeven68BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT1 097 |
| ✓ | BbhLogicalDeductionSeven69BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT932 |
| ✓ | BbhLogicalDeductionSeven70BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT1 694 |
| ✓ | BbhLogicalDeductionSeven71BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT785 |
| ✓ | BbhLogicalDeductionSeven72BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 135 OUT695 |
| ✓ | BbhLogicalDeductionSeven73BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 530 |
Reasoning - BBH logical deduction - order 7 objects (set 3)
31 passed |
100.0% |
IN35 412 OUT32 920 | |
| ✓ | BbhLogicalDeductionSeven100BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT1 283 |
| ✓ | BbhLogicalDeductionSeven101BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT828 |
| ✓ | BbhLogicalDeductionSeven102BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 266 |
| ✓ | BbhLogicalDeductionSeven103BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT614 |
| ✓ | BbhLogicalDeductionSeven104BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT942 |
| ✓ | BbhLogicalDeductionSeven74BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 023 |
| ✓ | BbhLogicalDeductionSeven75BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT844 |
| ✓ | BbhLogicalDeductionSeven76BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT674 |
| ✓ | BbhLogicalDeductionSeven77BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT813 |
| ✓ | BbhLogicalDeductionSeven78BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT1 413 |
| ✓ | BbhLogicalDeductionSeven79BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT814 |
| ✓ | BbhLogicalDeductionSeven80BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT762 |
| ✓ | BbhLogicalDeductionSeven81BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 183 OUT1 406 |
| ✓ | BbhLogicalDeductionSeven82BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 695 |
| ✓ | BbhLogicalDeductionSeven83BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 138 OUT945 |
| ✓ | BbhLogicalDeductionSeven84BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT1 028 |
| ✓ | BbhLogicalDeductionSeven85BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT939 |
| ✓ | BbhLogicalDeductionSeven86BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 260 |
| ✓ | BbhLogicalDeductionSeven87BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT546 |
| ✓ | BbhLogicalDeductionSeven88BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT548 |
| ✓ | BbhLogicalDeductionSeven89BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 135 OUT608 |
| ✓ | BbhLogicalDeductionSeven90BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT993 |
| ✓ | BbhLogicalDeductionSeven91BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 269 |
| ✓ | BbhLogicalDeductionSeven92BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 232 |
| ✓ | BbhLogicalDeductionSeven93BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT927 |
| ✓ | BbhLogicalDeductionSeven94BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 283 |
| ✓ | BbhLogicalDeductionSeven95BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 840 |
| ✓ | BbhLogicalDeductionSeven96BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 487 |
| ✓ | BbhLogicalDeductionSeven97BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT1 219 |
| ✓ | BbhLogicalDeductionSeven98BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT747 |
| ✓ | BbhLogicalDeductionSeven99BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 154 OUT1 672 |
Reasoning - BBH logical deduction - order 7 objects (set 4)
32 passed |
100.0% |
IN36 356 OUT38 076 | |
| ✓ | BbhLogicalDeductionSeven105BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 110 |
| ✓ | BbhLogicalDeductionSeven106BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 005 |
| ✓ | BbhLogicalDeductionSeven107BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 296 |
| ✓ | BbhLogicalDeductionSeven108BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT671 |
| ✓ | BbhLogicalDeductionSeven109BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 662 |
| ✓ | BbhLogicalDeductionSeven110BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT902 |
| ✓ | BbhLogicalDeductionSeven111BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT862 |
| ✓ | BbhLogicalDeductionSeven112BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 101 OUT1 511 |
| ✓ | BbhLogicalDeductionSeven113BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT670 |
| ✓ | BbhLogicalDeductionSeven114BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT667 |
| ✓ | BbhLogicalDeductionSeven115BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT644 |
| ✓ | BbhLogicalDeductionSeven116BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 728 |
| ✓ | BbhLogicalDeductionSeven117BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 302 |
| ✓ | BbhLogicalDeductionSeven118BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT958 |
| ✓ | BbhLogicalDeductionSeven119BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 857 |
| ✓ | BbhLogicalDeductionSeven120BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 139 |
| ✓ | BbhLogicalDeductionSeven121BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 098 OUT1 572 |
| ✓ | BbhLogicalDeductionSeven122BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 230 |
| ✓ | BbhLogicalDeductionSeven123BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 094 OUT1 194 |
| ✓ | BbhLogicalDeductionSeven124BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 239 |
| ✓ | BbhLogicalDeductionSeven125BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 244 |
| ✓ | BbhLogicalDeductionSeven126BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT744 |
| ✓ | BbhLogicalDeductionSeven127BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT871 |
| ✓ | BbhLogicalDeductionSeven128BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 156 |
| ✓ | BbhLogicalDeductionSeven129BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 278 |
| ✓ | BbhLogicalDeductionSeven130BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 090 OUT3 317 |
| ✓ | BbhLogicalDeductionSeven131BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 393 |
| ✓ | BbhLogicalDeductionSeven132BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT967 |
| ✓ | BbhLogicalDeductionSeven133BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT600 |
| ✓ | BbhLogicalDeductionSeven134BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT616 |
| ✓ | BbhLogicalDeductionSeven135BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 136 OUT868 |
| ✓ | BbhLogicalDeductionSeven136BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 803 |
Reasoning - BBH logical deduction - order 7 objects (set 5)
32 passed |
100.0% |
IN36 467 OUT36 912 | |
| ✓ | BbhLogicalDeductionSeven137BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 174 |
| ✓ | BbhLogicalDeductionSeven138BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT878 |
| ✓ | BbhLogicalDeductionSeven139BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT472 |
| ✓ | BbhLogicalDeductionSeven140BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT716 |
| ✓ | BbhLogicalDeductionSeven141BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 055 |
| ✓ | BbhLogicalDeductionSeven142BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 156 |
| ✓ | BbhLogicalDeductionSeven143BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT844 |
| ✓ | BbhLogicalDeductionSeven144BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT773 |
| ✓ | BbhLogicalDeductionSeven145BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT1 312 |
| ✓ | BbhLogicalDeductionSeven146BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT791 |
| ✓ | BbhLogicalDeductionSeven147BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT1 153 |
| ✓ | BbhLogicalDeductionSeven148BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT684 |
| ✓ | BbhLogicalDeductionSeven149BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 918 |
| ✓ | BbhLogicalDeductionSeven150BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 416 |
| ✓ | BbhLogicalDeductionSeven151BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 573 |
| ✓ | BbhLogicalDeductionSeven152BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT2 136 |
| ✓ | BbhLogicalDeductionSeven153BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 094 OUT1 297 |
| ✓ | BbhLogicalDeductionSeven154BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 091 |
| ✓ | BbhLogicalDeductionSeven155BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 052 |
| ✓ | BbhLogicalDeductionSeven156BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 042 |
| ✓ | BbhLogicalDeductionSeven157BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 084 OUT1 538 |
| ✓ | BbhLogicalDeductionSeven158BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT937 |
| ✓ | BbhLogicalDeductionSeven159BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 058 |
| ✓ | BbhLogicalDeductionSeven160BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 054 |
| ✓ | BbhLogicalDeductionSeven161BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 090 OUT1 288 |
| ✓ | BbhLogicalDeductionSeven162BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 688 |
| ✓ | BbhLogicalDeductionSeven163BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT931 |
| ✓ | BbhLogicalDeductionSeven164BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 299 |
| ✓ | BbhLogicalDeductionSeven165BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT697 |
| ✓ | BbhLogicalDeductionSeven166BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 931 |
| ✓ | BbhLogicalDeductionSeven167BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 091 OUT952 |
| ✓ | BbhLogicalDeductionSeven168BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 006 |
Reasoning - BBH logical deduction - order 7 objects (set 6)
32 passed |
100.0% |
IN36 304 OUT31 135 | |
| ✓ | BbhLogicalDeductionSeven169BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 228 |
| ✓ | BbhLogicalDeductionSeven170BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT776 |
| ✓ | BbhLogicalDeductionSeven171BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT544 |
| ✓ | BbhLogicalDeductionSeven172BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 087 OUT1 006 |
| ✓ | BbhLogicalDeductionSeven173BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT958 |
| ✓ | BbhLogicalDeductionSeven174BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 203 |
| ✓ | BbhLogicalDeductionSeven175BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT795 |
| ✓ | BbhLogicalDeductionSeven176BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT968 |
| ✓ | BbhLogicalDeductionSeven177BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 081 |
| ✓ | BbhLogicalDeductionSeven178BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT671 |
| ✓ | BbhLogicalDeductionSeven179BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT810 |
| ✓ | BbhLogicalDeductionSeven180BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT729 |
| ✓ | BbhLogicalDeductionSeven181BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 762 |
| ✓ | BbhLogicalDeductionSeven182BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT456 |
| ✓ | BbhLogicalDeductionSeven183BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT744 |
| ✓ | BbhLogicalDeductionSeven184BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 959 |
| ✓ | BbhLogicalDeductionSeven185BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 251 |
| ✓ | BbhLogicalDeductionSeven186BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT587 |
| ✓ | BbhLogicalDeductionSeven187BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 425 |
| ✓ | BbhLogicalDeductionSeven188BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT791 |
| ✓ | BbhLogicalDeductionSeven189BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 177 OUT969 |
| ✓ | BbhLogicalDeductionSeven190BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT796 |
| ✓ | BbhLogicalDeductionSeven191BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 005 |
| ✓ | BbhLogicalDeductionSeven192BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT585 |
| ✓ | BbhLogicalDeductionSeven193BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 217 |
| ✓ | BbhLogicalDeductionSeven194BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 000 |
| ✓ | BbhLogicalDeductionSeven195BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT796 |
| ✓ | BbhLogicalDeductionSeven196BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 031 |
| ✓ | BbhLogicalDeductionSeven197BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT1 431 |
| ✓ | BbhLogicalDeductionSeven198BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 063 |
| ✓ | BbhLogicalDeductionSeven199BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT633 |
| ✓ | BbhLogicalDeductionSeven200BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 124 OUT865 |
Reasoning - BBH logical deduction - order 7 objects (set 7)
32 passed |
100.0% |
IN36 429 OUT39 706 | |
| ✓ | BbhLogicalDeductionSeven201BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT2 098 |
| ✓ | BbhLogicalDeductionSeven202BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT945 |
| ✓ | BbhLogicalDeductionSeven203BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 205 |
| ✓ | BbhLogicalDeductionSeven204BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 227 |
| ✓ | BbhLogicalDeductionSeven205BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 212 |
| ✓ | BbhLogicalDeductionSeven206BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 140 OUT2 105 |
| ✓ | BbhLogicalDeductionSeven207BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 641 |
| ✓ | BbhLogicalDeductionSeven208BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 093 OUT568 |
| ✓ | BbhLogicalDeductionSeven209BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 094 OUT833 |
| ✓ | BbhLogicalDeductionSeven210BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT2 067 |
| ✓ | BbhLogicalDeductionSeven211BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 921 |
| ✓ | BbhLogicalDeductionSeven212BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT901 |
| ✓ | BbhLogicalDeductionSeven213BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 391 |
| ✓ | BbhLogicalDeductionSeven214BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT563 |
| ✓ | BbhLogicalDeductionSeven215BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT900 |
| ✓ | BbhLogicalDeductionSeven216BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT694 |
| ✓ | BbhLogicalDeductionSeven217BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 124 OUT992 |
| ✓ | BbhLogicalDeductionSeven218BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT1 691 |
| ✓ | BbhLogicalDeductionSeven219BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 036 |
| ✓ | BbhLogicalDeductionSeven220BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT508 |
| ✓ | BbhLogicalDeductionSeven221BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 090 OUT1 564 |
| ✓ | BbhLogicalDeductionSeven222BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT2 588 |
| ✓ | BbhLogicalDeductionSeven223BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT766 |
| ✓ | BbhLogicalDeductionSeven224BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT920 |
| ✓ | BbhLogicalDeductionSeven225BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 183 OUT1 627 |
| ✓ | BbhLogicalDeductionSeven226BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT625 |
| ✓ | BbhLogicalDeductionSeven227BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT892 |
| ✓ | BbhLogicalDeductionSeven228BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 028 |
| ✓ | BbhLogicalDeductionSeven229BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT765 |
| ✓ | BbhLogicalDeductionSeven230BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 142 OUT1 471 |
| ✓ | BbhLogicalDeductionSeven231BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT902 |
| ✓ | BbhLogicalDeductionSeven232BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 060 |
Reasoning - BBH logical deduction - order 7 objects (set 8)
30 passed |
100.0% |
IN33 188 OUT22 382 | |
| ✓ | BbhLogicalDeductionSeven233BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT982 |
| ✓ | BbhLogicalDeductionSeven234BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT992 |
| ✓ | BbhLogicalDeductionSeven235BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 846 |
| ✓ | BbhLogicalDeductionSeven236BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 921 |
| ✓ | BbhLogicalDeductionSeven237BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 381 |
| ✓ | BbhLogicalDeductionSeven238BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT998 |
| ✓ | BbhLogicalDeductionSeven239BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 213 |
| ✓ | BbhLogicalDeductionSeven240BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT814 |
| ✓ | BbhLogicalDeductionSeven241BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT1 133 |
| ✓ | BbhLogicalDeductionSeven242BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 177 OUT840 |
| ✓ | BbhLogicalDeductionSeven243BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT794 |
| ✓ | BbhLogicalDeductionSeven244BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 113 |
| ✓ | BbhLogicalDeductionSeven245BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT1 032 |
| ✓ | BbhLogicalDeductionSeven246BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT671 |
| ✓ | BbhLogicalDeductionSeven247BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT686 |
| ✓ | BbhLogicalDeductionSeven248BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT705 |
| ✓ | BbhLogicalDeductionSeven249BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 138 OUT1 074 |
| ✓ | BbhLogicalDeductionThree10BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT202 |
| ✓ | BbhLogicalDeductionThree11BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT296 |
| ✓ | BbhLogicalDeductionThree12BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT316 |
| ✓ | BbhLogicalDeductionThree13BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT268 |
| ✓ | BbhLogicalDeductionThree14BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT641 |
| ✓ | BbhLogicalDeductionThree15BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT400 |
| ✓ | BbhLogicalDeductionThree16BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT327 |
| ✓ | BbhLogicalDeductionThree17BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT250 |
| ✓ | BbhLogicalDeductionThree18BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT301 |
| ✓ | BbhLogicalDeductionThree19BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT219 |
| ✓ | BbhLogicalDeductionThree20BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT312 |
| ✓ | BbhLogicalDeductionThree8BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT409 |
| ✓ | BbhLogicalDeductionThree9BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT246 |
Reasoning - BBH movie recommendation (set 1)
30 passed2 failed |
93.8% |
IN32 573 OUT20 949 | |
| ✓ | BbhLogicalDeductionThreeObjects243BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT293 |
| ✓ | BbhLogicalDeductionThreeObjects244BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT401 |
| ✓ | BbhLogicalDeductionThreeObjects245BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT301 |
| ✓ | BbhLogicalDeductionThreeObjects246BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT335 |
| ✓ | BbhLogicalDeductionThreeObjects247BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT279 |
| ✓ | BbhLogicalDeductionThreeObjects248BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT622 |
| ✓ | BbhLogicalDeductionThreeObjects249BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT184 |
| ✓ | BbhMovieRecommendation10BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 551 |
| ✓ | BbhMovieRecommendation11BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT734 |
| ✓ | BbhMovieRecommendation12BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT517 |
| ✓ | BbhMovieRecommendation13BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT497 |
| ✓ | BbhMovieRecommendation14BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT544 |
| ✓ | BbhMovieRecommendation15BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT867 |
| ✓ | BbhMovieRecommendation16BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT558 |
| ✓ | BbhMovieRecommendation17BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT659 |
| ✓ | BbhMovieRecommendation18BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT621 |
| ✓ | BbhMovieRecommendation19BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT706 |
| ✓ | BbhMovieRecommendation20BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT598 |
| ✓ | BbhMovieRecommendation21BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT767 |
| ✓ | BbhMovieRecommendation22BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT617 |
| ✓ | BbhMovieRecommendation23BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT463 |
| ✓ | BbhMovieRecommendation24BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT956 |
| × | BbhMovieRecommendation25BTFind a movie similar to Army of Darkness, Go, The Sixth Sense, Alien:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation25BT: expected (A), got [(B)] |
Failed | IN1 000 OUT861 |
| ✓ | BbhMovieRecommendation26BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT651 |
| ✓ | BbhMovieRecommendation27BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT453 |
| ✓ | BbhMovieRecommendation28BTFind a movie similar to Toy Story, The Mask, Batman, The Lion King:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 205 |
| ✓ | BbhMovieRecommendation29BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT476 |
| ✓ | BbhMovieRecommendation30BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT778 |
| ✓ | BbhMovieRecommendation31BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT875 |
| ✓ | BbhMovieRecommendation32BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT726 |
| ✓ | BbhMovieRecommendation8BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT581 |
| × | BbhMovieRecommendation9BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhMovieRecommendation9BT: expected (C), got [(B)] |
Failed | IN1 023 OUT1 273 |
Reasoning - BBH movie recommendation (set 2)
29 passed3 failed |
90.6% |
IN32 324 OUT27 017 | |
| ✓ | BbhMovieRecommendation33BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT840 |
| ✓ | BbhMovieRecommendation34BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 727 |
| ✓ | BbhMovieRecommendation35BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 296 |
| ✓ | BbhMovieRecommendation36BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT732 |
| × | BbhMovieRecommendation37BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation37BT: expected (A), got [(D)] |
Failed | IN1 007 OUT1 839 |
| ✓ | BbhMovieRecommendation38BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT702 |
| ✓ | BbhMovieRecommendation39BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT565 |
| ✓ | BbhMovieRecommendation40BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 616 |
| ✓ | BbhMovieRecommendation41BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT501 |
| ✓ | BbhMovieRecommendation42BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT576 |
| ✓ | BbhMovieRecommendation43BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT733 |
| ✓ | BbhMovieRecommendation44BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT713 |
| ✓ | BbhMovieRecommendation45BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT902 |
| ✓ | BbhMovieRecommendation46BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT498 |
| ✓ | BbhMovieRecommendation47BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT477 |
| ✓ | BbhMovieRecommendation48BTFind a movie similar to Batman, The Mask, The Lion King, Pretty Woman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT428 |
| ✓ | BbhMovieRecommendation49BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT496 |
| ✓ | BbhMovieRecommendation50BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT740 |
| ✓ | BbhMovieRecommendation51BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT707 |
| × | BbhMovieRecommendation52BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation52BT: expected (A), got [(B)] |
Failed | IN1 002 OUT511 |
| ✓ | BbhMovieRecommendation53BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT523 |
| ✓ | BbhMovieRecommendation54BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT617 |
| ✓ | BbhMovieRecommendation55BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT806 |
| ✓ | BbhMovieRecommendation56BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT502 |
| ✓ | BbhMovieRecommendation57BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT544 |
| ✓ | BbhMovieRecommendation58BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT676 |
| ✓ | BbhMovieRecommendation59BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT604 |
| ✓ | BbhMovieRecommendation60BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT573 |
| ✓ | BbhMovieRecommendation61BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 154 |
| × | BbhMovieRecommendation62BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhMovieRecommendation62BT: expected (B), got [(C)] |
Failed | IN1 008 OUT1 746 |
| ✓ | BbhMovieRecommendation63BTFind a movie similar to Pretty Woman, Aladdin, The Lion King, Batman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT714 |
| ✓ | BbhMovieRecommendation64BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 959 |
Reasoning - BBH movie recommendation (set 3)
29 passed1 failed |
96.7% |
IN30 348 OUT22 527 | |
| ✓ | BbhMovieRecommendation65BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT599 |
| ✓ | BbhMovieRecommendation66BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 248 |
| ✓ | BbhMovieRecommendation67BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT498 |
| ✓ | BbhMovieRecommendation68BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT906 |
| ✓ | BbhMovieRecommendation69BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT474 |
| ✓ | BbhMovieRecommendation70BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT605 |
| ✓ | BbhMovieRecommendation71BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT725 |
| × | BbhMovieRecommendation72BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhMovieRecommendation72BT: expected (C), got [(A)] |
Failed | IN1 002 OUT945 |
| ✓ | BbhMovieRecommendation73BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT751 |
| ✓ | BbhMovieRecommendation74BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT712 |
| ✓ | BbhMovieRecommendation75BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT460 |
| ✓ | BbhMovieRecommendation76BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT631 |
| ✓ | BbhMovieRecommendation77BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT480 |
| ✓ | BbhMovieRecommendation78BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 496 |
| ✓ | BbhMovieRecommendation79BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT554 |
| ✓ | BbhMovieRecommendation80BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 522 |
| ✓ | BbhMovieRecommendation81BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT434 |
| ✓ | BbhMovieRecommendation82BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT560 |
| ✓ | BbhMovieRecommendation83BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT551 |
| ✓ | BbhMovieRecommendation84BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT749 |
| ✓ | BbhMovieRecommendation85BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT704 |
| ✓ | BbhMovieRecommendation86BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT781 |
| ✓ | BbhMovieRecommendation87BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT479 |
| ✓ | BbhMovieRecommendation88BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT1 128 |
| ✓ | BbhMovieRecommendation89BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT539 |
| ✓ | BbhMovieRecommendation90BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 199 |
| ✓ | BbhMovieRecommendation91BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT609 |
| ✓ | BbhMovieRecommendation92BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 038 |
| ✓ | BbhMovieRecommendation93BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT612 |
| ✓ | BbhMovieRecommendation94BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT538 |
Reasoning - BBH movie recommendation (set 4)
26 passed6 failed |
81.3% |
IN32 296 OUT33 495 | |
| ✓ | BbhMovieRecommendation100BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 205 |
| ✓ | BbhMovieRecommendation101BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT528 |
| ✓ | BbhMovieRecommendation102BUFind a movie similar to The Mask, Batman, Pretty Woman, The Fugitive:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT723 |
| ✓ | BbhMovieRecommendation103BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT479 |
| ✓ | BbhMovieRecommendation104BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT535 |
| × | BbhMovieRecommendation105BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation105BU: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 008 OUT3 134 |
| × | BbhMovieRecommendation106BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation106BU: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 003 OUT2 611 |
| ✓ | BbhMovieRecommendation107BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT626 |
| ✓ | BbhMovieRecommendation108BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT834 |
| ✓ | BbhMovieRecommendation109BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT643 |
| ✓ | BbhMovieRecommendation110BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT647 |
| ✓ | BbhMovieRecommendation111BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 119 |
| ✓ | BbhMovieRecommendation112BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT575 |
| ✓ | BbhMovieRecommendation113BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT620 |
| ✓ | BbhMovieRecommendation114BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT755 |
| × | BbhMovieRecommendation115BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation115BU: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 015 OUT2 733 |
| × | BbhMovieRecommendation116BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation116BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 020 OUT566 |
| ✓ | BbhMovieRecommendation117BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT941 |
| × | BbhMovieRecommendation118BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation118BU: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT1 304 |
| ✓ | BbhMovieRecommendation119BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT974 |
| ✓ | BbhMovieRecommendation120BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 980 |
| ✓ | BbhMovieRecommendation121BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT586 |
| ✓ | BbhMovieRecommendation122BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT538 |
| ✓ | BbhMovieRecommendation123BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT517 |
| ✓ | BbhMovieRecommendation124BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT659 |
| ✓ | BbhMovieRecommendation125BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 011 |
| ✓ | BbhMovieRecommendation126BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT561 |
| ✓ | BbhMovieRecommendation95BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT524 |
| ✓ | BbhMovieRecommendation96BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT831 |
| ✓ | BbhMovieRecommendation97BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT596 |
| ✓ | BbhMovieRecommendation98BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT775 |
| × | BbhMovieRecommendation99BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation99BU: expected (B), got [a]
Values:
got = "a" |
Failed | IN996 OUT2 365 |
Reasoning - BBH movie recommendation (set 5)
27 passed5 failed |
84.4% |
IN32 324 OUT29 832 | |
| ✓ | BbhMovieRecommendation127BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT701 |
| ✓ | BbhMovieRecommendation128BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT582 |
| ✓ | BbhMovieRecommendation129BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT588 |
| ✓ | BbhMovieRecommendation130BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT684 |
| ✓ | BbhMovieRecommendation131BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 242 |
| × | BbhMovieRecommendation132BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation132BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 021 OUT731 |
| × | BbhMovieRecommendation133BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation133BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 003 OUT986 |
| ✓ | BbhMovieRecommendation134BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT623 |
| × | BbhMovieRecommendation135BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation135BU: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 004 OUT992 |
| ✓ | BbhMovieRecommendation136BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT798 |
| ✓ | BbhMovieRecommendation137BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT665 |
| ✓ | BbhMovieRecommendation138BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 256 |
| ✓ | BbhMovieRecommendation139BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 281 |
| ✓ | BbhMovieRecommendation140BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT977 |
| ✓ | BbhMovieRecommendation141BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 584 |
| ✓ | BbhMovieRecommendation142BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT542 |
| ✓ | BbhMovieRecommendation143BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT602 |
| ✓ | BbhMovieRecommendation144BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 710 |
| ✓ | BbhMovieRecommendation145BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT691 |
| ✓ | BbhMovieRecommendation146BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT562 |
| ✓ | BbhMovieRecommendation147BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 767 |
| ✓ | BbhMovieRecommendation148BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT560 |
| ✓ | BbhMovieRecommendation149BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT345 |
| × | BbhMovieRecommendation150BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation150BU: expected (D), got [b]
Values:
got = "b" |
Failed | IN1 007 OUT1 677 |
| ✓ | BbhMovieRecommendation151BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT992 |
| ✓ | BbhMovieRecommendation152BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT878 |
| ✓ | BbhMovieRecommendation153BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT1 169 |
| ✓ | BbhMovieRecommendation154BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT453 |
| × | BbhMovieRecommendation155BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation155BU: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 009 OUT438 |
| ✓ | BbhMovieRecommendation156BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT929 |
| ✓ | BbhMovieRecommendation157BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 108 |
| ✓ | BbhMovieRecommendation158BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT719 |
Reasoning - BBH movie recommendation (set 6)
28 passed4 failed |
87.5% |
IN32 264 OUT30 037 | |
| ✓ | BbhMovieRecommendation159BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT469 |
| ✓ | BbhMovieRecommendation160BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT846 |
| ✓ | BbhMovieRecommendation161BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT678 |
| ✓ | BbhMovieRecommendation162BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT788 |
| × | BbhMovieRecommendation163BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Monsters, Inc")
Context: BbhMovieRecommendation163BU: expected Monsters, Inc, got [a]
Values:
got = "a" |
Failed | IN1 007 OUT1 984 |
| ✓ | BbhMovieRecommendation164BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT602 |
| ✓ | BbhMovieRecommendation165BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT583 |
| × | BbhMovieRecommendation166BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation166BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN999 OUT543 |
| ✓ | BbhMovieRecommendation167BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 683 |
| ✓ | BbhMovieRecommendation168BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT610 |
| ✓ | BbhMovieRecommendation169BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 678 |
| ✓ | BbhMovieRecommendation170BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT876 |
| ✓ | BbhMovieRecommendation171BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT667 |
| ✓ | BbhMovieRecommendation172BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT510 |
| ✓ | BbhMovieRecommendation173BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 682 |
| ✓ | BbhMovieRecommendation174BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT801 |
| ✓ | BbhMovieRecommendation175BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT701 |
| ✓ | BbhMovieRecommendation176BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT627 |
| × | BbhMovieRecommendation177BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation177BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 009 OUT533 |
| ✓ | BbhMovieRecommendation178BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT829 |
| ✓ | BbhMovieRecommendation179BUFind a movie similar to American Beauty, Pi, X-Men, The Godfather:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 016 |
| ✓ | BbhMovieRecommendation180BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT502 |
| ✓ | BbhMovieRecommendation181BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT524 |
| ✓ | BbhMovieRecommendation182BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 136 |
| ✓ | BbhMovieRecommendation183BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT609 |
| ✓ | BbhMovieRecommendation184BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT690 |
| ✓ | BbhMovieRecommendation185BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 731 |
| ✓ | BbhMovieRecommendation186BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 585 |
| ✓ | BbhMovieRecommendation187BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT502 |
| × | BbhMovieRecommendation188BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation188BU: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 008 OUT2 639 |
| ✓ | BbhMovieRecommendation189BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT996 |
| ✓ | BbhMovieRecommendation190BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT417 |
Reasoning - BBH movie recommendation (set 7)
28 passed4 failed |
87.5% |
IN32 336 OUT25 516 | |
| ✓ | BbhMovieRecommendation191BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 094 |
| ✓ | BbhMovieRecommendation192BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT633 |
| ✓ | BbhMovieRecommendation193BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT489 |
| ✓ | BbhMovieRecommendation194BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 314 |
| × | BbhMovieRecommendation195BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation195BU: expected (A), got [d]
Values:
got = "d" |
Failed | IN998 OUT709 |
| ✓ | BbhMovieRecommendation196BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT881 |
| ✓ | BbhMovieRecommendation197BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT723 |
| ✓ | BbhMovieRecommendation198BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT571 |
| ✓ | BbhMovieRecommendation199BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT720 |
| × | BbhMovieRecommendation200BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation200BU: expected (A), got [c]
Values:
got = "c" |
Failed | IN1 023 OUT552 |
| ✓ | BbhMovieRecommendation201BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT817 |
| ✓ | BbhMovieRecommendation202BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT544 |
| ✓ | BbhMovieRecommendation203BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT708 |
| ✓ | BbhMovieRecommendation204BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT602 |
| ✓ | BbhMovieRecommendation205BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT873 |
| ✓ | BbhMovieRecommendation206BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT626 |
| ✓ | BbhMovieRecommendation207BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT917 |
| ✓ | BbhMovieRecommendation208BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT685 |
| ✓ | BbhMovieRecommendation209BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 057 |
| ✓ | BbhMovieRecommendation210BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 119 |
| ✓ | BbhMovieRecommendation211BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 060 |
| ✓ | BbhMovieRecommendation212BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT496 |
| ✓ | BbhMovieRecommendation213BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT540 |
| ✓ | BbhMovieRecommendation214BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT639 |
| ✓ | BbhMovieRecommendation215BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 686 |
| ✓ | BbhMovieRecommendation216BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT599 |
| ✓ | BbhMovieRecommendation217BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT787 |
| ✓ | BbhMovieRecommendation218BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT770 |
| × | BbhMovieRecommendation219BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation219BU: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 003 OUT448 |
| ✓ | BbhMovieRecommendation220BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 103 |
| ✓ | BbhMovieRecommendation221BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT671 |
| × | BbhMovieRecommendation222BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation222BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 006 OUT1 083 |
Reasoning - BBH movie recommendation (set 8)
25 passed5 failed |
83.3% |
IN30 201 OUT19 452 | |
| × | BbhMovieRecommendation223BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation223BU: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 007 OUT505 |
| ✓ | BbhMovieRecommendation224BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT648 |
| ✓ | BbhMovieRecommendation225BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT584 |
| ✓ | BbhMovieRecommendation226BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT920 |
| ✓ | BbhMovieRecommendation227BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT973 |
| ✓ | BbhMovieRecommendation228BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT533 |
| ✓ | BbhMovieRecommendation229BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT482 |
| × | BbhMovieRecommendation230BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation230BU: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 004 OUT1 080 |
| ✓ | BbhMovieRecommendation231BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT766 |
| ✓ | BbhMovieRecommendation232BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT696 |
| × | BbhMovieRecommendation233BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation233BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 011 OUT556 |
| ✓ | BbhMovieRecommendation234BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT522 |
| ✓ | BbhMovieRecommendation235BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT515 |
| × | BbhMovieRecommendation236BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation236BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 008 OUT566 |
| ✓ | BbhMovieRecommendation237BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT644 |
| ✓ | BbhMovieRecommendation238BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT557 |
| ✓ | BbhMovieRecommendation239BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT406 |
| ✓ | BbhMovieRecommendation240BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT846 |
| ✓ | BbhMovieRecommendation241BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT499 |
| ✓ | BbhMovieRecommendation242BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT431 |
| ✓ | BbhMovieRecommendation243BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT592 |
| ✓ | BbhMovieRecommendation244BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT622 |
| ✓ | BbhMovieRecommendation245BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT941 |
| ✓ | BbhMovieRecommendation246BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT494 |
| × | BbhMovieRecommendation247BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation247BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 007 OUT708 |
| ✓ | BbhMovieRecommendation248BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT534 |
| ✓ | BbhMovieRecommendation249BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 428 |
| ✓ | BbhMultistepArithmeticTwo10BU((6 - 0 * 5 + -3) * (6 - -7 + -2 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT394 |
| ✓ | BbhMultistepArithmeticTwo8BU((-7 * -9 + 8 * -3) * (5 + -7 - 4 * -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT615 |
| ✓ | BbhMultistepArithmeticTwo9BU((-9 - 1 * 5 * -5) - (6 + -3 - -1 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT395 |
Reasoning - BBH multi-step arithmetic (set 1)
32 passed |
100.0% |
IN31 392 OUT13 048 | |
| ✓ | BbhMultistepArithmeticTwo11BV((2 - -2 + -7 * 8) * (-7 * -8 * 3 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT640 |
| ✓ | BbhMultistepArithmeticTwo12BV((8 - 2 + -2 * 6) * (8 + -6 + -8 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT378 |
| ✓ | BbhMultistepArithmeticTwo13BV((-6 + -9 - -6 + -4) * (-1 - -6 + -4 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT376 |
| ✓ | BbhMultistepArithmeticTwo14BV((-5 - 4 * -8 + 8) * (4 + 3 - 9 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT500 |
| ✓ | BbhMultistepArithmeticTwo15BV((-5 * -7 * -6 + 9) * (-2 - 8 + -5 + 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT369 |
| ✓ | BbhMultistepArithmeticTwo16BV((8 + 9 - 4 - -9) + (8 + 7 - 6 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT338 |
| ✓ | BbhMultistepArithmeticTwo17BV((5 * -1 + -6 * -3) + (-1 + -8 - 5 + 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT358 |
| ✓ | BbhMultistepArithmeticTwo18BV((-6 - 6 + 7 - 7) - (5 + 3 - 9 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT399 |
| ✓ | BbhMultistepArithmeticTwo19BV((7 - 4 + -3 * 4) - (5 + -8 - 6 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT376 |
| ✓ | BbhMultistepArithmeticTwo20BV((-6 * -1 - 2 + -2) + (9 - 4 + -1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT320 |
| ✓ | BbhMultistepArithmeticTwo21BV((8 * -6 + 6 * 1) - (-3 * 7 * 0 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT330 |
| ✓ | BbhMultistepArithmeticTwo22BV((-6 - 8 - -7 * -2) - (-9 - 5 + 7 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT383 |
| ✓ | BbhMultistepArithmeticTwo23BV((1 - 7 - -8 * 3) + (-7 - -2 + -3 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT456 |
| ✓ | BbhMultistepArithmeticTwo24BV((5 * -8 - -5 * -9) * (2 - -7 * 6 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT490 |
| ✓ | BbhMultistepArithmeticTwo25BV((6 + 1 - 4 - 3) - (-4 * -6 * -3 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT332 |
| ✓ | BbhMultistepArithmeticTwo26BV((-9 - -9 + 0 + -3) + (-2 - -1 - 1 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT299 |
| ✓ | BbhMultistepArithmeticTwo27BV((9 - 3 + 2 + -1) - (5 - -1 - -6 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT368 |
| ✓ | BbhMultistepArithmeticTwo28BV((-8 + 6 * -2 + 4) * (-4 * 5 + 2 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT516 |
| ✓ | BbhMultistepArithmeticTwo29BV((3 + -1 * 7 * -6) - (-7 * -1 + -5 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT342 |
| ✓ | BbhMultistepArithmeticTwo30BV((3 - 3 - -3 + -2) + (8 - 2 + -8 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT365 |
| ✓ | BbhMultistepArithmeticTwo31BV((-2 + -9 - -2 * 2) - (6 * 1 * -2 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT429 |
| ✓ | BbhMultistepArithmeticTwo32BV((-7 + -2 * 1 - 6) + (3 + 9 * 7 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT409 |
| ✓ | BbhMultistepArithmeticTwo33BV((4 + 7 * 4 - -5) - (-4 - 1 - -4 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT465 |
| ✓ | BbhMultistepArithmeticTwo34BV((1 - 6 * 0 - -7) * (6 * 0 + 6 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT401 |
| ✓ | BbhMultistepArithmeticTwo35BV((-1 + -7 + -2 - 7) * (6 - 1 - -1 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT399 |
| ✓ | BbhMultistepArithmeticTwo36BV((-5 + 6 - -5 + -6) + (-4 + -6 - 3 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT361 |
| ✓ | BbhMultistepArithmeticTwo37BV((0 - -1 + -1 + -3) - (-3 * 0 * 8 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT346 |
| ✓ | BbhMultistepArithmeticTwo38BV((-8 + -6 * 9 - 7) + (8 + 2 * -9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT574 |
| ✓ | BbhMultistepArithmeticTwo39BV((-3 * 0 * 4 - -7) + (-9 * -6 - -8 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT319 |
| ✓ | BbhMultistepArithmeticTwo40BV((-5 - 2 + 9 * -1) * (7 - 5 + 9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT498 |
| ✓ | BbhMultistepArithmeticTwo41BV((5 - 7 - 8 - 6) * (-2 * -8 * -9 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT542 |
| ✓ | BbhMultistepArithmeticTwo42BV((-5 + 1 - 2 - -3) * (-9 * 3 + 9 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT370 |
Reasoning - BBH multi-step arithmetic (set 2)
32 passed |
100.0% |
IN31 392 OUT12 937 | |
| ✓ | BbhMultistepArithmeticTwo43BV((-2 * -8 * 9 + -5) + (7 - -1 - -4 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT287 |
| ✓ | BbhMultistepArithmeticTwo44BV((-2 - 5 * -5 - 6) + (-5 * -4 + 2 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT371 |
| ✓ | BbhMultistepArithmeticTwo45BV((-5 - 6 * 5 + -9) + (1 + -4 - -2 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT390 |
| ✓ | BbhMultistepArithmeticTwo46BV((-8 + 0 * 0 * -1) - (7 + -1 + -3 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT381 |
| ✓ | BbhMultistepArithmeticTwo47BV((-9 - -3 * 1 - -3) * (-9 - 5 - -2 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT450 |
| ✓ | BbhMultistepArithmeticTwo48BV((5 * -2 + 8 * 1) * (9 + -3 * -8 - -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT342 |
| ✓ | BbhMultistepArithmeticTwo49BV((-7 * 9 - -9 + 7) + (9 * -6 * -4 * -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT451 |
| ✓ | BbhMultistepArithmeticTwo50BV((4 - 3 * -1 * 3) * (3 + 8 + 1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT303 |
| ✓ | BbhMultistepArithmeticTwo51BV((-3 - 8 * 5 + 5) - (-1 * -3 - -2 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT458 |
| ✓ | BbhMultistepArithmeticTwo52BV((1 + 3 - -5 - 4) + (-3 - -3 + 6 * 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT418 |
| ✓ | BbhMultistepArithmeticTwo53BV((3 * 9 - 3 * 2) + (-1 * -7 - 1 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT384 |
| ✓ | BbhMultistepArithmeticTwo54BV((7 * 9 * 3 * 6) * (-8 * -6 + -9 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT666 |
| ✓ | BbhMultistepArithmeticTwo55BV((1 + 1 + -7 * -6) - (4 * 2 * -5 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT373 |
| ✓ | BbhMultistepArithmeticTwo56BV((1 + 1 * 3 - 2) * (-8 - -2 - 9 + -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT372 |
| ✓ | BbhMultistepArithmeticTwo57BV((5 + 6 * 4 * 9) + (-9 - 0 * 3 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT391 |
| ✓ | BbhMultistepArithmeticTwo58BV((-1 - 9 + -9 * -9) - (1 + -3 - 5 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT429 |
| ✓ | BbhMultistepArithmeticTwo59BV((-1 + -6 - -4 + -6) - (-2 * -5 - -1 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT413 |
| ✓ | BbhMultistepArithmeticTwo60BV((-2 - -7 + 5 * -3) + (2 - 8 + 5 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT441 |
| ✓ | BbhMultistepArithmeticTwo61BV((3 - -6 - 7 + 3) + (-7 + 6 * 9 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT323 |
| ✓ | BbhMultistepArithmeticTwo62BV((-1 - -9 * 8 - -9) + (-9 - 2 * -6 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT480 |
| ✓ | BbhMultistepArithmeticTwo63BV((-5 + 6 + -6 * -1) + (-5 + -5 + 9 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT336 |
| ✓ | BbhMultistepArithmeticTwo64BV((4 + -1 + -7 + 2) - (-7 + -9 * -6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT465 |
| ✓ | BbhMultistepArithmeticTwo65BV((8 * 9 * 0 - -1) - (-9 - -7 + -4 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT357 |
| ✓ | BbhMultistepArithmeticTwo66BV((4 + 5 * -9 * -8) - (-9 + -7 - -6 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT450 |
| ✓ | BbhMultistepArithmeticTwo67BV((-8 + 5 - 0 - 6) * (-5 * -4 - 6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT313 |
| ✓ | BbhMultistepArithmeticTwo68BV((-4 + -5 * 7 + 7) + (3 * -3 - -4 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT425 |
| ✓ | BbhMultistepArithmeticTwo69BV((0 * 8 - -9 * 6) - (-3 * -9 * -1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT472 |
| ✓ | BbhMultistepArithmeticTwo70BV((0 * 9 + -8 * 7) - (7 - 9 + 1 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT308 |
| ✓ | BbhMultistepArithmeticTwo71BV((9 * 4 - 6 - -8) + (6 - 8 - -4 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT531 |
| ✓ | BbhMultistepArithmeticTwo72BV((6 - 1 - -3 + -2) + (5 * 5 + -1 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT398 |
| ✓ | BbhMultistepArithmeticTwo73BV((-4 + 4 + 3 * 1) - (-9 - -3 + -6 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT402 |
| ✓ | BbhMultistepArithmeticTwo74BV((-3 - -6 - -7 + 3) + (5 * 5 * 1 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT357 |
Reasoning - BBH multi-step arithmetic (set 3)
32 passed |
100.0% |
IN31 392 OUT13 492 | |
| ✓ | BbhMultistepArithmeticTwo100BV((7 - 5 * 7 + -1) + (6 * 4 - 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT334 |
| ✓ | BbhMultistepArithmeticTwo101BV((6 + -8 * -3 - -5) * (2 - 7 * -2 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT599 |
| ✓ | BbhMultistepArithmeticTwo102BV((-1 * -1 * -6 - 7) - (-4 * 0 - -4 - 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT330 |
| ✓ | BbhMultistepArithmeticTwo103BV((-6 + 1 - -4 * -2) + (4 - -8 - -1 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT475 |
| ✓ | BbhMultistepArithmeticTwo104BV((7 + -9 * -5 + -3) * (-8 * 8 - -5 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT612 |
| ✓ | BbhMultistepArithmeticTwo105BV((3 * 4 * 0 * -3) - (6 + -4 + 2 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT361 |
| ✓ | BbhMultistepArithmeticTwo106BV((2 + -6 * -7 - 7) + (-2 - -5 + -4 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT425 |
| ✓ | BbhMultistepArithmeticTwo75BV((-8 * 4 + 7 - 5) + (9 + -6 + -2 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT375 |
| ✓ | BbhMultistepArithmeticTwo76BV((-8 - -6 * 7 * 0) * (0 + -5 * -8 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT372 |
| ✓ | BbhMultistepArithmeticTwo77BV((5 * 1 * 3 - 9) * (-7 + 4 - -7 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT351 |
| ✓ | BbhMultistepArithmeticTwo78BV((5 - -2 * -9 + 8) - (3 - 4 * -3 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT504 |
| ✓ | BbhMultistepArithmeticTwo79BV((7 + 1 + 9 * 1) + (7 - -3 + 3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT422 |
| ✓ | BbhMultistepArithmeticTwo80BV((-6 + -2 * 3 * 0) + (3 + -4 - 7 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT296 |
| ✓ | BbhMultistepArithmeticTwo81BV((-8 + -4 * -9 * 0) + (2 * -9 + 3 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT280 |
| ✓ | BbhMultistepArithmeticTwo82BV((-8 + 6 - 8 * -9) - (9 - 9 * -2 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT566 |
| ✓ | BbhMultistepArithmeticTwo83BV((9 - -6 * 9 * 4) - (6 - 4 * -8 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT496 |
| ✓ | BbhMultistepArithmeticTwo84BV((8 + 3 * -8 - -8) - (2 + 2 * -9 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT446 |
| ✓ | BbhMultistepArithmeticTwo85BV((-9 + 8 - -4 * 2) - (1 - 9 + -6 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT576 |
| ✓ | BbhMultistepArithmeticTwo86BV((-1 * -6 - -1 - -8) - (8 + -7 * 8 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT418 |
| ✓ | BbhMultistepArithmeticTwo87BV((-3 * 6 - 0 * 6) - (-6 * -6 * 7 + -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT410 |
| ✓ | BbhMultistepArithmeticTwo88BV((-6 - -1 - -5 * -5) + (8 + -5 - 9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT468 |
| ✓ | BbhMultistepArithmeticTwo89BV((-9 + -4 + 0 * -1) * (4 * -3 * -8 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT382 |
| ✓ | BbhMultistepArithmeticTwo90BV((-2 - 2 * 2 + 0) - (7 - 6 - -7 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT394 |
| ✓ | BbhMultistepArithmeticTwo91BV((-7 - -4 * 2 - 9) * (9 - 6 - 1 + -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT379 |
| ✓ | BbhMultistepArithmeticTwo92BV((-9 * 4 * -9 - -9) * (5 * 2 * 6 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT454 |
| ✓ | BbhMultistepArithmeticTwo93BV((-9 + -3 + 6 - 8) - (-7 * -2 * -6 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT305 |
| ✓ | BbhMultistepArithmeticTwo94BV((-7 + 5 * -6 * -6) * (7 * -5 + -6 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT502 |
| ✓ | BbhMultistepArithmeticTwo95BV((-8 * 0 + 2 * 2) * (-9 - 1 * -9 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT459 |
| ✓ | BbhMultistepArithmeticTwo96BV((9 + 2 + 3 - -3) + (-5 + 6 - -6 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT387 |
| ✓ | BbhMultistepArithmeticTwo97BV((5 - 6 - 7 + 6) - (-1 - 1 * -7 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT402 |
| ✓ | BbhMultistepArithmeticTwo98BV((-9 - -7 + 2 + 2) + (-7 + -9 - -1 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT421 |
| ✓ | BbhMultistepArithmeticTwo99BV((-6 * -4 + -1 + 1) + (-9 * -6 + 7 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT291 |
Reasoning - BBH multi-step arithmetic (set 4)
31 passed |
100.0% |
IN30 411 OUT12 477 | |
| ✓ | BbhMultistepArithmeticTwo107BV((-3 + -4 * 5 * -7) - (-1 * -4 + -9 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT446 |
| ✓ | BbhMultistepArithmeticTwo108BV((0 + -9 * 6 * 4) + (5 - -3 + 8 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT422 |
| ✓ | BbhMultistepArithmeticTwo109BV((-4 - -6 - -5 - 5) - (-2 + -4 - -8 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT374 |
| ✓ | BbhMultistepArithmeticTwo110BV((-3 + -4 * 3 - -8) + (8 - 1 - -4 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT444 |
| ✓ | BbhMultistepArithmeticTwo111BV((-4 - 5 + 2 + 7) - (-9 * 6 * -1 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT442 |
| ✓ | BbhMultistepArithmeticTwo112BV((8 + 1 + 6 + 6) - (7 - -7 - 9 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT292 |
| ✓ | BbhMultistepArithmeticTwo113BV((-9 * -5 - 5 + 7) - (-3 - 8 + 9 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT423 |
| ✓ | BbhMultistepArithmeticTwo114BV((0 + 1 - -6 * 9) * (4 + 7 + -9 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT433 |
| ✓ | BbhMultistepArithmeticTwo115BV((-1 + -6 + -3 * -2) * (-9 * -1 + 2 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT414 |
| ✓ | BbhMultistepArithmeticTwo116BV((6 * 5 + 0 - -3) * (4 * 5 * 3 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT427 |
| ✓ | BbhMultistepArithmeticTwo117BV((-2 * -1 + 1 * -3) - (7 * -7 + -5 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT357 |
| ✓ | BbhMultistepArithmeticTwo118BV((-1 + 5 - -1 + 3) + (0 - -3 - -5 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT427 |
| ✓ | BbhMultistepArithmeticTwo119BV((0 - 5 + 0 + 3) * (-7 + -8 * 4 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT433 |
| ✓ | BbhMultistepArithmeticTwo120BV((-2 - 3 - 0 - 2) * (1 - -7 - -3 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT350 |
| ✓ | BbhMultistepArithmeticTwo121BV((3 + 3 - -4 - 2) + (-2 - -7 * -2 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT369 |
| ✓ | BbhMultistepArithmeticTwo122BV((-9 - -8 + 3 * -1) * (-9 * -2 * 4 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT394 |
| ✓ | BbhMultistepArithmeticTwo123BV((-1 - 9 * 6 + 3) * (0 - 8 - -4 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT362 |
| ✓ | BbhMultistepArithmeticTwo124BV((-9 - 8 * 9 - 0) + (-4 + 4 + -2 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT370 |
| ✓ | BbhMultistepArithmeticTwo125BV((-6 - -5 - -9 - -9) * (-6 * 4 - 8 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT471 |
| ✓ | BbhMultistepArithmeticTwo126BV((1 + 2 * -1 - -8) - (-8 - 2 - -1 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT389 |
| ✓ | BbhMultistepArithmeticTwo127BV((-6 * -2 + -2 - -9) + (-9 * 3 * -1 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT336 |
| ✓ | BbhMultistepArithmeticTwo128BV((-1 * -3 - 2 + -7) + (4 * 8 * -6 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT350 |
| ✓ | BbhMultistepArithmeticTwo129BV((8 + 6 * -1 - -3) * (-5 * 0 - -7 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT350 |
| ✓ | BbhMultistepArithmeticTwo130BV((-5 - -1 - 7 * 8) * (-1 - -4 * -7 * 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT580 |
| ✓ | BbhMultistepArithmeticTwo131BV((4 - 8 - -9 * 7) - (5 + -8 * 9 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT602 |
| ✓ | BbhMultistepArithmeticTwo132BV((-7 - -8 + 5 - -4) + (6 - -6 * 4 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT426 |
| ✓ | BbhMultistepArithmeticTwo133BV((-3 * 4 * 3 + -5) + (-7 - 5 + 7 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT358 |
| ✓ | BbhMultistepArithmeticTwo134BV((-2 - 1 - -6 + -9) - (4 * 8 + -8 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT397 |
| ✓ | BbhMultistepArithmeticTwo135BV((-2 * 4 - 3 - 3) - (-9 * -3 - -8 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT396 |
| ✓ | BbhMultistepArithmeticTwo136BV((-4 + -6 + -6 - -9) + (-2 * -5 * 5 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT330 |
| ✓ | BbhMultistepArithmeticTwo137BV((-5 * 5 + 1 - 4) + (9 * 0 - -7 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT313 |
Reasoning - BBH multi-step arithmetic (set 5)
31 passed |
100.0% |
IN30 411 OUT12 121 | |
| ✓ | BbhMultistepArithmeticTwo138BV((1 - -8 + 0 - -3) + (-3 + -4 + 1 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT374 |
| ✓ | BbhMultistepArithmeticTwo139BV((8 - -5 + -9 + 7) + (-9 * -1 * -6 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT338 |
| ✓ | BbhMultistepArithmeticTwo140BV((0 * 3 - -9 + -1) - (-5 * -6 - 6 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT420 |
| ✓ | BbhMultistepArithmeticTwo141BV((-4 + -9 + -7 * 8) - (-8 + 0 * -5 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT518 |
| ✓ | BbhMultistepArithmeticTwo142BV((7 + -7 * 9 * 5) - (-2 + -6 - -8 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT480 |
| ✓ | BbhMultistepArithmeticTwo143BV((-4 - -8 + 2 + -8) * (-3 * 0 + 4 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT350 |
| ✓ | BbhMultistepArithmeticTwo144BV((3 - 9 - 3 - 4) + (1 * -2 * -3 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT298 |
| ✓ | BbhMultistepArithmeticTwo145BV((-6 * -4 + 1 + -1) * (-1 + 2 * 7 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT449 |
| ✓ | BbhMultistepArithmeticTwo146BV((-9 * -3 - 8 - 5) + (1 * -2 - -4 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT313 |
| ✓ | BbhMultistepArithmeticTwo147BV((3 + -6 * -2 + 7) * (-6 - -4 - -6 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT415 |
| ✓ | BbhMultistepArithmeticTwo148BV((-7 * 2 + -4 * -8) - (-5 * 0 * 1 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT316 |
| ✓ | BbhMultistepArithmeticTwo149BV((-4 * 4 + -1 * 5) - (-6 + 1 - -8 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT399 |
| ✓ | BbhMultistepArithmeticTwo150BV((7 - -9 + 0 - 0) + (-9 + 5 - -9 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT437 |
| ✓ | BbhMultistepArithmeticTwo151BV((4 + 3 + -9 + -2) * (7 * 2 + 8 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT313 |
| ✓ | BbhMultistepArithmeticTwo152BV((3 - 1 + -3 * -4) - (8 * 9 - -7 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT438 |
| ✓ | BbhMultistepArithmeticTwo153BV((-6 * -7 * -8 - -2) + (-5 - 2 + 0 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT482 |
| ✓ | BbhMultistepArithmeticTwo154BV((5 * 7 * -4 + 7) * (-5 - -6 + -8 * -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT449 |
| ✓ | BbhMultistepArithmeticTwo155BV((4 - 2 * 2 + 3) + (-3 - -9 - -4 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT360 |
| ✓ | BbhMultistepArithmeticTwo156BV((8 - 7 - -3 + -9) - (-6 * -6 + -4 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT389 |
| ✓ | BbhMultistepArithmeticTwo157BV((-7 + 9 + -9 * -4) - (-7 * 6 - 7 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT402 |
| ✓ | BbhMultistepArithmeticTwo158BV((-6 * -2 + 6 - -8) - (-2 * -4 * 8 + 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT376 |
| ✓ | BbhMultistepArithmeticTwo159BV((-1 + 5 * 4 + 9) - (-7 - 6 * -8 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT404 |
| ✓ | BbhMultistepArithmeticTwo160BV((-4 + -2 + -6 + -1) + (-4 * 2 + -5 * 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT264 |
| ✓ | BbhMultistepArithmeticTwo161BV((-8 - -1 - -2 * -7) + (3 + 2 - 3 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT516 |
| ✓ | BbhMultistepArithmeticTwo162BV((4 + 5 - 7 + 9) * (-1 - -4 + -7 - -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT385 |
| ✓ | BbhMultistepArithmeticTwo163BV((7 + 7 * 3 - 3) - (-3 + 4 - -6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT390 |
| ✓ | BbhMultistepArithmeticTwo164BV((-4 + 5 - 4 - -6) * (-3 * -1 * 2 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT348 |
| ✓ | BbhMultistepArithmeticTwo165BV((2 + 7 - -5 - 1) - (0 - 5 - 9 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT344 |
| ✓ | BbhMultistepArithmeticTwo166BV((4 * -4 + -6 - -2) - (9 + -7 + -8 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT364 |
| ✓ | BbhMultistepArithmeticTwo167BV((3 - 2 - 3 * 3) + (-8 * -1 * -9 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT428 |
| ✓ | BbhMultistepArithmeticTwo168BV((-1 * -5 - -3 + -9) + (-5 - 3 * -2 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT362 |
Reasoning - BBH multi-step arithmetic (set 6)
32 passed |
100.0% |
IN31 392 OUT12 497 | |
| ✓ | BbhMultistepArithmeticTwo169BW((5 * -1 - 7 * 5) - (6 - -5 * 3 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT452 |
| ✓ | BbhMultistepArithmeticTwo170BW((9 - -5 - 1 + -2) * (-5 * 3 - -5 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT391 |
| ✓ | BbhMultistepArithmeticTwo171BW((-8 - -3 * 5 + 4) * (2 * 7 - -3 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT488 |
| ✓ | BbhMultistepArithmeticTwo172BW((5 - 7 + -6 + 4) * (2 - -9 + 3 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT306 |
| ✓ | BbhMultistepArithmeticTwo173BW((-5 - -1 - 6 * 0) + (-3 + 5 * 0 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT352 |
| ✓ | BbhMultistepArithmeticTwo174BW((-3 - 9 - 3 - 1) + (-4 * -9 * -7 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT444 |
| ✓ | BbhMultistepArithmeticTwo175BW((1 * -1 + 5 * -4) + (-2 - 5 * 4 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT341 |
| ✓ | BbhMultistepArithmeticTwo176BW((-3 - -8 - -7 + -2) - (-2 - 8 + -1 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT455 |
| ✓ | BbhMultistepArithmeticTwo177BW((-5 * -6 + 3 * -3) - (-1 + -8 * 8 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT675 |
| ✓ | BbhMultistepArithmeticTwo178BW((-5 - -7 - 4 - 8) - (-4 + 4 + 1 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT268 |
| ✓ | BbhMultistepArithmeticTwo179BW((9 + 7 - 4 - 7) - (-5 + 6 + -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT305 |
| ✓ | BbhMultistepArithmeticTwo180BW((8 * -6 + -9 + 9) * (0 - 4 - 0 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT357 |
| ✓ | BbhMultistepArithmeticTwo181BW((-4 * 1 + 8 + 9) * (7 * -6 * -5 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT538 |
| ✓ | BbhMultistepArithmeticTwo182BW((-5 - 6 * 2 + 4) - (1 - 9 - 0 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT385 |
| ✓ | BbhMultistepArithmeticTwo183BW((-7 - 1 + 3 * -2) + (0 - 4 + 6 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT328 |
| ✓ | BbhMultistepArithmeticTwo184BW((7 * 2 + 4 - -6) * (6 - 7 - 7 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT442 |
| ✓ | BbhMultistepArithmeticTwo185BW((-6 * -9 + 1 * 7) - (0 + 3 * 5 - 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT321 |
| ✓ | BbhMultistepArithmeticTwo186BW((-8 - -1 + -8 + -3) * (-4 * 6 - 2 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT488 |
| ✓ | BbhMultistepArithmeticTwo187BW((0 * -4 - -7 * 5) + (-4 * 0 + 6 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT351 |
| ✓ | BbhMultistepArithmeticTwo188BW((9 * -9 + -4 * -8) - (5 * 0 + 8 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT304 |
| ✓ | BbhMultistepArithmeticTwo189BW((-6 + -9 - -6 + -1) + (-8 * 5 * -6 + 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT317 |
| ✓ | BbhMultistepArithmeticTwo190BW((5 + 4 + 7 - 6) - (-8 * 0 - 9 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT310 |
| ✓ | BbhMultistepArithmeticTwo191BW((9 - 6 + -4 + -5) * (4 - -8 + -9 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT335 |
| ✓ | BbhMultistepArithmeticTwo192BW((-9 * 6 - -3 * -3) + (-4 - -9 + 4 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT371 |
| ✓ | BbhMultistepArithmeticTwo193BW((-8 * -8 - -5 * -6) * (3 + -9 - 5 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT447 |
| ✓ | BbhMultistepArithmeticTwo194BW((0 + 3 * 4 * 9) - (4 - -5 + -4 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT338 |
| ✓ | BbhMultistepArithmeticTwo195BW((-4 + -7 - 3 + 3) - (-3 + -2 - -3 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT351 |
| ✓ | BbhMultistepArithmeticTwo196BW((-7 * -2 * -1 + -9) - (-7 + 5 - 0 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT354 |
| ✓ | BbhMultistepArithmeticTwo197BW((-3 + 2 * -7 * 5) * (-9 - 9 - -9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT464 |
| ✓ | BbhMultistepArithmeticTwo198BW((1 + 0 + 2 - 4) + (-9 + 6 * -5 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT309 |
| ✓ | BbhMultistepArithmeticTwo199BW((-3 + -4 * 4 + -8) + (1 - -3 + 4 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT477 |
| ✓ | BbhMultistepArithmeticTwo200BW((3 * -7 * -4 * 2) + (7 - -3 - -3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT433 |
Reasoning - BBH multi-step arithmetic (set 7)
32 passed |
100.0% |
IN31 392 OUT12 269 | |
| ✓ | BbhMultistepArithmeticTwo201BW((-5 * -6 + -9 - 3) + (-7 - -2 - 6 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT381 |
| ✓ | BbhMultistepArithmeticTwo202BW((6 - 5 + 5 + -9) - (9 + 8 * -6 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT292 |
| ✓ | BbhMultistepArithmeticTwo203BW((8 * 3 - -1 - -3) * (-4 * 7 * -3 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT471 |
| ✓ | BbhMultistepArithmeticTwo204BW((5 + 3 - 6 + 1) + (-5 + 5 - 6 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT246 |
| ✓ | BbhMultistepArithmeticTwo205BW((-9 + 6 + 1 - 7) * (2 - -2 + -9 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT474 |
| ✓ | BbhMultistepArithmeticTwo206BW((1 * 0 - -5 - 3) + (9 * 6 + -4 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT334 |
| ✓ | BbhMultistepArithmeticTwo207BW((1 - 2 + 5 * 2) + (-2 - -7 + 3 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT438 |
| ✓ | BbhMultistepArithmeticTwo208BW((5 + -7 + 1 - 5) - (-3 * -1 * 7 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT387 |
| ✓ | BbhMultistepArithmeticTwo209BW((-7 * -2 + -4 + -4) - (9 * 1 * -6 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT349 |
| ✓ | BbhMultistepArithmeticTwo210BW((1 + 0 - 3 + 0) - (-7 - 9 * 1 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT339 |
| ✓ | BbhMultistepArithmeticTwo211BW((7 * 6 - -2 - 7) * (4 * 5 * -3 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT445 |
| ✓ | BbhMultistepArithmeticTwo212BW((3 * -9 * 3 - 7) - (-9 + 4 * 8 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT377 |
| ✓ | BbhMultistepArithmeticTwo213BW((9 + 6 - 9 + 8) * (9 - -9 * -6 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT465 |
| ✓ | BbhMultistepArithmeticTwo214BW((5 * 6 + -6 - 8) * (-5 - 6 - 5 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT366 |
| ✓ | BbhMultistepArithmeticTwo215BW((5 + 4 - -1 * -1) + (-6 - -7 + -1 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT428 |
| ✓ | BbhMultistepArithmeticTwo216BW((9 - 5 - 4 + -2) + (-9 + 1 - 1 - 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT268 |
| ✓ | BbhMultistepArithmeticTwo217BW((6 - 9 * -6 - -9) + (3 * 3 + -5 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT419 |
| ✓ | BbhMultistepArithmeticTwo218BW((-2 * 5 + -3 + 0) * (-9 + 2 + 5 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT364 |
| ✓ | BbhMultistepArithmeticTwo219BW((5 * -3 * 3 * 7) * (0 + -9 - -5 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT359 |
| ✓ | BbhMultistepArithmeticTwo220BW((8 - 3 + 0 + 7) + (-9 * -6 * 2 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT288 |
| ✓ | BbhMultistepArithmeticTwo221BW((-2 * 9 * -8 + 8) - (0 * -4 - -6 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT312 |
| ✓ | BbhMultistepArithmeticTwo222BW((-7 * -6 * 0 - 5) * (-3 - 9 * -8 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT335 |
| ✓ | BbhMultistepArithmeticTwo223BW((3 * 8 * -1 * -9) - (-2 + 2 * -8 * 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT450 |
| ✓ | BbhMultistepArithmeticTwo224BW((-9 + 5 * 0 - -8) + (6 - -5 * -7 + -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT507 |
| ✓ | BbhMultistepArithmeticTwo225BW((-2 - -7 - 5 * 2) + (-3 - 5 * 3 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT447 |
| ✓ | BbhMultistepArithmeticTwo226BW((-9 + -7 + -1 - -1) * (3 - -7 - -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT400 |
| ✓ | BbhMultistepArithmeticTwo227BW((-1 + 6 + 9 + 3) - (-5 - 9 - -3 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT300 |
| ✓ | BbhMultistepArithmeticTwo228BW((-7 + 9 * 9 * 6) - (-2 - -5 + 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT371 |
| ✓ | BbhMultistepArithmeticTwo229BW((-8 - -7 + 0 + 3) + (-3 + 5 - -5 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT414 |
| ✓ | BbhMultistepArithmeticTwo230BW((8 - 4 - -8 - -7) + (6 - 5 - -1 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT392 |
| ✓ | BbhMultistepArithmeticTwo231BW((-6 * 8 + -6 - -5) + (-4 - 9 * 5 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT524 |
| ✓ | BbhMultistepArithmeticTwo232BW((5 * -9 - 4 - 4) - (-3 * 2 * 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT327 |
Reasoning - BBH multi-step arithmetic (set 8)
32 passed |
100.0% |
IN31 751 OUT14 977 | |
| ✓ | BbhMultistepArithmeticTwo233BW((4 * -9 * 3 * -8) + (-9 - 4 - -9 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT406 |
| ✓ | BbhMultistepArithmeticTwo234BW((-7 - 2 - 0 + 2) * (-4 * -5 + -8 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT331 |
| ✓ | BbhMultistepArithmeticTwo235BW((-1 + 7 * 7 * -9) * (4 + 3 + -8 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT479 |
| ✓ | BbhMultistepArithmeticTwo236BW((7 * 8 + 1 + 3) - (3 + -1 - -9 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT380 |
| ✓ | BbhMultistepArithmeticTwo237BW((3 + -9 + 8 + -4) * (-4 + -6 - 4 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT283 |
| ✓ | BbhMultistepArithmeticTwo238BW((1 * 5 * -8 - -2) * (-2 * 6 - -5 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT391 |
| ✓ | BbhMultistepArithmeticTwo239BW((-8 - -1 * -8 * -2) * (-6 + 8 + -5 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT448 |
| ✓ | BbhMultistepArithmeticTwo240BW((4 - -6 + -9 + -7) - (9 + -3 - -7 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT490 |
| ✓ | BbhMultistepArithmeticTwo241BW((0 * -7 - 6 * 3) * (-9 + -5 - 2 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT480 |
| ✓ | BbhMultistepArithmeticTwo242BW((-8 * 1 * -1 + -5) + (-2 - -3 + 1 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT322 |
| ✓ | BbhMultistepArithmeticTwo243BW((8 - -9 * 4 * 9) * (-4 * -3 * 7 * 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 153 |
| ✓ | BbhMultistepArithmeticTwo244BW((1 * -9 - 4 + -9) + (-1 + -6 + 9 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT293 |
| ✓ | BbhMultistepArithmeticTwo245BW((-1 + 2 - 9 + 0) * (0 - -2 + -6 - 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT400 |
| ✓ | BbhMultistepArithmeticTwo246BW((7 * -5 * -9 + 7) - (2 + -8 * 6 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT409 |
| ✓ | BbhMultistepArithmeticTwo247BW((-5 + -8 - -6 * 7) + (9 * -7 - -5 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT413 |
| ✓ | BbhMultistepArithmeticTwo248BW((-6 * 9 - -5 + 8) * (-8 * -6 - -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT391 |
| ✓ | BbhMultistepArithmeticTwo249BW((5 - -2 * 7 + -2) - (0 * 9 * -6 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT373 |
| ✓ | BbhNavigate10BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT796 |
| ✓ | BbhNavigate11BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT612 |
| ✓ | BbhNavigate12BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT211 |
| ✓ | BbhNavigate13BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT325 |
| ✓ | BbhNavigate14BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT145 |
| ✓ | BbhNavigate15BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT324 |
| ✓ | BbhNavigate16BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT141 |
| ✓ | BbhNavigate17BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT430 |
| ✓ | BbhNavigate18BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT1 044 |
| ✓ | BbhNavigate19BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT467 |
| ✓ | BbhNavigate20BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT456 |
| ✓ | BbhNavigate21BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT227 |
| ✓ | BbhNavigate22BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT673 |
| ✓ | BbhNavigate8BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT1 252 |
| ✓ | BbhNavigate9BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT432 |
Reasoning - BBH navigation - returns to start? (set 1)
32 passed |
100.0% |
IN32 212 OUT10 072 | |
| ✓ | BbhNavigate23BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT258 |
| ✓ | BbhNavigate24BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT190 |
| ✓ | BbhNavigate25BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT490 |
| ✓ | BbhNavigate26BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT308 |
| ✓ | BbhNavigate27BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT270 |
| ✓ | BbhNavigate28BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT256 |
| ✓ | BbhNavigate29BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT134 |
| ✓ | BbhNavigate30BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT365 |
| ✓ | BbhNavigate31BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT322 |
| ✓ | BbhNavigate32BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT176 |
| ✓ | BbhNavigate33BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT446 |
| ✓ | BbhNavigate34BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT400 |
| ✓ | BbhNavigate35BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT196 |
| ✓ | BbhNavigate36BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT327 |
| ✓ | BbhNavigate37BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT208 |
| ✓ | BbhNavigate38BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT202 |
| ✓ | BbhNavigate39BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT113 |
| ✓ | BbhNavigate40BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT207 |
| ✓ | BbhNavigate41BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT618 |
| ✓ | BbhNavigate42BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT420 |
| ✓ | BbhNavigate43BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT273 |
| ✓ | BbhNavigate44BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT166 |
| ✓ | BbhNavigate45BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT245 |
| ✓ | BbhNavigate46BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT511 |
| ✓ | BbhNavigate47BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT621 |
| ✓ | BbhNavigate48BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT173 |
| ✓ | BbhNavigate49BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT307 |
| ✓ | BbhNavigate50BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT258 |
| ✓ | BbhNavigate51BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT116 |
| ✓ | BbhNavigate52BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT715 |
| ✓ | BbhNavigate53BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT536 |
| ✓ | BbhNavigate54BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT245 |
Reasoning - BBH navigation - returns to start? (set 2)
30 passed |
100.0% |
IN30 187 OUT10 148 | |
| ✓ | BbhNavigate55BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT228 |
| ✓ | BbhNavigate56BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT330 |
| ✓ | BbhNavigate57BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT449 |
| ✓ | BbhNavigate58BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT398 |
| ✓ | BbhNavigate59BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT166 |
| ✓ | BbhNavigate60BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT988 |
| ✓ | BbhNavigate61BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT465 |
| ✓ | BbhNavigate62BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT523 |
| ✓ | BbhNavigate63BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT385 |
| ✓ | BbhNavigate64BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT204 |
| ✓ | BbhNavigate65BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT123 |
| ✓ | BbhNavigate66BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT370 |
| ✓ | BbhNavigate67BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT152 |
| ✓ | BbhNavigate68BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT379 |
| ✓ | BbhNavigate69BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT118 |
| ✓ | BbhNavigate70BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT112 |
| ✓ | BbhNavigate71BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT139 |
| ✓ | BbhNavigate72BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT248 |
| ✓ | BbhNavigate73BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT111 |
| ✓ | BbhNavigate74BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT515 |
| ✓ | BbhNavigate75BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT149 |
| ✓ | BbhNavigate76BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT842 |
| ✓ | BbhNavigate77BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT441 |
| ✓ | BbhNavigate78BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT303 |
| ✓ | BbhNavigate79BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT130 |
| ✓ | BbhNavigate80BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT647 |
| ✓ | BbhNavigate81BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT282 |
| ✓ | BbhNavigate82BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT233 |
| ✓ | BbhNavigate83BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT360 |
| ✓ | BbhNavigate84BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT358 |
Reasoning - BBH navigation - returns to start? (set 3)
32 passed |
100.0% |
IN32 213 OUT11 184 | |
| ✓ | BbhNavigate100CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT370 |
| ✓ | BbhNavigate101CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT352 |
| ✓ | BbhNavigate102CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT177 |
| ✓ | BbhNavigate103CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT367 |
| ✓ | BbhNavigate104CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT297 |
| ✓ | BbhNavigate105CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT390 |
| ✓ | BbhNavigate106CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT325 |
| ✓ | BbhNavigate107CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT145 |
| ✓ | BbhNavigate108CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT498 |
| ✓ | BbhNavigate109CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT448 |
| ✓ | BbhNavigate110CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT263 |
| ✓ | BbhNavigate111CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT401 |
| ✓ | BbhNavigate112CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT716 |
| ✓ | BbhNavigate113CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT284 |
| ✓ | BbhNavigate114CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT164 |
| ✓ | BbhNavigate115CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT269 |
| ✓ | BbhNavigate116CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT158 |
| ✓ | BbhNavigate85CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT410 |
| ✓ | BbhNavigate86CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT295 |
| ✓ | BbhNavigate87CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT262 |
| ✓ | BbhNavigate88CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT527 |
| ✓ | BbhNavigate89CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT192 |
| ✓ | BbhNavigate90CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT467 |
| ✓ | BbhNavigate91CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT238 |
| ✓ | BbhNavigate92CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT303 |
| ✓ | BbhNavigate93CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT403 |
| ✓ | BbhNavigate94CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT476 |
| ✓ | BbhNavigate95CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT629 |
| ✓ | BbhNavigate96CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT315 |
| ✓ | BbhNavigate97CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT393 |
| ✓ | BbhNavigate98CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT483 |
| ✓ | BbhNavigate99CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT167 |
Reasoning - BBH navigation - returns to start? (set 4)
32 passed |
100.0% |
IN32 201 OUT10 315 | |
| ✓ | BbhNavigate117CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT348 |
| ✓ | BbhNavigate118CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT160 |
| ✓ | BbhNavigate119CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT131 |
| ✓ | BbhNavigate120CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT238 |
| ✓ | BbhNavigate121CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT177 |
| ✓ | BbhNavigate122CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT271 |
| ✓ | BbhNavigate123CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT795 |
| ✓ | BbhNavigate124CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT416 |
| ✓ | BbhNavigate125CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT193 |
| ✓ | BbhNavigate126CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT431 |
| ✓ | BbhNavigate127CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT519 |
| ✓ | BbhNavigate128CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT178 |
| ✓ | BbhNavigate129CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT295 |
| ✓ | BbhNavigate130CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT240 |
| ✓ | BbhNavigate131CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT783 |
| ✓ | BbhNavigate132CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT312 |
| ✓ | BbhNavigate133CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT404 |
| ✓ | BbhNavigate134CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT139 |
| ✓ | BbhNavigate135CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT238 |
| ✓ | BbhNavigate136CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT480 |
| ✓ | BbhNavigate137CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT378 |
| ✓ | BbhNavigate138CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT383 |
| ✓ | BbhNavigate139CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT596 |
| ✓ | BbhNavigate140CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT265 |
| ✓ | BbhNavigate141CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT175 |
| ✓ | BbhNavigate142CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT272 |
| ✓ | BbhNavigate143CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT326 |
| ✓ | BbhNavigate144CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT255 |
| ✓ | BbhNavigate145CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT173 |
| ✓ | BbhNavigate146CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT245 |
| ✓ | BbhNavigate147CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT210 |
| ✓ | BbhNavigate148CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT289 |
Reasoning - BBH navigation - returns to start? (set 5)
32 passed |
100.0% |
IN32 138 OUT11 321 | |
| ✓ | BbhNavigate149CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT435 |
| ✓ | BbhNavigate150CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT269 |
| ✓ | BbhNavigate151CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT177 |
| ✓ | BbhNavigate152CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT140 |
| ✓ | BbhNavigate153CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT303 |
| ✓ | BbhNavigate154CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT394 |
| ✓ | BbhNavigate155CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT152 |
| ✓ | BbhNavigate156CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT262 |
| ✓ | BbhNavigate157CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT268 |
| ✓ | BbhNavigate158CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT645 |
| ✓ | BbhNavigate159CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT235 |
| ✓ | BbhNavigate160CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT176 |
| ✓ | BbhNavigate161CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT167 |
| ✓ | BbhNavigate162CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT484 |
| ✓ | BbhNavigate163CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT610 |
| ✓ | BbhNavigate164CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT225 |
| ✓ | BbhNavigate165CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT207 |
| ✓ | BbhNavigate166CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT745 |
| ✓ | BbhNavigate167CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT898 |
| ✓ | BbhNavigate168CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT297 |
| ✓ | BbhNavigate169CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT163 |
| ✓ | BbhNavigate170CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT185 |
| ✓ | BbhNavigate171CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT386 |
| ✓ | BbhNavigate172CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT405 |
| ✓ | BbhNavigate173CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT430 |
| ✓ | BbhNavigate174CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT285 |
| ✓ | BbhNavigate175CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT347 |
| ✓ | BbhNavigate176CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT379 |
| ✓ | BbhNavigate177CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT526 |
| ✓ | BbhNavigate178CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT272 |
| ✓ | BbhNavigate179CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT514 |
| ✓ | BbhNavigate180CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT340 |
Reasoning - BBH navigation - returns to start? (set 6)
32 passed |
100.0% |
IN32 216 OUT12 255 | |
| ✓ | BbhNavigate181CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT630 |
| ✓ | BbhNavigate182CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT337 |
| ✓ | BbhNavigate183CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT225 |
| ✓ | BbhNavigate184CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT519 |
| ✓ | BbhNavigate185CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT273 |
| ✓ | BbhNavigate186CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 466 |
| ✓ | BbhNavigate187CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT541 |
| ✓ | BbhNavigate188CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT519 |
| ✓ | BbhNavigate189CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT328 |
| ✓ | BbhNavigate190CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT216 |
| ✓ | BbhNavigate191CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT216 |
| ✓ | BbhNavigate192CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT525 |
| ✓ | BbhNavigate193CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT315 |
| ✓ | BbhNavigate194CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT222 |
| ✓ | BbhNavigate195CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT180 |
| ✓ | BbhNavigate196CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT489 |
| ✓ | BbhNavigate197CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT428 |
| ✓ | BbhNavigate198CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT352 |
| ✓ | BbhNavigate199CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT323 |
| ✓ | BbhNavigate200CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT317 |
| ✓ | BbhNavigate201CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT404 |
| ✓ | BbhNavigate202CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT221 |
| ✓ | BbhNavigate203CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT333 |
| ✓ | BbhNavigate204CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT330 |
| ✓ | BbhNavigate205CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT312 |
| ✓ | BbhNavigate206CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT250 |
| ✓ | BbhNavigate207CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT482 |
| ✓ | BbhNavigate208CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT291 |
| ✓ | BbhNavigate209CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT189 |
| ✓ | BbhNavigate210CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT135 |
| ✓ | BbhNavigate211CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT343 |
| ✓ | BbhNavigate212CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT544 |
Reasoning - BBH navigation - returns to start? (set 7)
30 passed |
100.0% |
IN30 113 OUT9 790 | |
| ✓ | BbhNavigate213CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT364 |
| ✓ | BbhNavigate214CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT390 |
| ✓ | BbhNavigate215CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT275 |
| ✓ | BbhNavigate216CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT300 |
| ✓ | BbhNavigate217CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT494 |
| ✓ | BbhNavigate218CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT486 |
| ✓ | BbhNavigate219CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT368 |
| ✓ | BbhNavigate220CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT304 |
| ✓ | BbhNavigate221CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT188 |
| ✓ | BbhNavigate222CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT227 |
| ✓ | BbhNavigate223CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT680 |
| ✓ | BbhNavigate224CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT226 |
| ✓ | BbhNavigate225CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT222 |
| ✓ | BbhNavigate226CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT228 |
| ✓ | BbhNavigate227CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT371 |
| ✓ | BbhNavigate228CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT144 |
| ✓ | BbhNavigate229CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT711 |
| ✓ | BbhNavigate230CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT194 |
| ✓ | BbhNavigate231CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT402 |
| ✓ | BbhNavigate232CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT216 |
| ✓ | BbhNavigate233CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT112 |
| ✓ | BbhNavigate234CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT483 |
| ✓ | BbhNavigate235CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT321 |
| ✓ | BbhNavigate236CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT275 |
| ✓ | BbhNavigate237CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT142 |
| ✓ | BbhNavigate238CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT514 |
| ✓ | BbhNavigate239CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT383 |
| ✓ | BbhNavigate240CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT151 |
| ✓ | BbhNavigate241CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT304 |
| ✓ | BbhNavigate242CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT315 |
Reasoning - BBH object counting (set 1)
32 passed |
100.0% |
IN31 770 OUT5 578 | |
| ✓ | BbhNavigate243CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 036 OUT452 |
| ✓ | BbhNavigate244CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT438 |
| ✓ | BbhNavigate245CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT322 |
| ✓ | BbhNavigate246CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT155 |
| ✓ | BbhNavigate247CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT326 |
| ✓ | BbhNavigate248CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT447 |
| ✓ | BbhNavigate249CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT304 |
| ✓ | BbhObjectCounting10CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT114 |
| ✓ | BbhObjectCounting11CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT237 |
| ✓ | BbhObjectCounting12CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT56 |
| ✓ | BbhObjectCounting13CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT109 |
| ✓ | BbhObjectCounting14CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT110 |
| ✓ | BbhObjectCounting15CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT161 |
| ✓ | BbhObjectCounting16CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT104 |
| ✓ | BbhObjectCounting17CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT69 |
| ✓ | BbhObjectCounting18CBI have a blackberry, a plum, and a banana. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT43 |
| ✓ | BbhObjectCounting19CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT102 |
| ✓ | BbhObjectCounting20CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT48 |
| ✓ | BbhObjectCounting21CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT165 |
| ✓ | BbhObjectCounting22CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT86 |
| ✓ | BbhObjectCounting23CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT303 |
| ✓ | BbhObjectCounting24CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT103 |
| ✓ | BbhObjectCounting25CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT229 |
| ✓ | BbhObjectCounting26CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT40 |
| ✓ | BbhObjectCounting27CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT59 |
| ✓ | BbhObjectCounting28CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT67 |
| ✓ | BbhObjectCounting29CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT239 |
| ✓ | BbhObjectCounting30CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT236 |
| ✓ | BbhObjectCounting31CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT81 |
| ✓ | BbhObjectCounting32CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT119 |
| ✓ | BbhObjectCounting8CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT134 |
| ✓ | BbhObjectCounting9CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT120 |
Reasoning - BBH object counting (set 2)
32 passed |
100.0% |
IN31 581 OUT4 307 | |
| ✓ | BbhObjectCounting33CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT35 |
| ✓ | BbhObjectCounting34CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT36 |
| ✓ | BbhObjectCounting35CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT59 |
| ✓ | BbhObjectCounting36CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT81 |
| ✓ | BbhObjectCounting37CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT81 |
| ✓ | BbhObjectCounting38CBI have a raspberry, a grape, and an orange. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT42 |
| ✓ | BbhObjectCounting39CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT181 |
| ✓ | BbhObjectCounting40CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT108 |
| ✓ | BbhObjectCounting41CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT55 |
| ✓ | BbhObjectCounting42CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT108 |
| ✓ | BbhObjectCounting43CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT128 |
| ✓ | BbhObjectCounting44CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT56 |
| ✓ | BbhObjectCounting45CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT112 |
| ✓ | BbhObjectCounting46CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT84 |
| ✓ | BbhObjectCounting47CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT180 |
| ✓ | BbhObjectCounting48CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT113 |
| ✓ | BbhObjectCounting49CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT391 |
| ✓ | BbhObjectCounting50CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT169 |
| ✓ | BbhObjectCounting51CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT314 |
| ✓ | BbhObjectCounting52CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT253 |
| ✓ | BbhObjectCounting53CBI have a frog, and a mouse. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT38 |
| ✓ | BbhObjectCounting54CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT134 |
| ✓ | BbhObjectCounting55CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT236 |
| ✓ | BbhObjectCounting56CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT224 |
| ✓ | BbhObjectCounting57CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT122 |
| ✓ | BbhObjectCounting58CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT137 |
| ✓ | BbhObjectCounting59CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT103 |
| ✓ | BbhObjectCounting60CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT343 |
| ✓ | BbhObjectCounting61CBI have a rabbit, a frog, and a snail. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT43 |
| ✓ | BbhObjectCounting62CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT153 |
| ✓ | BbhObjectCounting63CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT51 |
| ✓ | BbhObjectCounting64CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT137 |
Reasoning - BBH object counting (set 3)
32 passed |
100.0% |
IN31 651 OUT4 969 | |
| ✓ | BbhObjectCounting65CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT105 |
| ✓ | BbhObjectCounting66CBI have a grape, and a plum. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT44 |
| ✓ | BbhObjectCounting67CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT91 |
| ✓ | BbhObjectCounting68CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT51 |
| ✓ | BbhObjectCounting69CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT47 |
| ✓ | BbhObjectCounting70CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT357 |
| ✓ | BbhObjectCounting71CBI have a dog, a pig, and a snail. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT73 |
| ✓ | BbhObjectCounting72CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT117 |
| ✓ | BbhObjectCounting73CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT122 |
| ✓ | BbhObjectCounting74CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT272 |
| ✓ | BbhObjectCounting75CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT326 |
| ✓ | BbhObjectCounting76CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT83 |
| ✓ | BbhObjectCounting77CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT93 |
| ✓ | BbhObjectCounting78CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT116 |
| ✓ | BbhObjectCounting79CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT271 |
| ✓ | BbhObjectCounting80CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT105 |
| ✓ | BbhObjectCounting81CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT86 |
| ✓ | BbhObjectCounting82CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT329 |
| ✓ | BbhObjectCounting83CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT321 |
| ✓ | BbhObjectCounting84CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT65 |
| ✓ | BbhObjectCounting85CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT262 |
| ✓ | BbhObjectCounting86CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT199 |
| ✓ | BbhObjectCounting87CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT122 |
| ✓ | BbhObjectCounting88CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT213 |
| ✓ | BbhObjectCounting89CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT85 |
| ✓ | BbhObjectCounting90CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT174 |
| ✓ | BbhObjectCounting91CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT129 |
| ✓ | BbhObjectCounting92CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT233 |
| ✓ | BbhObjectCounting93CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT77 |
| ✓ | BbhObjectCounting94CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT240 |
| ✓ | BbhObjectCounting95CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT92 |
| ✓ | BbhObjectCounting96CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT69 |
Reasoning - BBH object counting (set 4)
31 passed |
100.0% |
IN30 522 OUT3 764 | |
| ✓ | BbhObjectCounting100CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT167 |
| ✓ | BbhObjectCounting101CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT187 |
| ✓ | BbhObjectCounting102CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT81 |
| ✓ | BbhObjectCounting103CBI have a table, and three cars. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT71 |
| ✓ | BbhObjectCounting104CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT135 |
| ✓ | BbhObjectCounting105CBI have a microwave, a bed, and a couch. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT41 |
| ✓ | BbhObjectCounting106CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT36 |
| ✓ | BbhObjectCounting107CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT325 |
| ✓ | BbhObjectCounting108CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT224 |
| ✓ | BbhObjectCounting109CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT188 |
| ✓ | BbhObjectCounting110CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT56 |
| ✓ | BbhObjectCounting111CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT52 |
| ✓ | BbhObjectCounting112CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT153 |
| ✓ | BbhObjectCounting113CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT79 |
| ✓ | BbhObjectCounting114CBI have a donkey, a goat, and a cow. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT47 |
| ✓ | BbhObjectCounting115CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT55 |
| ✓ | BbhObjectCounting116CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT120 |
| ✓ | BbhObjectCounting117CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT74 |
| ✓ | BbhObjectCounting118CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT109 |
| ✓ | BbhObjectCounting119CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT244 |
| ✓ | BbhObjectCounting120CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT143 |
| ✓ | BbhObjectCounting121CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT48 |
| ✓ | BbhObjectCounting122CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT124 |
| ✓ | BbhObjectCounting123CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT181 |
| ✓ | BbhObjectCounting124CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT122 |
| ✓ | BbhObjectCounting125CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT104 |
| ✓ | BbhObjectCounting126CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT68 |
| ✓ | BbhObjectCounting127CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT72 |
| ✓ | BbhObjectCounting97CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT279 |
| ✓ | BbhObjectCounting98CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT107 |
| ✓ | BbhObjectCounting99CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT72 |
Reasoning - BBH object counting (set 5)
31 passed |
100.0% |
IN30 467 OUT3 191 | |
| ✓ | BbhObjectCounting128CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT61 |
| ✓ | BbhObjectCounting129CBI have two grapes, and a plum. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT51 |
| ✓ | BbhObjectCounting130CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT45 |
| ✓ | BbhObjectCounting131CBI have a pig, and a donkey. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT53 |
| ✓ | BbhObjectCounting132CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT128 |
| ✓ | BbhObjectCounting133CBI have a microwave, and a bed. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT39 |
| ✓ | BbhObjectCounting134CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT80 |
| ✓ | BbhObjectCounting135CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT247 |
| ✓ | BbhObjectCounting136CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT56 |
| ✓ | BbhObjectCounting137CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT117 |
| ✓ | BbhObjectCounting138CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT43 |
| ✓ | BbhObjectCounting139CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT250 |
| ✓ | BbhObjectCounting140CBI have a chair, and two microwaves. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT49 |
| ✓ | BbhObjectCounting141CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT108 |
| ✓ | BbhObjectCounting142CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT61 |
| ✓ | BbhObjectCounting143CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT136 |
| ✓ | BbhObjectCounting144CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT183 |
| ✓ | BbhObjectCounting145CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT230 |
| ✓ | BbhObjectCounting146CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT106 |
| ✓ | BbhObjectCounting147CBI have a cat, and a fish. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT58 |
| ✓ | BbhObjectCounting148CBI have a head of broccoli, and a yam. How many vegetables do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT57 |
| ✓ | BbhObjectCounting149CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT95 |
| ✓ | BbhObjectCounting150CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT101 |
| ✓ | BbhObjectCounting151CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT37 |
| ✓ | BbhObjectCounting152CBI have four snails, and a cow. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT64 |
| ✓ | BbhObjectCounting153CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT103 |
| ✓ | BbhObjectCounting154CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT45 |
| ✓ | BbhObjectCounting155CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT81 |
| ✓ | BbhObjectCounting156CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT210 |
| ✓ | BbhObjectCounting157CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT95 |
| ✓ | BbhObjectCounting158CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT202 |
Reasoning - BBH object counting (set 6)
32 passed |
100.0% |
IN31 544 OUT4 448 | |
| ✓ | BbhObjectCounting159CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT215 |
| ✓ | BbhObjectCounting160CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT67 |
| ✓ | BbhObjectCounting161CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT129 |
| ✓ | BbhObjectCounting162CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT82 |
| ✓ | BbhObjectCounting163CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT349 |
| ✓ | BbhObjectCounting164CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT63 |
| ✓ | BbhObjectCounting165CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT218 |
| ✓ | BbhObjectCounting166CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT143 |
| ✓ | BbhObjectCounting167CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT288 |
| ✓ | BbhObjectCounting168CCI have a toaster, a table, and a fridge. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT35 |
| ✓ | BbhObjectCounting169CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT79 |
| ✓ | BbhObjectCounting170CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT113 |
| ✓ | BbhObjectCounting171CCI have a garlic, and an onion. How many vegetables do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT52 |
| ✓ | BbhObjectCounting172CCI have a grape, a banana, and a blackberry. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT39 |
| ✓ | BbhObjectCounting173CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT256 |
| ✓ | BbhObjectCounting174CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT289 |
| ✓ | BbhObjectCounting175CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT365 |
| ✓ | BbhObjectCounting176CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT65 |
| ✓ | BbhObjectCounting177CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT87 |
| ✓ | BbhObjectCounting178CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT38 |
| ✓ | BbhObjectCounting179CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT444 |
| ✓ | BbhObjectCounting180CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT63 |
| ✓ | BbhObjectCounting181CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT140 |
| ✓ | BbhObjectCounting182CCI have a microwave, and a chair. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT47 |
| ✓ | BbhObjectCounting183CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT71 |
| ✓ | BbhObjectCounting184CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT107 |
| ✓ | BbhObjectCounting185CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT99 |
| ✓ | BbhObjectCounting186CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT232 |
| ✓ | BbhObjectCounting187CCI have a grape, an apple, and a banana. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT43 |
| ✓ | BbhObjectCounting188CCI have two blackberries, and a grape. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT57 |
| ✓ | BbhObjectCounting189CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT92 |
| ✓ | BbhObjectCounting190CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT81 |
Reasoning - BBH object counting (set 7)
30 passed2 failed |
93.8% |
IN31 621 OUT4 462 | |
| ✓ | BbhObjectCounting191CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT180 |
| ✓ | BbhObjectCounting192CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT48 |
| ✓ | BbhObjectCounting193CCI have a dog, a chicken, and four snails. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT150 |
| ✓ | BbhObjectCounting194CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT136 |
| ✓ | BbhObjectCounting195CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT246 |
| ✓ | BbhObjectCounting196CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT49 |
| ✓ | BbhObjectCounting197CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT272 |
| ✓ | BbhObjectCounting198CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT152 |
| ✓ | BbhObjectCounting199CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT273 |
| ✓ | BbhObjectCounting200CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT70 |
| ✓ | BbhObjectCounting201CCI have an apple, and two blackberries. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT33 |
| ✓ | BbhObjectCounting202CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT79 |
| ✓ | BbhObjectCounting203CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT248 |
| ✓ | BbhObjectCounting204CCI have two lamps, and two microwaves. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT54 |
| ✓ | BbhObjectCounting205CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT60 |
| ✓ | BbhObjectCounting206CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT168 |
| ✓ | BbhObjectCounting207CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT38 |
| × | BbhObjectCounting208CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "10")
Context: BbhObjectCounting208CC: expected 10, got [9] |
Failed | IN989 OUT137 |
| ✓ | BbhObjectCounting209CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT304 |
| ✓ | BbhObjectCounting210CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT138 |
| ✓ | BbhObjectCounting211CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT289 |
| ✓ | BbhObjectCounting212CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT92 |
| ✓ | BbhObjectCounting213CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT62 |
| ✓ | BbhObjectCounting214CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT168 |
| ✓ | BbhObjectCounting215CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT91 |
| ✓ | BbhObjectCounting216CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT128 |
| ✓ | BbhObjectCounting217CCI have a toaster, a car, and a table. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT35 |
| × | BbhObjectCounting218CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "11")
Context: BbhObjectCounting218CC: expected 11, got [10] |
Failed | IN1 005 OUT359 |
| ✓ | BbhObjectCounting219CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT137 |
| ✓ | BbhObjectCounting220CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT52 |
| ✓ | BbhObjectCounting221CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT144 |
| ✓ | BbhObjectCounting222CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT70 |
Reasoning - BBH object counting (set 8)
27 passed |
100.0% |
IN26 689 OUT3 969 | |
| ✓ | BbhObjectCounting223CCI have an oven, a bed, and a toaster. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT36 |
| ✓ | BbhObjectCounting224CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT423 |
| ✓ | BbhObjectCounting225CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT53 |
| ✓ | BbhObjectCounting226CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT97 |
| ✓ | BbhObjectCounting227CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT102 |
| ✓ | BbhObjectCounting228CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT97 |
| ✓ | BbhObjectCounting229CCI have two cars, and an oven. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT48 |
| ✓ | BbhObjectCounting230CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT125 |
| ✓ | BbhObjectCounting231CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT251 |
| ✓ | BbhObjectCounting232CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT120 |
| ✓ | BbhObjectCounting233CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT184 |
| ✓ | BbhObjectCounting234CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT44 |
| ✓ | BbhObjectCounting235CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT255 |
| ✓ | BbhObjectCounting236CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT73 |
| ✓ | BbhObjectCounting237CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT264 |
| ✓ | BbhObjectCounting238CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT60 |
| ✓ | BbhObjectCounting239CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT270 |
| ✓ | BbhObjectCounting240CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT337 |
| ✓ | BbhObjectCounting241CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT57 |
| ✓ | BbhObjectCounting242CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT91 |
| ✓ | BbhObjectCounting243CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT129 |
| ✓ | BbhObjectCounting244CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT140 |
| ✓ | BbhObjectCounting245CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT203 |
| ✓ | BbhObjectCounting246CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT65 |
| ✓ | BbhObjectCounting247CCI have a fish, a snail, and two rabbits. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT44 |
| ✓ | BbhObjectCounting248CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT86 |
| ✓ | BbhObjectCounting249CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT315 |
Reasoning - BBH penguins in a table (set 1)
33 passed |
100.0% |
IN38 044 OUT9 172 | |
| ✓ | BbhPenguinsInATable10CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT245 |
| ✓ | BbhPenguinsInATable11CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 203 OUT196 |
| ✓ | BbhPenguinsInATable12CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT341 |
| ✓ | BbhPenguinsInATable13CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 226 OUT220 |
| ✓ | BbhPenguinsInATable14CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT218 |
| ✓ | BbhPenguinsInATable15CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT555 |
| ✓ | BbhPenguinsInATable16CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 228 OUT307 |
| ✓ | BbhPenguinsInATable17CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT148 |
| ✓ | BbhPenguinsInATable18CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT382 |
| ✓ | BbhPenguinsInATable19CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT227 |
| ✓ | BbhPenguinsInATable20CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT218 |
| ✓ | BbhPenguinsInATable21CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT303 |
| ✓ | BbhPenguinsInATable22CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT189 |
| ✓ | BbhPenguinsInATable23CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT495 |
| ✓ | BbhPenguinsInATable24CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT240 |
| ✓ | BbhPenguinsInATable25CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT283 |
| ✓ | BbhPenguinsInATable26CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 220 OUT147 |
| ✓ | BbhPenguinsInATable27CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT458 |
| ✓ | BbhPenguinsInATable28CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT259 |
| ✓ | BbhPenguinsInATable29CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 222 OUT207 |
| ✓ | BbhPenguinsInATable30CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 220 OUT282 |
| ✓ | BbhPenguinsInATable31CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 227 OUT195 |
| ✓ | BbhPenguinsInATable32CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 195 OUT217 |
| ✓ | BbhPenguinsInATable33CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 197 OUT217 |
| ✓ | BbhPenguinsInATable34CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT547 |
| ✓ | BbhPenguinsInATable35CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT131 |
| ✓ | BbhPenguinsInATable36CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT272 |
| ✓ | BbhPenguinsInATable37CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT184 |
| ✓ | BbhPenguinsInATable38CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT219 |
| ✓ | BbhPenguinsInATable39CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT317 |
| ✓ | BbhPenguinsInATable40CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT539 |
| ✓ | BbhPenguinsInATable8CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT142 |
| ✓ | BbhPenguinsInATable9CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT272 |
Reasoning - BBH penguins in a table (set 2)
34 passed |
100.0% |
IN38 697 OUT8 185 | |
| ✓ | BbhPenguinsInATable41CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT261 |
| ✓ | BbhPenguinsInATable42CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT260 |
| ✓ | BbhPenguinsInATable43CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 226 OUT365 |
| ✓ | BbhPenguinsInATable44CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT237 |
| ✓ | BbhPenguinsInATable45CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT154 |
| ✓ | BbhPenguinsInATable46CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 219 OUT176 |
| ✓ | BbhPenguinsInATable47CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT192 |
| ✓ | BbhPenguinsInATable48CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT279 |
| ✓ | BbhPenguinsInATable49CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT342 |
| ✓ | BbhPenguinsInATable50CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT237 |
| ✓ | BbhPenguinsInATable51CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT170 |
| ✓ | BbhPenguinsInATable52CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT234 |
| ✓ | BbhPenguinsInATable53CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT182 |
| ✓ | BbhPenguinsInATable54CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT216 |
| ✓ | BbhPenguinsInATable55CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT273 |
| ✓ | BbhPenguinsInATable56CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT242 |
| ✓ | BbhPenguinsInATable57CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT271 |
| ✓ | BbhPenguinsInATable58CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT138 |
| ✓ | BbhPenguinsInATable59CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT158 |
| ✓ | BbhPenguinsInATable60CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT120 |
| ✓ | BbhPenguinsInATable61CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT211 |
| ✓ | BbhPenguinsInATable62CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT175 |
| ✓ | BbhPenguinsInATable63CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT193 |
| ✓ | BbhPenguinsInATable64CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 230 OUT437 |
| ✓ | BbhPenguinsInATable65CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT314 |
| ✓ | BbhPenguinsInATable66CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT278 |
| ✓ | BbhPenguinsInATable67CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT239 |
| ✓ | BbhPenguinsInATable68CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT541 |
| ✓ | BbhPenguinsInATable69CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT209 |
| ✓ | BbhPenguinsInATable70CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT183 |
| ✓ | BbhPenguinsInATable71CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT302 |
| ✓ | BbhPenguinsInATable72CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT188 |
| ✓ | BbhPenguinsInATable73CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT202 |
| ✓ | BbhPenguinsInATable74CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT206 |
Reasoning - BBH penguins in a table (set 3)
36 passed |
100.0% |
IN41 075 OUT9 971 | |
| ✓ | BbhPenguins100CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT533 |
| ✓ | BbhPenguins101CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT182 |
| ✓ | BbhPenguins102CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT269 |
| ✓ | BbhPenguins103CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT226 |
| ✓ | BbhPenguins104CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT48 |
| ✓ | BbhPenguins105CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT298 |
| ✓ | BbhPenguins106CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT218 |
| ✓ | BbhPenguins107CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT428 |
| ✓ | BbhPenguins108CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT437 |
| ✓ | BbhPenguins109CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT148 |
| ✓ | BbhPenguins110CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT232 |
| ✓ | BbhPenguins75CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT245 |
| ✓ | BbhPenguins76CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT422 |
| ✓ | BbhPenguins77CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT225 |
| ✓ | BbhPenguins78CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT241 |
| ✓ | BbhPenguins79CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT366 |
| ✓ | BbhPenguins80CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 195 OUT207 |
| ✓ | BbhPenguins81CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT218 |
| ✓ | BbhPenguins82CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT348 |
| ✓ | BbhPenguins83CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT507 |
| ✓ | BbhPenguins84CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 210 OUT353 |
| ✓ | BbhPenguins85CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT291 |
| ✓ | BbhPenguins86CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT163 |
| ✓ | BbhPenguins87CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT213 |
| ✓ | BbhPenguins88CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT403 |
| ✓ | BbhPenguins89CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT316 |
| ✓ | BbhPenguins90CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT201 |
| ✓ | BbhPenguins91CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT249 |
| ✓ | BbhPenguins92CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT443 |
| ✓ | BbhPenguins93CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT148 |
| ✓ | BbhPenguins94CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT176 |
| ✓ | BbhPenguins95CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT255 |
| ✓ | BbhPenguins96CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 231 OUT310 |
| ✓ | BbhPenguins97CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT207 |
| ✓ | BbhPenguins98CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 192 OUT226 |
| ✓ | BbhPenguins99CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT219 |
Reasoning - BBH penguins in a table (set 4)
35 passed |
100.0% |
IN39 978 OUT9 084 | |
| ✓ | BbhPenguins111CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT202 |
| ✓ | BbhPenguins112CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT165 |
| ✓ | BbhPenguins113CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT301 |
| ✓ | BbhPenguins114CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 202 OUT218 |
| ✓ | BbhPenguins115CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT143 |
| ✓ | BbhPenguins116CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT232 |
| ✓ | BbhPenguins117CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 227 OUT380 |
| ✓ | BbhPenguins118CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT187 |
| ✓ | BbhPenguins119CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT176 |
| ✓ | BbhPenguins120CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT431 |
| ✓ | BbhPenguins121CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT351 |
| ✓ | BbhPenguins122CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 206 OUT210 |
| ✓ | BbhPenguins123CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT228 |
| ✓ | BbhPenguins124CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT61 |
| ✓ | BbhPenguins125CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 194 OUT159 |
| ✓ | BbhPenguins126CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT559 |
| ✓ | BbhPenguins127CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT608 |
| ✓ | BbhPenguins128CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT203 |
| ✓ | BbhPenguins129CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT203 |
| ✓ | BbhPenguins130CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT369 |
| ✓ | BbhPenguins131CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT279 |
| ✓ | BbhPenguins132CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 232 OUT164 |
| ✓ | BbhPenguins133CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 205 OUT391 |
| ✓ | BbhPenguins134CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT224 |
| ✓ | BbhPenguins135CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT189 |
| ✓ | BbhPenguins136CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT216 |
| ✓ | BbhPenguins137CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT201 |
| ✓ | BbhPenguins138CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT206 |
| ✓ | BbhPenguins139CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT319 |
| ✓ | BbhPenguins140CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT449 |
| ✓ | BbhPenguins141CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT330 |
| ✓ | BbhPenguins142CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT132 |
| ✓ | BbhPenguins143CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT193 |
| ✓ | BbhPenguins144CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT189 |
| ✓ | BbhPenguins145CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT216 |
Reasoning - BBH reasoning about colored objects (set 1)
33 passed |
100.0% |
IN35 257 OUT6 176 | |
| ✓ | BbhColoredObjects10CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT205 |
| ✓ | BbhColoredObjects11CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT198 |
| ✓ | BbhColoredObjects12CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT267 |
| ✓ | BbhColoredObjects13CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT230 |
| ✓ | BbhColoredObjects14CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT211 |
| ✓ | BbhColoredObjects15CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT98 |
| ✓ | BbhColoredObjects16CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT348 |
| ✓ | BbhColoredObjects17CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT74 |
| ✓ | BbhColoredObjects18CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT290 |
| ✓ | BbhColoredObjects19CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT271 |
| ✓ | BbhColoredObjects20CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT223 |
| ✓ | BbhColoredObjects21CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT223 |
| ✓ | BbhColoredObjects22CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT155 |
| ✓ | BbhColoredObjects23CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT63 |
| ✓ | BbhColoredObjects24CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT307 |
| ✓ | BbhColoredObjects25CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT279 |
| ✓ | BbhColoredObjects26CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT186 |
| ✓ | BbhColoredObjects27CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT240 |
| ✓ | BbhColoredObjects28CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT226 |
| ✓ | BbhColoredObjects29CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT162 |
| ✓ | BbhColoredObjects30CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT121 |
| ✓ | BbhColoredObjects31CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT101 |
| ✓ | BbhColoredObjects32CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT195 |
| ✓ | BbhColoredObjects33CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT167 |
| ✓ | BbhColoredObjects34CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT59 |
| ✓ | BbhColoredObjects35CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT202 |
| ✓ | BbhColoredObjects36CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT179 |
| ✓ | BbhColoredObjects37CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT100 |
| ✓ | BbhColoredObjects38CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT91 |
| ✓ | BbhColoredObjects39CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT188 |
| ✓ | BbhColoredObjects40CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT228 |
| ✓ | BbhColoredObjects8CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT43 |
| ✓ | BbhColoredObjects9CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT246 |
Reasoning - BBH reasoning about colored objects (set 2)
29 passed1 failed |
96.7% |
IN31 842 OUT5 862 | |
| ✓ | BbhColoredObjects41CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT179 |
| ✓ | BbhColoredObjects42CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT276 |
| ✓ | BbhColoredObjects43CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT47 |
| ✓ | BbhColoredObjects44CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT201 |
| ✓ | BbhColoredObjects45CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT32 |
| ✓ | BbhColoredObjects46CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT33 |
| ✓ | BbhColoredObjects47CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT102 |
| ✓ | BbhColoredObjects48CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT205 |
| ✓ | BbhColoredObjects49CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT286 |
| ✓ | BbhColoredObjects50CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT294 |
| ✓ | BbhColoredObjects51CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT141 |
| ✓ | BbhColoredObjects52CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT156 |
| ✓ | BbhColoredObjects53CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT227 |
| ✓ | BbhColoredObjects54CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT88 |
| ✓ | BbhColoredObjects55CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT257 |
| ✓ | BbhColoredObjects56CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT321 |
| ✓ | BbhColoredObjects57CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT329 |
| × | BbhColoredObjects58CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(H)")
Context: BbhColoredObjects58CD: expected (H), got [g]
Values:
got = "g" |
Failed | IN1 099 OUT284 |
| ✓ | BbhColoredObjects59CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT146 |
| ✓ | BbhColoredObjects60CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT234 |
| ✓ | BbhColoredObjects61CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT178 |
| ✓ | BbhColoredObjects62CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT53 |
| ✓ | BbhColoredObjects63CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT295 |
| ✓ | BbhColoredObjects64CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT225 |
| ✓ | BbhColoredObjects65CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT242 |
| ✓ | BbhColoredObjects66CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT232 |
| ✓ | BbhColoredObjects67CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT224 |
| ✓ | BbhColoredObjects68CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT393 |
| ✓ | BbhColoredObjects69CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT142 |
| ✓ | BbhColoredObjects70CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT40 |
Reasoning - BBH reasoning about colored objects (set 3)
24 passed |
100.0% |
IN25 658 OUT5 200 | |
| ✓ | BbhColoredObjects71CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT183 |
| ✓ | BbhColoredObjects72CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT192 |
| ✓ | BbhColoredObjects73CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT255 |
| ✓ | BbhColoredObjects74CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT185 |
| ✓ | BbhColoredObjects75CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT971 |
| ✓ | BbhColoredObjects76CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT242 |
| ✓ | BbhColoredObjects77CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT63 |
| ✓ | BbhColoredObjects78CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT281 |
| ✓ | BbhColoredObjects79CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT152 |
| ✓ | BbhColoredObjects80CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT166 |
| ✓ | BbhColoredObjects81CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT249 |
| ✓ | BbhColoredObjects82CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT87 |
| ✓ | BbhColoredObjects83CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT48 |
| ✓ | BbhColoredObjects84CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT110 |
| ✓ | BbhColoredObjects85CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT289 |
| ✓ | BbhColoredObjects86CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT320 |
| ✓ | BbhColoredObjects87CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT305 |
| ✓ | BbhColoredObjects88CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT225 |
| ✓ | BbhColoredObjects89CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT122 |
| ✓ | BbhColoredObjects90CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT48 |
| ✓ | BbhColoredObjects91CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT141 |
| ✓ | BbhColoredObjects92CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT155 |
| ✓ | BbhColoredObjects93CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT175 |
| ✓ | BbhColoredObjects94CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT236 |
Reasoning - BBH reasoning about colored objects (set 4)
31 passed1 failed |
96.9% |
IN34 112 OUT7 066 | |
| ✓ | BbhReasoningAboutColoredObjects100CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT155 |
| ✓ | BbhReasoningAboutColoredObjects101CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT117 |
| ✓ | BbhReasoningAboutColoredObjects102CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT237 |
| × | BbhReasoningAboutColoredObjects103CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhReasoningAboutColoredObjects103CE: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 042 OUT261 |
| ✓ | BbhReasoningAboutColoredObjects104CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT197 |
| ✓ | BbhReasoningAboutColoredObjects105CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT319 |
| ✓ | BbhReasoningAboutColoredObjects106CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT286 |
| ✓ | BbhReasoningAboutColoredObjects107CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT187 |
| ✓ | BbhReasoningAboutColoredObjects108CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT172 |
| ✓ | BbhReasoningAboutColoredObjects109CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT364 |
| ✓ | BbhReasoningAboutColoredObjects110CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT130 |
| ✓ | BbhReasoningAboutColoredObjects111CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT114 |
| ✓ | BbhReasoningAboutColoredObjects112CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT53 |
| ✓ | BbhReasoningAboutColoredObjects113CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT52 |
| ✓ | BbhReasoningAboutColoredObjects114CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT273 |
| ✓ | BbhReasoningAboutColoredObjects115CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT72 |
| ✓ | BbhReasoningAboutColoredObjects116CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT36 |
| ✓ | BbhReasoningAboutColoredObjects117CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT115 |
| ✓ | BbhReasoningAboutColoredObjects118CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT210 |
| ✓ | BbhReasoningAboutColoredObjects119CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT375 |
| ✓ | BbhReasoningAboutColoredObjects120CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT229 |
| ✓ | BbhReasoningAboutColoredObjects121CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT53 |
| ✓ | BbhReasoningAboutColoredObjects122CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT199 |
| ✓ | BbhReasoningAboutColoredObjects123CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 368 |
| ✓ | BbhReasoningAboutColoredObjects124CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT99 |
| ✓ | BbhReasoningAboutColoredObjects125CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT270 |
| ✓ | BbhReasoningAboutColoredObjects126CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT179 |
| ✓ | BbhReasoningAboutColoredObjects95CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT186 |
| ✓ | BbhReasoningAboutColoredObjects96CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT295 |
| ✓ | BbhReasoningAboutColoredObjects97CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT295 |
| ✓ | BbhReasoningAboutColoredObjects98CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT133 |
| ✓ | BbhReasoningAboutColoredObjects99CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT35 |
Reasoning - BBH reasoning about colored objects (set 5)
32 passed |
100.0% |
IN33 832 OUT6 832 | |
| ✓ | BbhReasoningAboutColoredObjects127CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT277 |
| ✓ | BbhReasoningAboutColoredObjects128CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT122 |
| ✓ | BbhReasoningAboutColoredObjects129CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT167 |
| ✓ | BbhReasoningAboutColoredObjects130CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT57 |
| ✓ | BbhReasoningAboutColoredObjects131CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT321 |
| ✓ | BbhReasoningAboutColoredObjects132CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT426 |
| ✓ | BbhReasoningAboutColoredObjects133CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT228 |
| ✓ | BbhReasoningAboutColoredObjects134CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT196 |
| ✓ | BbhReasoningAboutColoredObjects135CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT88 |
| ✓ | BbhReasoningAboutColoredObjects136CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT289 |
| ✓ | BbhReasoningAboutColoredObjects137CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT48 |
| ✓ | BbhReasoningAboutColoredObjects138CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT231 |
| ✓ | BbhReasoningAboutColoredObjects139CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT199 |
| ✓ | BbhReasoningAboutColoredObjects140CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT321 |
| ✓ | BbhReasoningAboutColoredObjects141CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT38 |
| ✓ | BbhReasoningAboutColoredObjects142CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT61 |
| ✓ | BbhReasoningAboutColoredObjects143CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT206 |
| ✓ | BbhReasoningAboutColoredObjects144CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT354 |
| ✓ | BbhReasoningAboutColoredObjects145CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT345 |
| ✓ | BbhReasoningAboutColoredObjects146CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT145 |
| ✓ | BbhReasoningAboutColoredObjects147CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT274 |
| ✓ | BbhReasoningAboutColoredObjects148CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT281 |
| ✓ | BbhReasoningAboutColoredObjects149CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT230 |
| ✓ | BbhReasoningAboutColoredObjects150CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT267 |
| ✓ | BbhReasoningAboutColoredObjects151CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT418 |
| ✓ | BbhReasoningAboutColoredObjects152CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT57 |
| ✓ | BbhReasoningAboutColoredObjects153CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT129 |
| ✓ | BbhReasoningAboutColoredObjects154CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT192 |
| ✓ | BbhReasoningAboutColoredObjects155CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT419 |
| ✓ | BbhReasoningAboutColoredObjects156CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT245 |
| ✓ | BbhReasoningAboutColoredObjects157CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT145 |
| ✓ | BbhReasoningAboutColoredObjects158CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT56 |
Reasoning - BBH reasoning about colored objects (set 6)
32 passed |
100.0% |
IN34 075 OUT5 339 | |
| ✓ | BbhReasoningAboutColoredObjects159CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT117 |
| ✓ | BbhReasoningAboutColoredObjects160CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT281 |
| ✓ | BbhReasoningAboutColoredObjects161CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT187 |
| ✓ | BbhReasoningAboutColoredObjects162CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT178 |
| ✓ | BbhReasoningAboutColoredObjects163CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT47 |
| ✓ | BbhReasoningAboutColoredObjects164CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT148 |
| ✓ | BbhReasoningAboutColoredObjects165CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT190 |
| ✓ | BbhReasoningAboutColoredObjects166CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT113 |
| ✓ | BbhReasoningAboutColoredObjects167CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT292 |
| ✓ | BbhReasoningAboutColoredObjects168CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT260 |
| ✓ | BbhReasoningAboutColoredObjects169CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT115 |
| ✓ | BbhReasoningAboutColoredObjects170CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT104 |
| ✓ | BbhReasoningAboutColoredObjects171CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT317 |
| ✓ | BbhReasoningAboutColoredObjects172CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT78 |
| ✓ | BbhReasoningAboutColoredObjects173CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT49 |
| ✓ | BbhReasoningAboutColoredObjects174CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 036 OUT306 |
| ✓ | BbhReasoningAboutColoredObjects175CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT404 |
| ✓ | BbhReasoningAboutColoredObjects176CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT193 |
| ✓ | BbhReasoningAboutColoredObjects177CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT123 |
| ✓ | BbhReasoningAboutColoredObjects178CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT92 |
| ✓ | BbhReasoningAboutColoredObjects179CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT139 |
| ✓ | BbhReasoningAboutColoredObjects180CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT103 |
| ✓ | BbhReasoningAboutColoredObjects181CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT170 |
| ✓ | BbhReasoningAboutColoredObjects182CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT304 |
| ✓ | BbhReasoningAboutColoredObjects183CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT97 |
| ✓ | BbhReasoningAboutColoredObjects184CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT190 |
| ✓ | BbhReasoningAboutColoredObjects185CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT110 |
| ✓ | BbhReasoningAboutColoredObjects186CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT48 |
| ✓ | BbhReasoningAboutColoredObjects187CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT122 |
| ✓ | BbhReasoningAboutColoredObjects188CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT184 |
| ✓ | BbhReasoningAboutColoredObjects189CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT40 |
| ✓ | BbhReasoningAboutColoredObjects190CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT238 |
Reasoning - BBH reasoning about colored objects (set 7)
32 passed |
100.0% |
IN34 010 OUT6 025 | |
| ✓ | BbhReasoningAboutColoredObjects191CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT113 |
| ✓ | BbhReasoningAboutColoredObjects192CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT268 |
| ✓ | BbhReasoningAboutColoredObjects193CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT60 |
| ✓ | BbhReasoningAboutColoredObjects194CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT48 |
| ✓ | BbhReasoningAboutColoredObjects195CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT92 |
| ✓ | BbhReasoningAboutColoredObjects196CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT364 |
| ✓ | BbhReasoningAboutColoredObjects197CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT277 |
| ✓ | BbhReasoningAboutColoredObjects198CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT170 |
| ✓ | BbhReasoningAboutColoredObjects199CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT448 |
| ✓ | BbhReasoningAboutColoredObjects200CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT297 |
| ✓ | BbhReasoningAboutColoredObjects201CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT184 |
| ✓ | BbhReasoningAboutColoredObjects202CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT109 |
| ✓ | BbhReasoningAboutColoredObjects203CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT52 |
| ✓ | BbhReasoningAboutColoredObjects204CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT184 |
| ✓ | BbhReasoningAboutColoredObjects205CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT45 |
| ✓ | BbhReasoningAboutColoredObjects206CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT53 |
| ✓ | BbhReasoningAboutColoredObjects207CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT29 |
| ✓ | BbhReasoningAboutColoredObjects208CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT122 |
| ✓ | BbhReasoningAboutColoredObjects209CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT333 |
| ✓ | BbhReasoningAboutColoredObjects210CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT177 |
| ✓ | BbhReasoningAboutColoredObjects211CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT34 |
| ✓ | BbhReasoningAboutColoredObjects212CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT199 |
| ✓ | BbhReasoningAboutColoredObjects213CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT362 |
| ✓ | BbhReasoningAboutColoredObjects214CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT237 |
| ✓ | BbhReasoningAboutColoredObjects215CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT253 |
| ✓ | BbhReasoningAboutColoredObjects216CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT116 |
| ✓ | BbhReasoningAboutColoredObjects217CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT166 |
| ✓ | BbhReasoningAboutColoredObjects218CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT266 |
| ✓ | BbhReasoningAboutColoredObjects219CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT60 |
| ✓ | BbhReasoningAboutColoredObjects220CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT490 |
| ✓ | BbhReasoningAboutColoredObjects221CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT183 |
| ✓ | BbhReasoningAboutColoredObjects222CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT234 |
Reasoning - BBH reasoning about colored objects (set 8)
26 passed4 failed |
86.7% |
IN31 734 OUT7 412 | |
| ✓ | BbhReasoningAboutColoredObjects223CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT271 |
| ✓ | BbhReasoningAboutColoredObjects224CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT149 |
| ✓ | BbhReasoningAboutColoredObjects225CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT64 |
| ✓ | BbhReasoningAboutColoredObjects226CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT203 |
| ✓ | BbhReasoningAboutColoredObjects227CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT276 |
| ✓ | BbhReasoningAboutColoredObjects228CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT232 |
| ✓ | BbhReasoningAboutColoredObjects229CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT101 |
| ✓ | BbhReasoningAboutColoredObjects230CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT33 |
| ✓ | BbhReasoningAboutColoredObjects231CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT32 |
| × | BbhReasoningAboutColoredObjects232CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(F)")
Context: BbhReasoningAboutColoredObjects232CE: expected (F), got [e]
Values:
got = "e" |
Failed | IN1 052 OUT1 410 |
| ✓ | BbhReasoningAboutColoredObjects233CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT180 |
| ✓ | BbhReasoningAboutColoredObjects234CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT254 |
| ✓ | BbhReasoningAboutColoredObjects235CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT316 |
| ✓ | BbhReasoningAboutColoredObjects236CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT293 |
| ✓ | BbhReasoningAboutColoredObjects237CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT126 |
| × | BbhReasoningAboutColoredObjects238CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhReasoningAboutColoredObjects238CE: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 084 OUT293 |
| ✓ | BbhReasoningAboutColoredObjects239CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT108 |
| ✓ | BbhReasoningAboutColoredObjects240CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT129 |
| ✓ | BbhReasoningAboutColoredObjects241CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT54 |
| ✓ | BbhReasoningAboutColoredObjects242CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT154 |
| ✓ | BbhReasoningAboutColoredObjects243CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT278 |
| ✓ | BbhReasoningAboutColoredObjects244CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT250 |
| ✓ | BbhReasoningAboutColoredObjects245CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT247 |
| ✓ | BbhReasoningAboutColoredObjects246CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT288 |
| ✓ | BbhReasoningAboutColoredObjects247CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT65 |
| ✓ | BbhReasoningAboutColoredObjects248CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT130 |
| ✓ | BbhReasoningAboutColoredObjects249CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT161 |
| × | BbhRuinNames10CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames10CE: expected (A), got [d]
Values:
got = "d" |
Failed | IN998 OUT226 |
| ✓ | BbhRuinNames8CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT214 |
| × | BbhRuinNames9CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames9CE: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 000 OUT875 |
Reasoning - BBH ruin names (humorous edits) (set 1)
24 passed8 failed |
75.0% |
IN32 164 OUT20 728 | |
| ✓ | BbhRuinNames11CFWhich of the following is a humorous edit of this artist or movie name: 'the who!'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 137 |
| ✓ | BbhRuinNames12CFWhich of the following is a humorous edit of this artist or movie name: 'braveheart'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT324 |
| × | BbhRuinNames13CFWhich of the following is a humorous edit of this artist or movie name: 'the three stooges'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames13CF: expected (D), got [b]
Values:
got = "b" |
Failed | IN1 008 OUT1 169 |
| ✓ | BbhRuinNames14CFWhich of the following is a humorous edit of this artist or movie name: 'the deer hunter'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT307 |
| × | BbhRuinNames15CFWhich of the following is a humorous edit of this artist or movie name: 'boyz 2 men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames15CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 013 OUT1 587 |
| ✓ | BbhRuinNames16CFWhich of the following is a humorous edit of this artist or movie name: 'the beastie boys'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT396 |
| ✓ | BbhRuinNames17CFWhich of the following is a humorous edit of this artist or movie name: 'joy division'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 170 |
| ✓ | BbhRuinNames18CFWhich of the following is a humorous edit of this artist or movie name: 'herbie hancock'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT787 |
| ✓ | BbhRuinNames19CFWhich of the following is a humorous edit of this artist or movie name: 'bon iver'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT884 |
| × | BbhRuinNames20CFWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames20CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT314 |
| ✓ | BbhRuinNames21CFWhich of the following is a humorous edit of this artist or movie name: 'cinderella'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT253 |
| ✓ | BbhRuinNames22CFWhich of the following is a humorous edit of this artist or movie name: 'the italian job'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 241 |
| ✓ | BbhRuinNames23CFWhich of the following is a humorous edit of this artist or movie name: 'a few good men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT313 |
| ✓ | BbhRuinNames24CFWhich of the following is a humorous edit of this artist or movie name: 'raiders of the lost ark'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT332 |
| × | BbhRuinNames25CFWhich of the following is a humorous edit of this artist or movie name: 'little shop of horrors'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames25CF: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 010 OUT1 846 |
| ✓ | BbhRuinNames26CFWhich of the following is a humorous edit of this artist or movie name: 'free willy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT819 |
| ✓ | BbhRuinNames27CFWhich of the following is a humorous edit of this artist or movie name: 'pixies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT640 |
| ✓ | BbhRuinNames28CFWhich of the following is a humorous edit of this artist or movie name: 'the princess and the frog'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT409 |
| ✓ | BbhRuinNames29CFWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT235 |
| ✓ | BbhRuinNames30CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT660 |
| ✓ | BbhRuinNames31CFWhich of the following is a humorous edit of this artist or movie name: 'pantera'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT298 |
| × | BbhRuinNames32CFWhich of the following is a humorous edit of this artist or movie name: 'top gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames32CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN997 OUT298 |
| ✓ | BbhRuinNames33CFWhich of the following is a humorous edit of this artist or movie name: 'die another day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT274 |
| ✓ | BbhRuinNames34CFWhich of the following is a humorous edit of this artist or movie name: 'the moody blues'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT254 |
| ✓ | BbhRuinNames35CFWhich of the following is a humorous edit of this artist or movie name: 'fall out boy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT253 |
| × | BbhRuinNames36CFWhich of the following is a humorous edit of this artist or movie name: 'love story'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames36CF: expected (A), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT745 |
| ✓ | BbhRuinNames37CFWhich of the following is a humorous edit of this artist or movie name: 'whitney houston'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT736 |
| ✓ | BbhRuinNames38CFWhich of the following is a humorous edit of this artist or movie name: 'bridge of spies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT348 |
| × | BbhRuinNames39CFWhich of the following is a humorous edit of this artist or movie name: 'breaking benjamin'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames39CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 007 OUT299 |
| × | BbhRuinNames40CFWhich of the following is a humorous edit of this artist or movie name: 'frank kappa'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames40CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN1 001 OUT1 859 |
| ✓ | BbhRuinNames41CFWhich of the following is a humorous edit of this artist or movie name: 'jurassic park'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT245 |
| ✓ | BbhRuinNames42CFWhich of the following is a humorous edit of this artist or movie name: 'the band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT296 |
Reasoning - BBH ruin names (humorous edits) (set 2)
27 passed5 failed |
84.4% |
IN32 200 OUT18 014 | |
| ✓ | BbhRuinNames43CFWhich of the following is a humorous edit of this artist or movie name: 'creedence clearwater revival'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT364 |
| × | BbhRuinNames44CFWhich of the following is a humorous edit of this artist or movie name: 'block 57'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames44CF: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 010 OUT2 005 |
| ✓ | BbhRuinNames45CFWhich of the following is a humorous edit of this artist or movie name: 'get shorty'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT387 |
| ✓ | BbhRuinNames46CFWhich of the following is a humorous edit of this artist or movie name: 'radiohead'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT255 |
| ✓ | BbhRuinNames47CFWhich of the following is a humorous edit of this artist or movie name: 'mean girls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT285 |
| ✓ | BbhRuinNames48CFWhich of the following is a humorous edit of this artist or movie name: 'seven brides for seven brothers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT329 |
| ✓ | BbhRuinNames49CFWhich of the following is a humorous edit of this artist or movie name: 'the green mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT276 |
| ✓ | BbhRuinNames50CFWhich of the following is a humorous edit of this artist or movie name: 'cat on a hot tin roof'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT373 |
| ✓ | BbhRuinNames51CFWhich of the following is a humorous edit of this artist or movie name: 'new kids on the block'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT399 |
| ✓ | BbhRuinNames52CFWhich of the following is a humorous edit of this artist or movie name: 'superclunk'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT458 |
| ✓ | BbhRuinNames53CFWhich of the following is a humorous edit of this artist or movie name: 'my big fat greek wedding'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT334 |
| ✓ | BbhRuinNames54CFWhich of the following is a humorous edit of this artist or movie name: 'snakes on a plane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT393 |
| × | BbhRuinNames55CFWhich of the following is a humorous edit of this artist or movie name: 'rocky'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames55CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN997 OUT680 |
| ✓ | BbhRuinNames56CFWhich of the following is a humorous edit of this artist or movie name: 'pink floyd'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT319 |
| ✓ | BbhRuinNames57CFWhich of the following is a humorous edit of this artist or movie name: 'bad religion'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT346 |
| × | BbhRuinNames58CFWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames58CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT274 |
| ✓ | BbhRuinNames59CFWhich of the following is a humorous edit of this artist or movie name: 'ice cube'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT590 |
| ✓ | BbhRuinNames60CFWhich of the following is a humorous edit of this artist or movie name: 'new order'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT846 |
| ✓ | BbhRuinNames61CFWhich of the following is a humorous edit of this artist or movie name: 'dolly dots'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 329 |
| ✓ | BbhRuinNames62CFWhich of the following is a humorous edit of this artist or movie name: 'die another day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT337 |
| ✓ | BbhRuinNames63CFWhich of the following is a humorous edit of this artist or movie name: 'mad max'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT402 |
| ✓ | BbhRuinNames64CFWhich of the following is a humorous edit of this artist or movie name: '8 mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT465 |
| ✓ | BbhRuinNames65CFWhich of the following is a humorous edit of this artist or movie name: 'i spit on your grave'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT298 |
| ✓ | BbhRuinNames66CFWhich of the following is a humorous edit of this artist or movie name: 'planet of the apes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT262 |
| ✓ | BbhRuinNames67CFWhich of the following is a humorous edit of this artist or movie name: 'the rolling stones'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT264 |
| ✓ | BbhRuinNames68CFWhich of the following is a humorous edit of this artist or movie name: 'reservoir dogs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT865 |
| × | BbhRuinNames69CFWhich of the following is a humorous edit of this artist or movie name: 'the men who stare at goats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames69CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 019 OUT1 091 |
| ✓ | BbhRuinNames70CFWhich of the following is a humorous edit of this artist or movie name: 'bridge of spies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT335 |
| ✓ | BbhRuinNames71CFWhich of the following is a humorous edit of this artist or movie name: 'last action hero'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT514 |
| ✓ | BbhRuinNames72CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT674 |
| ✓ | BbhRuinNames73CFWhich of the following is a humorous edit of this artist or movie name: 'the full monty'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT426 |
| × | BbhRuinNames74CFWhich of the following is a humorous edit of this artist or movie name: 'fast and furious'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames74CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 005 OUT1 839 |
Reasoning - BBH ruin names (humorous edits) (set 3)
28 passed4 failed |
87.5% |
IN32 223 OUT13 917 | |
| ✓ | BbhRuinNames100CFWhich of the following is a humorous edit of this artist or movie name: 'the battle of britain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT387 |
| ✓ | BbhRuinNames101CFWhich of the following is a humorous edit of this artist or movie name: 'dead man walking'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT915 |
| ✓ | BbhRuinNames102CFWhich of the following is a humorous edit of this artist or movie name: 'suicide squad'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT267 |
| ✓ | BbhRuinNames103CFWhich of the following is a humorous edit of this artist or movie name: 'the dark knight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT248 |
| ✓ | BbhRuinNames104CFWhich of the following is a humorous edit of this artist or movie name: 'sleepless in seattle'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT229 |
| ✓ | BbhRuinNames105CFWhich of the following is a humorous edit of this artist or movie name: 'pink floyd'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT401 |
| ✓ | BbhRuinNames106CFWhich of the following is a humorous edit of this artist or movie name: 'the man with the golden gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT354 |
| ✓ | BbhRuinNames75CFWhich of the following is a humorous edit of this artist or movie name: 'skinny puppy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT566 |
| ✓ | BbhRuinNames76CFWhich of the following is a humorous edit of this artist or movie name: 'depeche mode'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT729 |
| ✓ | BbhRuinNames77CFWhich of the following is a humorous edit of this artist or movie name: 'sly and the family stone'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT462 |
| ✓ | BbhRuinNames78CFWhich of the following is a humorous edit of this artist or movie name: 'the departed'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT462 |
| ✓ | BbhRuinNames79CFWhich of the following is a humorous edit of this artist or movie name: 'blade runner'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT270 |
| ✓ | BbhRuinNames80CFWhich of the following is a humorous edit of this artist or movie name: 'the avengers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT402 |
| ✓ | BbhRuinNames81CFWhich of the following is a humorous edit of this artist or movie name: 'die hard'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT892 |
| ✓ | BbhRuinNames82CFWhich of the following is a humorous edit of this artist or movie name: 'weezer'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT296 |
| ✓ | BbhRuinNames83CFWhich of the following is a humorous edit of this artist or movie name: 'the longest day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT282 |
| × | BbhRuinNames84CFWhich of the following is a humorous edit of this artist or movie name: 'babs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames84CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN996 OUT404 |
| ✓ | BbhRuinNames85CFWhich of the following is a humorous edit of this artist or movie name: 'moonlight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT246 |
| ✓ | BbhRuinNames86CFWhich of the following is a humorous edit of this artist or movie name: 'animal farm'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT272 |
| ✓ | BbhRuinNames87CFWhich of the following is a humorous edit of this artist or movie name: 'seven brides for seven brothers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT454 |
| ✓ | BbhRuinNames88CFWhich of the following is a humorous edit of this artist or movie name: 'kiss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT273 |
| ✓ | BbhRuinNames89CFWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT258 |
| ✓ | BbhRuinNames90CFWhich of the following is a humorous edit of this artist or movie name: 'en vogue'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT388 |
| ✓ | BbhRuinNames91CFWhich of the following is a humorous edit of this artist or movie name: 'top gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 135 |
| ✓ | BbhRuinNames92CFWhich of the following is a humorous edit of this artist or movie name: 'the girl with all the gifts'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT347 |
| ✓ | BbhRuinNames93CFWhich of the following is a humorous edit of this artist or movie name: 'the sisters of mercy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT344 |
| × | BbhRuinNames94CFWhich of the following is a humorous edit of this artist or movie name: 'the jam'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames94CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN998 OUT384 |
| × | BbhRuinNames95CFWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames95CF: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 011 OUT1 003 |
| ✓ | BbhRuinNames96CFWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT309 |
| ✓ | BbhRuinNames97CFWhich of the following is a humorous edit of this artist or movie name: 'dream theater'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT272 |
| ✓ | BbhRuinNames98CFWhich of the following is a humorous edit of this artist or movie name: 'belle and sebastian'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT367 |
| × | BbhRuinNames99CFWhich of the following is a humorous edit of this artist or movie name: 'earth, wind, & fire'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("dearth, wind, & fire")
Context: BbhRuinNames99CF: expected dearth, wind, & fire, got [i]
Values:
got = "i" |
Failed | IN1 033 OUT299 |
Reasoning - BBH ruin names (humorous edits) (set 4)
31 passed1 failed |
96.9% |
IN32 207 OUT19 421 | |
| ✓ | BbhRuinNames107CFWhich of the following is a humorous edit of this artist or movie name: 'the mash'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT907 |
| ✓ | BbhRuinNames108CFWhich of the following is a humorous edit of this artist or movie name: 'missy elliott'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT704 |
| ✓ | BbhRuinNames109CFWhich of the following is a humorous edit of this artist or movie name: 'sleater-kinney'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT487 |
| ✓ | BbhRuinNames110CFWhich of the following is a humorous edit of this artist or movie name: 'dirty dancing'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 684 |
| ✓ | BbhRuinNames111CFWhich of the following is a humorous edit of this artist or movie name: 'jay and silent bob strike back'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT2 088 |
| ✓ | BbhRuinNames112CFWhich of the following is a humorous edit of this artist or movie name: 'a bridge too far'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT285 |
| ✓ | BbhRuinNames113CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT244 |
| ✓ | BbhRuinNames114CFWhich of the following is a humorous edit of this artist or movie name: 'night club'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT325 |
| ✓ | BbhRuinNames115CFWhich of the following is a humorous edit of this artist or movie name: 'no time to die'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT343 |
| ✓ | BbhRuinNames116CFWhich of the following is a humorous edit of this artist or movie name: 'batman begins'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT687 |
| ✓ | BbhRuinNames117CFWhich of the following is a humorous edit of this artist or movie name: 'gold finger'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT405 |
| ✓ | BbhRuinNames118CFWhich of the following is a humorous edit of this artist or movie name: 'calamity jane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 022 |
| × | BbhRuinNames119CFWhich of the following is a humorous edit of this artist or movie name: 'saving private ryan'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames119CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 010 OUT313 |
| ✓ | BbhRuinNames120CFWhich of the following is a humorous edit of this artist or movie name: 'the cult'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 013 |
| ✓ | BbhRuinNames121CFWhich of the following is a humorous edit of this artist or movie name: 'johnny got his gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT837 |
| ✓ | BbhRuinNames122CFWhich of the following is a humorous edit of this artist or movie name: 'counting crows'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT274 |
| ✓ | BbhRuinNames123CFWhich of the following is a humorous edit of this artist or movie name: 'lawrence of arabia'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT389 |
| ✓ | BbhRuinNames124CFWhich of the following is a humorous edit of this artist or movie name: 'jungle book'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT350 |
| ✓ | BbhRuinNames125CFWhich of the following is a humorous edit of this artist or movie name: 'paint your wagon'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT436 |
| ✓ | BbhRuinNames126CFWhich of the following is a humorous edit of this artist or movie name: 'blade'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 631 |
| ✓ | BbhRuinNames127CFWhich of the following is a humorous edit of this artist or movie name: 'the sisterhood of the traveling pants'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT319 |
| ✓ | BbhRuinNames128CFWhich of the following is a humorous edit of this artist or movie name: 'green day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT251 |
| ✓ | BbhRuinNames129CFWhich of the following is a humorous edit of this artist or movie name: 'the sting'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT693 |
| ✓ | BbhRuinNames130CFWhich of the following is a humorous edit of this artist or movie name: 'the artist'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT456 |
| ✓ | BbhRuinNames131CFWhich of the following is a humorous edit of this artist or movie name: 'the hope'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT335 |
| ✓ | BbhRuinNames132CFWhich of the following is a humorous edit of this artist or movie name: 'blade runner'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 134 |
| ✓ | BbhRuinNames133CFWhich of the following is a humorous edit of this artist or movie name: 'a view to a kill'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT279 |
| ✓ | BbhRuinNames134CFWhich of the following is a humorous edit of this artist or movie name: 'hope floats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT336 |
| ✓ | BbhRuinNames135CFWhich of the following is a humorous edit of this artist or movie name: 'the apartment'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT391 |
| ✓ | BbhRuinNames136CFWhich of the following is a humorous edit of this artist or movie name: 'schindler's list'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT293 |
| ✓ | BbhRuinNames137CFWhich of the following is a humorous edit of this artist or movie name: 'beauty and the beast'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT246 |
| ✓ | BbhRuinNames138CFWhich of the following is a humorous edit of this artist or movie name: 'news of the world'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT264 |
Reasoning - BBH ruin names (humorous edits) (set 5)
24 passed6 failed |
80.0% |
IN30 199 OUT17 895 | |
| ✓ | BbhRuinNames139CFWhich of the following is a humorous edit of this artist or movie name: 'the guns of navarone'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT407 |
| ✓ | BbhRuinNames140CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT594 |
| ✓ | BbhRuinNames141CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT368 |
| ✓ | BbhRuinNames142CFWhich of the following is a humorous edit of this artist or movie name: 'hank williams'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT257 |
| ✓ | BbhRuinNames143CFWhich of the following is a humorous edit of this artist or movie name: 'black sabbath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT438 |
| × | BbhRuinNames144CFWhich of the following is a humorous edit of this artist or movie name: 'rita, sue and bob too'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("rita, sue and bob poo")
Context: BbhRuinNames144CF: expected rita, sue and bob poo, got [d]
Values:
got = "d" |
Failed | IN1 027 OUT1 144 |
| ✓ | BbhRuinNames145CFWhich of the following is a humorous edit of this artist or movie name: 'jethro tull'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT376 |
| ✓ | BbhRuinNames146CFWhich of the following is a humorous edit of this artist or movie name: 'the sound of music'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT272 |
| ✓ | BbhRuinNames147CFWhich of the following is a humorous edit of this artist or movie name: 'dream theater'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT312 |
| ✓ | BbhRuinNames148CFWhich of the following is a humorous edit of this artist or movie name: 'my fair lady'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT428 |
| ✓ | BbhRuinNames149CFWhich of the following is a humorous edit of this artist or movie name: 'offspring'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT714 |
| ✓ | BbhRuinNames150CFWhich of the following is a humorous edit of this artist or movie name: 'the longest day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 775 |
| ✓ | BbhRuinNames151CFWhich of the following is a humorous edit of this artist or movie name: 'apocalypse now'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT520 |
| × | BbhRuinNames152CFWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames152CF: expected (B), got [d]
Values:
got = "d" |
Failed | IN999 OUT294 |
| ✓ | BbhRuinNames153CFWhich of the following is a humorous edit of this artist or movie name: 'big country'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 133 |
| ✓ | BbhRuinNames154CFWhich of the following is a humorous edit of this artist or movie name: '12 angry men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT238 |
| ✓ | BbhRuinNames155CFWhich of the following is a humorous edit of this artist or movie name: 'in search of a midnight kiss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT314 |
| ✓ | BbhRuinNames156CFWhich of the following is a humorous edit of this artist or movie name: 'men in black'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT337 |
| ✓ | BbhRuinNames157CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT398 |
| ✓ | BbhRuinNames158CFWhich of the following is a humorous edit of this artist or movie name: 'raging bull'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT279 |
| × | BbhRuinNames159CFWhich of the following is a humorous edit of this artist or movie name: 'bad brains'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames159CF: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 001 OUT1 074 |
| ✓ | BbhRuinNames160CFWhich of the following is a humorous edit of this artist or movie name: 'thin lizzy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT653 |
| × | BbhRuinNames161CFWhich of the following is a humorous edit of this artist or movie name: 'ben folds five'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames161CF: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 005 OUT1 187 |
| ✓ | BbhRuinNames162CFWhich of the following is a humorous edit of this artist or movie name: 'citizen kane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT377 |
| ✓ | BbhRuinNames163CFWhich of the following is a humorous edit of this artist or movie name: 'dave matthews band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT747 |
| ✓ | BbhRuinNames164CFWhich of the following is a humorous edit of this artist or movie name: 'the sum of all fears'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 114 |
| ✓ | BbhRuinNames165CFWhich of the following is a humorous edit of this artist or movie name: 'city hall'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT287 |
| × | BbhRuinNames166CFWhich of the following is a humorous edit of this artist or movie name: 'the abyss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames166CF: expected (A), got [c]
Values:
got = "c" |
Failed | IN1 001 OUT564 |
| × | BbhRuinNames167CFWhich of the following is a humorous edit of this artist or movie name: 'mank'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames167CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN995 OUT428 |
| ✓ | BbhRuinNames168CFWhich of the following is a humorous edit of this artist or movie name: 'sex pistols'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT866 |
Reasoning - BBH ruin names (humorous edits) (set 6)
38 passed3 failed |
92.7% |
IN41 275 OUT21 531 | |
| ✓ | BbhRuinNames169CGWhich of the following is a humorous edit of this artist or movie name: 'life of brian'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT940 |
| ✓ | BbhRuinNames170CGWhich of the following is a humorous edit of this artist or movie name: 'black panther'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT849 |
| ✓ | BbhRuinNames171CGWhich of the following is a humorous edit of this artist or movie name: 'goldfinger'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT355 |
| ✓ | BbhRuinNames172CGWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT252 |
| ✓ | BbhRuinNames173CGWhich of the following is a humorous edit of this artist or movie name: 'air supply'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT252 |
| ✓ | BbhRuinNames174CGWhich of the following is a humorous edit of this artist or movie name: 'diana ross and the supremes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT415 |
| ✓ | BbhRuinNames175CGWhich of the following is a humorous edit of this artist or movie name: 'brokeback mountain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT499 |
| ✓ | BbhRuinNames176CGWhich of the following is a humorous edit of this artist or movie name: 'godzilla vs. kong'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT238 |
| ✓ | BbhRuinNames177CGWhich of the following is a humorous edit of this artist or movie name: 'the who!'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT255 |
| ✓ | BbhRuinNames178CGWhich of the following is a humorous edit of this artist or movie name: 'the green mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT644 |
| ✓ | BbhRuinNames179CGWhich of the following is a humorous edit of this artist or movie name: 'the dark knight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT242 |
| ✓ | BbhRuinNames180CGWhich of the following is a humorous edit of this artist or movie name: 'live and let die'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT420 |
| ✓ | BbhRuinNames181CGWhich of the following is a humorous edit of this artist or movie name: 'midnight cowboy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT847 |
| ✓ | BbhRuinNames182CGWhich of the following is a humorous edit of this artist or movie name: 'snow patrol'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT253 |
| ✓ | BbhRuinNames183CGWhich of the following is a humorous edit of this artist or movie name: 'the bangles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT354 |
| ✓ | BbhRuinNames184CGWhich of the following is a humorous edit of this artist or movie name: 'state champs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT250 |
| × | BbhRuinNames185CGWhich of the following is a humorous edit of this artist or movie name: 'the secret life of pets'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames185CG: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 013 OUT737 |
| ✓ | BbhRuinNames186CGWhich of the following is a humorous edit of this artist or movie name: 'full metal jacket'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 291 |
| × | BbhRuinNames187CGWhich of the following is a humorous edit of this artist or movie name: 'friday the 13th'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames187CG: expected (C), got [d]
Values:
got = "d" |
Failed | IN1 023 OUT502 |
| ✓ | BbhRuinNames188CGWhich of the following is a humorous edit of this artist or movie name: 'the bay city rollers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT275 |
| ✓ | BbhRuinNames189CGWhich of the following is a humorous edit of this artist or movie name: 'blue'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT315 |
| ✓ | BbhRuinNames190CGWhich of the following is a humorous edit of this artist or movie name: 'the gap band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT686 |
| ✓ | BbhRuinNames191CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT374 |
| ✓ | BbhRuinNames192CGWhich of the following is a humorous edit of this artist or movie name: 'the fall of the house of usher'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT425 |
| ✓ | BbhRuinNames193CGWhich of the following is a humorous edit of this artist or movie name: 'old joy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT389 |
| ✓ | BbhRuinNames194CGWhich of the following is a humorous edit of this artist or movie name: 'run dmc'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 150 |
| ✓ | BbhRuinNames195CGWhich of the following is a humorous edit of this artist or movie name: 'the grapes of wrath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT371 |
| ✓ | BbhRuinNames196CGWhich of the following is a humorous edit of this artist or movie name: 'drag me to hell'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT249 |
| ✓ | BbhRuinNames197CGWhich of the following is a humorous edit of this artist or movie name: 'talking heads'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT673 |
| ✓ | BbhRuinNames198CGWhich of the following is a humorous edit of this artist or movie name: 'paul oakenfold'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 155 |
| ✓ | BbhRuinNames199CGWhich of the following is a humorous edit of this artist or movie name: 'brief encounter'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT300 |
| ✓ | BbhRuinNames200CGWhich of the following is a humorous edit of this artist or movie name: 'a tribe called quest'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT233 |
| × | BbhRuinNames201CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames201CG: expected (C), got [a]
Values:
got = "a" |
Failed | IN998 OUT1 357 |
| ✓ | BbhRuinNames202CGWhich of the following is a humorous edit of this artist or movie name: 'the wizard of oz'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT258 |
| ✓ | BbhRuinNames203CGWhich of the following is a humorous edit of this artist or movie name: 'run dmc'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT462 |
| ✓ | BbhRuinNames204CGWhich of the following is a humorous edit of this artist or movie name: 'the third man'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT687 |
| ✓ | BbhRuinNames205CGWhich of the following is a humorous edit of this artist or movie name: 'cat on a hot tin roof'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT371 |
| ✓ | BbhRuinNames206CGWhich of the following is a humorous edit of this artist or movie name: 'notting hill'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT495 |
| ✓ | BbhRuinNames207CGWhich of the following is a humorous edit of this artist or movie name: 'good will hunting'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT800 |
| ✓ | BbhRuinNames208CGWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT253 |
| ✓ | BbhRuinNames209CGWhich of the following is a humorous edit of this artist or movie name: 'goo goo dolls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT658 |
Reasoning - BBH ruin names (humorous edits) (set 7)
36 passed4 failed |
90.0% |
IN40 287 OUT29 447 | |
| ✓ | BbhRuinNames210CGWhich of the following is a humorous edit of this artist or movie name: 'soundgarden'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT764 |
| ✓ | BbhRuinNames211CGWhich of the following is a humorous edit of this artist or movie name: 'peter and the test tube babies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT237 |
| ✓ | BbhRuinNames212CGWhich of the following is a humorous edit of this artist or movie name: 'blur'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT513 |
| ✓ | BbhRuinNames213CGWhich of the following is a humorous edit of this artist or movie name: 'black sabbath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT783 |
| ✓ | BbhRuinNames214CGWhich of the following is a humorous edit of this artist or movie name: 'trolls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 102 |
| × | BbhRuinNames215CGWhich of the following is a humorous edit of this artist or movie name: 'bad company'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames215CG: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT1 273 |
| ✓ | BbhRuinNames216CGWhich of the following is a humorous edit of this artist or movie name: '3:10 to yuma'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT1 135 |
| ✓ | BbhRuinNames217CGWhich of the following is a humorous edit of this artist or movie name: 'herman's hermits'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT468 |
| ✓ | BbhRuinNames218CGWhich of the following is a humorous edit of this artist or movie name: 'night of the demons'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT303 |
| ✓ | BbhRuinNames219CGWhich of the following is a humorous edit of this artist or movie name: 'the rolling stones'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT305 |
| ✓ | BbhRuinNames220CGWhich of the following is a humorous edit of this artist or movie name: 'blur'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 155 |
| ✓ | BbhRuinNames221CGWhich of the following is a humorous edit of this artist or movie name: 'a matter of life and death'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT352 |
| ✓ | BbhRuinNames222CGWhich of the following is a humorous edit of this artist or movie name: 'singin' in the rain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT708 |
| ✓ | BbhRuinNames223CGWhich of the following is a humorous edit of this artist or movie name: 'the man with two brains'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT338 |
| × | BbhRuinNames224CGWhich of the following is a humorous edit of this artist or movie name: 'born free'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames224CG: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 000 OUT1 516 |
| × | BbhRuinNames225CGWhich of the following is a humorous edit of this artist or movie name: 'the moody blues'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames225CG: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 010 OUT1 211 |
| ✓ | BbhRuinNames226CGWhich of the following is a humorous edit of this artist or movie name: 'jojo rabbit'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 142 |
| ✓ | BbhRuinNames227CGWhich of the following is a humorous edit of this artist or movie name: 'reservoir dogs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT709 |
| ✓ | BbhRuinNames228CGWhich of the following is a humorous edit of this artist or movie name: 'thunder road'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT336 |
| ✓ | BbhRuinNames229CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT275 |
| ✓ | BbhRuinNames230CGWhich of the following is a humorous edit of this artist or movie name: 'deadpool'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT330 |
| ✓ | BbhRuinNames231CGWhich of the following is a humorous edit of this artist or movie name: 'the lady vanishes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT351 |
| ✓ | BbhRuinNames232CGWhich of the following is a humorous edit of this artist or movie name: 'steely dan'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT775 |
| ✓ | BbhRuinNames233CGWhich of the following is a humorous edit of this artist or movie name: 'the men who stare at goats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT314 |
| ✓ | BbhRuinNames234CGWhich of the following is a humorous edit of this artist or movie name: 'the stone roses'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT302 |
| ✓ | BbhRuinNames235CGWhich of the following is a humorous edit of this artist or movie name: 'nine inch nails'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 254 |
| ✓ | BbhRuinNames236CGWhich of the following is a humorous edit of this artist or movie name: 'billy idol'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT354 |
| ✓ | BbhRuinNames237CGWhich of the following is a humorous edit of this artist or movie name: 'schindler's list'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT785 |
| ✓ | BbhRuinNames238CGWhich of the following is a humorous edit of this artist or movie name: 'sex pistols'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT776 |
| ✓ | BbhRuinNames239CGWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT389 |
| ✓ | BbhRuinNames240CGWhich of the following is a humorous edit of this artist or movie name: 'it's a wonderful life'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 223 |
| ✓ | BbhRuinNames241CGWhich of the following is a humorous edit of this artist or movie name: 'guns n' roses'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT961 |
| ✓ | BbhRuinNames242CGWhich of the following is a humorous edit of this artist or movie name: 'witness'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT298 |
| × | BbhRuinNames243CGWhich of the following is a humorous edit of this artist or movie name: 'killing joke'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames243CG: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 000 OUT1 814 |
| ✓ | BbhRuinNames244CGWhich of the following is a humorous edit of this artist or movie name: 'panic! at the disco'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT381 |
| ✓ | BbhRuinNames245CGWhich of the following is a humorous edit of this artist or movie name: 'spice girls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT367 |
| ✓ | BbhRuinNames246CGWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT251 |
| ✓ | BbhRuinNames247CGWhich of the following is a humorous edit of this artist or movie name: 'spandau ballet'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT493 |
| ✓ | BbhRuinNames248CGWhich of the following is a humorous edit of this artist or movie name: 'toto'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT2 799 |
| ✓ | BbhRuinNames249CGWhich of the following is a humorous edit of this artist or movie name: 'system of a down'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT605 |
Reasoning - BBH snarks - detect sarcasm (set 1)
26 passed6 failed |
81.3% |
IN33 458 OUT17 121 | |
| × | BbhSalientTranslation243CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSalientTranslation243CI: expected (A), got [e]
Values:
got = "e" |
Failed | IN1 204 OUT1 536 |
| ✓ | BbhSalientTranslation244CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 180 OUT885 |
| × | BbhSalientTranslation245CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSalientTranslation245CI: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 194 OUT436 |
| ✓ | BbhSalientTranslation246CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 237 OUT707 |
| ✓ | BbhSalientTranslation247CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 192 OUT366 |
| × | BbhSalientTranslation248CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(E)")
Context: BbhSalientTranslation248CI: expected (E), got [a]
Values:
got = "a" |
Failed | IN1 187 OUT736 |
| × | BbhSalientTranslation249CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhSalientTranslation249CI: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 234 OUT753 |
| ✓ | BbhSnarks10CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT313 |
| ✓ | BbhSnarks11CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT294 |
| ✓ | BbhSnarks12CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT299 |
| ✓ | BbhSnarks13CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT349 |
| ✓ | BbhSnarks14CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT342 |
| ✓ | BbhSnarks15CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT246 |
| ✓ | BbhSnarks16CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT254 |
| ✓ | BbhSnarks17CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT292 |
| ✓ | BbhSnarks18CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT238 |
| ✓ | BbhSnarks19CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT195 |
| ✓ | BbhSnarks20CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT1 125 |
| ✓ | BbhSnarks21CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT531 |
| × | BbhSnarks22CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks22CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN996 OUT1 746 |
| ✓ | BbhSnarks23CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT802 |
| ✓ | BbhSnarks24CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT276 |
| ✓ | BbhSnarks25CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT334 |
| ✓ | BbhSnarks26CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT346 |
| ✓ | BbhSnarks27CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT232 |
| ✓ | BbhSnarks28CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT291 |
| ✓ | BbhSnarks29CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT255 |
| ✓ | BbhSnarks30CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT333 |
| ✓ | BbhSnarks31CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT287 |
| × | BbhSnarks32CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks32CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN985 OUT1 767 |
| ✓ | BbhSnarks8CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT328 |
| ✓ | BbhSnarks9CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT227 |
Reasoning - BBH snarks - detect sarcasm (set 2)
28 passed4 failed |
87.5% |
IN32 212 OUT15 563 | |
| ✓ | BbhSnarks33CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT219 |
| ✓ | BbhSnarks34CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT275 |
| ✓ | BbhSnarks35CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT303 |
| ✓ | BbhSnarks36CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT203 |
| ✓ | BbhSnarks37CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT347 |
| ✓ | BbhSnarks38CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT296 |
| ✓ | BbhSnarks39CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT193 |
| × | BbhSnarks40CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks40CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN994 OUT1 081 |
| ✓ | BbhSnarks41CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT362 |
| ✓ | BbhSnarks42CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT290 |
| ✓ | BbhSnarks43CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT516 |
| ✓ | BbhSnarks44CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT303 |
| ✓ | BbhSnarks45CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 366 |
| ✓ | BbhSnarks46CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT173 |
| ✓ | BbhSnarks47CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT770 |
| ✓ | BbhSnarks48CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT321 |
| × | BbhSnarks49CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks49CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN990 OUT1 376 |
| ✓ | BbhSnarks50CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT394 |
| × | BbhSnarks51CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks51CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 008 OUT2 890 |
| ✓ | BbhSnarks52CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT268 |
| ✓ | BbhSnarks53CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT365 |
| ✓ | BbhSnarks54CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT192 |
| ✓ | BbhSnarks55CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT239 |
| ✓ | BbhSnarks56CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT299 |
| ✓ | BbhSnarks57CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT269 |
| × | BbhSnarks58CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks58CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN998 OUT238 |
| ✓ | BbhSnarks59CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT355 |
| ✓ | BbhSnarks60CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT445 |
| ✓ | BbhSnarks61CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT278 |
| ✓ | BbhSnarks62CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT275 |
| ✓ | BbhSnarks63CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT367 |
| ✓ | BbhSnarks64CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT295 |
Reasoning - BBH snarks - detect sarcasm (set 3)
28 passed2 failed2 skipped |
93.3% |
IN30 151 OUT16 427 | |
| ✓ | BbhSnarks65CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT542 |
| × | BbhSnarks66CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks66CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 008 OUT971 |
| ✓ | BbhSnarks67CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT170 |
| ✓ | BbhSnarks68CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT278 |
| ✓ | BbhSnarks69CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT343 |
| ✓ | BbhSnarks70CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT266 |
| ✓ | BbhSnarks71CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT282 |
| ✓ | BbhSnarks72CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT628 |
| ✓ | BbhSnarks73CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT251 |
| ✓ | BbhSnarks74CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT615 |
| ✓ | BbhSnarks75CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT309 |
| ✓ | BbhSnarks76CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT272 |
| × | BbhSnarks77CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks77CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 042 OUT1 306 |
| ✓ | BbhSnarks78CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT571 |
| ✓ | BbhSnarks79CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT285 |
| ✓ | BbhSnarks80CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT338 |
| ✓ | BbhSnarks81CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT229 |
| ✓ | BbhSnarks82CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT1 597 |
| ✓ | BbhSnarks83CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT278 |
| ✓ | BbhSnarks84CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT716 |
| ✓ | BbhSnarks85CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT407 |
| – | BbhSnarks86CIWhich statement is sarcastic?0 AI queries
0 tool callscategory:Reasoninglevel:Hardsource:BBH Skipped: non-deterministic answer: both options are identical |
Skipped | IN0 OUT0 |
| ✓ | BbhSnarks87CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT926 |
| – | BbhSnarks88CIWhich statement is sarcastic?0 AI queries
0 tool callscategory:Reasoninglevel:Hardsource:BBH Skipped: non-deterministic answer: malformed item: single truncated option |
Skipped | IN0 OUT0 |
| ✓ | BbhSnarks89CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT309 |
| ✓ | BbhSnarks90CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT256 |
| ✓ | BbhSnarks91CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT258 |
| ✓ | BbhSnarks92CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT245 |
| ✓ | BbhSnarks93CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT2 425 |
| ✓ | BbhSnarks94CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT313 |
| ✓ | BbhSnarks95CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT271 |
| ✓ | BbhSnarks96CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT770 |
Reasoning - BBH snarks - detect sarcasm (set 4)
29 passed3 failed |
90.6% |
IN31 981 OUT12 529 | |
| ✓ | BbhSnarks100CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT376 |
| ✓ | BbhSnarks101CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT300 |
| ✓ | BbhSnarks102CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT169 |
| ✓ | BbhSnarks103CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT427 |
| ✓ | BbhSnarks104CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT289 |
| ✓ | BbhSnarks105CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT332 |
| ✓ | BbhSnarks106CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT371 |
| ✓ | BbhSnarks107CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT501 |
| ✓ | BbhSnarks108CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT333 |
| ✓ | BbhSnarks109CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT910 |
| ✓ | BbhSnarks110CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT358 |
| ✓ | BbhSnarks111CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT336 |
| ✓ | BbhSnarks112CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT343 |
| ✓ | BbhSnarks113CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT318 |
| ✓ | BbhSnarks114CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT286 |
| ✓ | BbhSnarks115CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT268 |
| ✓ | BbhSnarks116CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT194 |
| × | BbhSnarks117CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks117CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 001 OUT1 071 |
| ✓ | BbhSnarks118CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT245 |
| ✓ | BbhSnarks119CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT297 |
| ✓ | BbhSnarks120CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT252 |
| ✓ | BbhSnarks121CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT287 |
| ✓ | BbhSnarks122CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT383 |
| ✓ | BbhSnarks123CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT222 |
| ✓ | BbhSnarks124CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT440 |
| × | BbhSnarks125CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks125CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 002 OUT295 |
| ✓ | BbhSnarks126CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT262 |
| ✓ | BbhSnarks127CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT261 |
| × | BbhSnarks128CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks128CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 000 OUT656 |
| ✓ | BbhSnarks97CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT270 |
| ✓ | BbhSnarks98CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT344 |
| ✓ | BbhSnarks99CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 133 |
Reasoning - BBH snarks - detect sarcasm (set 5)
30 passed |
100.0% |
IN30 115 OUT11 485 | |
| ✓ | BbhSnarks129CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT413 |
| ✓ | BbhSnarks130CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 037 |
| ✓ | BbhSnarks131CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT352 |
| ✓ | BbhSnarks132CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT248 |
| ✓ | BbhSnarks133CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT308 |
| ✓ | BbhSnarks134CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT214 |
| ✓ | BbhSnarks135CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT300 |
| ✓ | BbhSnarks136CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT337 |
| ✓ | BbhSnarks137CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT274 |
| ✓ | BbhSnarks138CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT309 |
| ✓ | BbhSnarks139CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT353 |
| ✓ | BbhSnarks140CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT279 |
| ✓ | BbhSnarks141CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT339 |
| ✓ | BbhSnarks142CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT243 |
| ✓ | BbhSnarks143CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT268 |
| ✓ | BbhSnarks144CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT339 |
| ✓ | BbhSnarks145CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT287 |
| ✓ | BbhSnarks146CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT244 |
| ✓ | BbhSnarks147CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT272 |
| ✓ | BbhSnarks148CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT823 |
| ✓ | BbhSnarks149CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT199 |
| ✓ | BbhSnarks150CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT610 |
| ✓ | BbhSnarks151CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT365 |
| ✓ | BbhSnarks152CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT436 |
| ✓ | BbhSnarks153CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT337 |
| ✓ | BbhSnarks154CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT871 |
| ✓ | BbhSnarks155CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT340 |
| ✓ | BbhSnarks156CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT321 |
| ✓ | BbhSnarks157CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT347 |
| ✓ | BbhSnarks158CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT420 |
Reasoning - BBH snarks - detect sarcasm (set 6)
29 passed3 failed |
90.6% |
IN31 659 OUT13 868 | |
| ✓ | BbhSnarks159CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT256 |
| ✓ | BbhSnarks160CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT393 |
| ✓ | BbhSnarks161CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT232 |
| ✓ | BbhSnarks162CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT634 |
| ✓ | BbhSnarks163CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT254 |
| ✓ | BbhSnarks164CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT453 |
| ✓ | BbhSnarks165CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT241 |
| ✓ | BbhSnarks166CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT311 |
| ✓ | BbhSnarks167CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT280 |
| × | BbhSnarks168CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks168CJ expected (B), got [a]
Values:
got = "a" |
Failed | IN992 OUT808 |
| ✓ | BbhSnarks169CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT431 |
| ✓ | BbhSnarks170CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT387 |
| ✓ | BbhSnarks171CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT260 |
| ✓ | BbhSnarks172CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT898 |
| ✓ | BbhSnarks173CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT286 |
| ✓ | BbhSnarks174CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT270 |
| ✓ | BbhSnarks175CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT270 |
| ✓ | BbhSnarks176CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT180 |
| ✓ | BbhSnarks177CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT321 |
| ✓ | BbhSportsUnderstanding10CJIs the following sentence plausible? "Carson Wentz took to the ice."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN971 OUT329 |
| ✓ | BbhSportsUnderstanding11CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN974 OUT319 |
| ✓ | BbhSportsUnderstanding12CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN977 OUT895 |
| ✓ | BbhSportsUnderstanding13CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT298 |
| ✓ | BbhSportsUnderstanding14CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT389 |
| ✓ | BbhSportsUnderstanding15CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN975 OUT1 076 |
| ✓ | BbhSportsUnderstanding16CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN971 OUT206 |
| ✓ | BbhSportsUnderstanding17CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT598 |
| ✓ | BbhSportsUnderstanding18CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT203 |
| × | BbhSportsUnderstanding19CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding19CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN976 OUT305 |
| ✓ | BbhSportsUnderstanding20CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT203 |
| × | BbhSportsUnderstanding8CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding8CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN975 OUT1 259 |
| ✓ | BbhSportsUnderstanding9CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN971 OUT623 |
Reasoning - BBH temporal sequences (set 1)
30 passed2 failed |
93.8% |
IN35 191 OUT13 554 | |
| ✓ | BbhSportsUnderstanding243CKIs the following sentence plausible? "Blake Snell hit a single."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT344 |
| × | BbhSportsUnderstanding244CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding244CK expected yes, got [no]
Values:
got = "no" |
Failed | IN976 OUT428 |
| ✓ | BbhSportsUnderstanding245CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN974 OUT241 |
| ✓ | BbhSportsUnderstanding246CKIs the following sentence plausible? "Pepe converted the first down."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT798 |
| × | BbhSportsUnderstanding247CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding247CK expected yes, got [no]
Values:
got = "no" |
Failed | IN977 OUT804 |
| ✓ | BbhSportsUnderstanding248CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN971 OUT369 |
| ✓ | BbhSportsUnderstanding249CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT299 |
| ✓ | BbhTemporalSequences10CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT353 |
| ✓ | BbhTemporalSequences11CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT424 |
| ✓ | BbhTemporalSequences12CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT449 |
| ✓ | BbhTemporalSequences13CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT417 |
| ✓ | BbhTemporalSequences14CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT449 |
| ✓ | BbhTemporalSequences15CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT506 |
| ✓ | BbhTemporalSequences16CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT561 |
| ✓ | BbhTemporalSequences17CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT381 |
| ✓ | BbhTemporalSequences18CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT395 |
| ✓ | BbhTemporalSequences19CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT491 |
| ✓ | BbhTemporalSequences20CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT366 |
| ✓ | BbhTemporalSequences21CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT351 |
| ✓ | BbhTemporalSequences22CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT282 |
| ✓ | BbhTemporalSequences23CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT402 |
| ✓ | BbhTemporalSequences24CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT319 |
| ✓ | BbhTemporalSequences25CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT407 |
| ✓ | BbhTemporalSequences26CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT348 |
| ✓ | BbhTemporalSequences27CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT461 |
| ✓ | BbhTemporalSequences28CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT463 |
| ✓ | BbhTemporalSequences29CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT465 |
| ✓ | BbhTemporalSequences30CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT358 |
| ✓ | BbhTemporalSequences31CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT401 |
| ✓ | BbhTemporalSequences32CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT347 |
| ✓ | BbhTemporalSequences8CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 159 OUT485 |
| ✓ | BbhTemporalSequences9CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT390 |
Reasoning - BBH temporal sequences (set 2)
30 passed |
100.0% |
IN33 963 OUT13 003 | |
| ✓ | BbhTemporalSequences33CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT414 |
| ✓ | BbhTemporalSequences34CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT475 |
| ✓ | BbhTemporalSequences35CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT376 |
| ✓ | BbhTemporalSequences36CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT370 |
| ✓ | BbhTemporalSequences37CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT388 |
| ✓ | BbhTemporalSequences38CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT430 |
| ✓ | BbhTemporalSequences39CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT416 |
| ✓ | BbhTemporalSequences40CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT453 |
| ✓ | BbhTemporalSequences41CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT553 |
| ✓ | BbhTemporalSequences42CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT425 |
| ✓ | BbhTemporalSequences43CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT352 |
| ✓ | BbhTemporalSequences44CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT544 |
| ✓ | BbhTemporalSequences45CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT458 |
| ✓ | BbhTemporalSequences46CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT500 |
| ✓ | BbhTemporalSequences47CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT425 |
| ✓ | BbhTemporalSequences48CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT436 |
| ✓ | BbhTemporalSequences49CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT516 |
| ✓ | BbhTemporalSequences50CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT406 |
| ✓ | BbhTemporalSequences51CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT424 |
| ✓ | BbhTemporalSequences52CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT388 |
| ✓ | BbhTemporalSequences53CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT406 |
| ✓ | BbhTemporalSequences54CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT390 |
| ✓ | BbhTemporalSequences55CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT547 |
| ✓ | BbhTemporalSequences56CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 158 OUT401 |
| ✓ | BbhTemporalSequences57CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT398 |
| ✓ | BbhTemporalSequences58CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT553 |
| ✓ | BbhTemporalSequences59CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT408 |
| ✓ | BbhTemporalSequences60CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT430 |
| ✓ | BbhTemporalSequences61CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT347 |
| ✓ | BbhTemporalSequences62CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT374 |
Reasoning - BBH temporal sequences (set 3)
32 passed |
100.0% |
IN36 175 OUT13 189 | |
| ✓ | BbhTemporalSequences63CLfind the time window when Samantha could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT351 |
| ✓ | BbhTemporalSequences64CLfind the time window when Betty could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT476 |
| ✓ | BbhTemporalSequences65CLfind the time window when William could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT474 |
| ✓ | BbhTemporalSequences66CLfind the time window when James could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT368 |
| ✓ | BbhTemporalSequences67CLfind the time window when Samantha could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT518 |
| ✓ | BbhTemporalSequences68CLfind the time window when William could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT427 |
| ✓ | BbhTemporalSequences69CLfind the time window when Kimberly could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT337 |
| ✓ | BbhTemporalSequences70CLfind the time window when Susan could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT359 |
| ✓ | BbhTemporalSequences71CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT500 |
| ✓ | BbhTemporalSequences72CLfind the time window when Ashley could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT400 |
| ✓ | BbhTemporalSequences73CLfind the time window when Jennifer could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT402 |
| ✓ | BbhTemporalSequences74CLfind the time window when James could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT331 |
| ✓ | BbhTemporalSequences75CLfind the time window when Anthony could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT417 |
| ✓ | BbhTemporalSequences76CLfind the time window when Thomas could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 159 OUT431 |
| ✓ | BbhTemporalSequences77CLfind the time window when Jennifer could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT344 |
| ✓ | BbhTemporalSequences78CLfind the time window when Sarah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT302 |
| ✓ | BbhTemporalSequences79CLfind the time window when Susan could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT423 |
| ✓ | BbhTemporalSequences80CLfind the time window when William could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT379 |
| ✓ | BbhTemporalSequences81CLfind the time window when Jessica could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT462 |
| ✓ | BbhTemporalSequences82CLfind the time window when Kimberly could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT353 |
| ✓ | BbhTemporalSequences83CLfind the time window when Elizabeth could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT317 |
| ✓ | BbhTemporalSequences84CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT579 |
| ✓ | BbhTemporalSequences85CLfind the time window when Michael could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT434 |
| ✓ | BbhTemporalSequences86CLfind the time window when Sarah could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 158 OUT464 |
| ✓ | BbhTemporalSequences87CLfind the time window when Mark could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 159 OUT468 |
| ✓ | BbhTemporalSequences88CLfind the time window when Sean could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT647 |
| ✓ | BbhTemporalSequences89CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT505 |
| ✓ | BbhTemporalSequences90CLfind the time window when Jessica could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT352 |
| ✓ | BbhTemporalSequences91CLfind the time window when Tiffany could have gone to the library1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT416 |
| ✓ | BbhTemporalSequences92CLfind the time window when James could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT345 |
| ✓ | BbhTemporalSequences93CLfind the time window when Nancy could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT270 |
| ✓ | BbhTemporalSequences94CLfind the time window when Tiffany could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT338 |
Reasoning - BBH temporal sequences (set 4)
32 passed |
100.0% |
IN36 069 OUT14 279 | |
| ✓ | BbhTemporalSequences100CLfind the time window when Steven could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT401 |
| ✓ | BbhTemporalSequences101CLfind the time window when Jessica could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT501 |
| ✓ | BbhTemporalSequences102CLfind the time window when Jessica could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT331 |
| ✓ | BbhTemporalSequences103CLfind the time window when Emily could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT527 |
| ✓ | BbhTemporalSequences104CLfind the time window when Anthony could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT473 |
| ✓ | BbhTemporalSequences105CLfind the time window when Mark could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT374 |
| ✓ | BbhTemporalSequences106CLfind the time window when William could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT434 |
| ✓ | BbhTemporalSequences107CLfind the time window when Hannah could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT453 |
| ✓ | BbhTemporalSequences108CLfind the time window when William could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT477 |
| ✓ | BbhTemporalSequences109CLfind the time window when Nancy could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT355 |
| ✓ | BbhTemporalSequences110CLfind the time window when Andrew could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT328 |
| ✓ | BbhTemporalSequences111CLfind the time window when David could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT407 |
| ✓ | BbhTemporalSequences112CLfind the time window when Anthony could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT334 |
| ✓ | BbhTemporalSequences113CLfind the time window when Sarah could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT332 |
| ✓ | BbhTemporalSequences114CLfind the time window when Mary could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT392 |
| ✓ | BbhTemporalSequences115CLfind the time window when Mark could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT429 |
| ✓ | BbhTemporalSequences116CLfind the time window when William could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT339 |
| ✓ | BbhTemporalSequences117CLfind the time window when Steven could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT399 |
| ✓ | BbhTemporalSequences118CLfind the time window when Leslie could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT398 |
| ✓ | BbhTemporalSequences119CLfind the time window when Sarah could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT419 |
| ✓ | BbhTemporalSequences120CLfind the time window when Andrew could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT521 |
| ✓ | BbhTemporalSequences121CLfind the time window when David could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT379 |
| ✓ | BbhTemporalSequences122CLfind the time window when Jessica could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 158 OUT485 |
| ✓ | BbhTemporalSequences123CLfind the time window when David could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT480 |
| ✓ | BbhTemporalSequences124CLfind the time window when Jessica could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 025 |
| ✓ | BbhTemporalSequences125CLfind the time window when Mary could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT497 |
| ✓ | BbhTemporalSequences126CLfind the time window when Michael could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT426 |
| ✓ | BbhTemporalSequences95CLfind the time window when Jennifer could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT409 |
| ✓ | BbhTemporalSequences96CLfind the time window when Sean could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT507 |
| ✓ | BbhTemporalSequences97CLfind the time window when Samantha could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT548 |
| ✓ | BbhTemporalSequences98CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT397 |
| ✓ | BbhTemporalSequences99CLfind the time window when Sarah could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 139 OUT502 |
Reasoning - BBH temporal sequences (set 5)
32 passed |
100.0% |
IN36 213 OUT13 191 | |
| ✓ | BbhTemporalSequences127CLfind the time window when David could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT306 |
| ✓ | BbhTemporalSequences128CLfind the time window when Andrew could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT336 |
| ✓ | BbhTemporalSequences129CLfind the time window when Leslie could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT399 |
| ✓ | BbhTemporalSequences130CLfind the time window when Mary could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT357 |
| ✓ | BbhTemporalSequences131CLfind the time window when Ashley could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT490 |
| ✓ | BbhTemporalSequences132CLfind the time window when Samantha could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT348 |
| ✓ | BbhTemporalSequences133CLfind the time window when Elizabeth could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT626 |
| ✓ | BbhTemporalSequences134CLfind the time window when Sean could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT428 |
| ✓ | BbhTemporalSequences135CLfind the time window when Elizabeth could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT387 |
| ✓ | BbhTemporalSequences136CLfind the time window when Michael could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT406 |
| ✓ | BbhTemporalSequences137CLfind the time window when Andrew could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT396 |
| ✓ | BbhTemporalSequences138CLfind the time window when Nancy could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT342 |
| ✓ | BbhTemporalSequences139CLfind the time window when Jason could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT342 |
| ✓ | BbhTemporalSequences140CLfind the time window when Anthony could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT586 |
| ✓ | BbhTemporalSequences141CLfind the time window when Leslie could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT409 |
| ✓ | BbhTemporalSequences142CLfind the time window when David could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT518 |
| ✓ | BbhTemporalSequences143CLfind the time window when Hannah could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT382 |
| ✓ | BbhTemporalSequences144CLfind the time window when Sarah could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT457 |
| ✓ | BbhTemporalSequences145CLfind the time window when Andrew could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT426 |
| ✓ | BbhTemporalSequences146CLfind the time window when William could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT427 |
| ✓ | BbhTemporalSequences147CLfind the time window when Elizabeth could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT452 |
| ✓ | BbhTemporalSequences148CLfind the time window when Sarah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT520 |
| ✓ | BbhTemporalSequences149CLfind the time window when David could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT411 |
| ✓ | BbhTemporalSequences150CLfind the time window when Jason could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT352 |
| ✓ | BbhTemporalSequences151CLfind the time window when Anthony could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT406 |
| ✓ | BbhTemporalSequences152CLfind the time window when Sarah could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT411 |
| ✓ | BbhTemporalSequences153CLfind the time window when Hannah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT407 |
| ✓ | BbhTemporalSequences154CLfind the time window when Mark could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT436 |
| ✓ | BbhTemporalSequences155CLfind the time window when Sarah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT320 |
| ✓ | BbhTemporalSequences156CLfind the time window when James could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT398 |
| ✓ | BbhTemporalSequences157CLfind the time window when Emily could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT371 |
| ✓ | BbhTemporalSequences158CLfind the time window when David could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT339 |
Reasoning - BBH temporal sequences (set 6)
32 passed |
100.0% |
IN36 181 OUT12 814 | |
| ✓ | BbhTemporalSequences159CLfind the time window when Jennifer could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT522 |
| ✓ | BbhTemporalSequences160CLfind the time window when Ashley could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT541 |
| ✓ | BbhTemporalSequences161CLfind the time window when William could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT348 |
| ✓ | BbhTemporalSequences162CLfind the time window when Jennifer could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT352 |
| ✓ | BbhTemporalSequences163CLfind the time window when Jennifer could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT371 |
| ✓ | BbhTemporalSequences164CLfind the time window when Jennifer could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT370 |
| ✓ | BbhTemporalSequences165CLfind the time window when Nancy could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT447 |
| ✓ | BbhTemporalSequences166CLfind the time window when Samantha could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT393 |
| ✓ | BbhTemporalSequences167CLfind the time window when Tiffany could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT393 |
| ✓ | BbhTemporalSequences168CLfind the time window when Leslie could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT338 |
| ✓ | BbhTemporalSequences169CLfind the time window when Hannah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT374 |
| ✓ | BbhTemporalSequences170CLfind the time window when Anthony could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT336 |
| ✓ | BbhTemporalSequences171CLfind the time window when Hannah could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT420 |
| ✓ | BbhTemporalSequences172CLfind the time window when Leslie could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT640 |
| ✓ | BbhTemporalSequences173CLfind the time window when Anthony could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT321 |
| ✓ | BbhTemporalSequences174CLfind the time window when Thomas could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT551 |
| ✓ | BbhTemporalSequences175CLfind the time window when Jason could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT327 |
| ✓ | BbhTemporalSequences176CLfind the time window when Betty could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT325 |
| ✓ | BbhTemporalSequences177CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT351 |
| ✓ | BbhTemporalSequences178CLfind the time window when David could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT327 |
| ✓ | BbhTemporalSequences179CLfind the time window when Leslie could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT367 |
| ✓ | BbhTemporalSequences180CLfind the time window when Ashley could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT344 |
| ✓ | BbhTemporalSequences181CLfind the time window when Michael could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT418 |
| ✓ | BbhTemporalSequences182CLfind the time window when Jason could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT416 |
| ✓ | BbhTemporalSequences183CLfind the time window when Sean could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT354 |
| ✓ | BbhTemporalSequences184CLfind the time window when Sarah could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT337 |
| ✓ | BbhTemporalSequences185CLfind the time window when Andrew could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT469 |
| ✓ | BbhTemporalSequences186CLfind the time window when James could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT412 |
| ✓ | BbhTemporalSequences187CLfind the time window when Ashley could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT480 |
| ✓ | BbhTemporalSequences188CLfind the time window when Ashley could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT434 |
| ✓ | BbhTemporalSequences189CLfind the time window when Sarah could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT335 |
| ✓ | BbhTemporalSequences190CLfind the time window when Michael could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT401 |
Reasoning - BBH temporal sequences (set 7)
30 passed |
100.0% |
IN33 866 OUT12 109 | |
| ✓ | BbhTemporalSequences191CLfind the time window when Anthony could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT323 |
| ✓ | BbhTemporalSequences192CLfind the time window when William could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT449 |
| ✓ | BbhTemporalSequences193CLfind the time window when Susan could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT564 |
| ✓ | BbhTemporalSequences194CLfind the time window when Sarah could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT258 |
| ✓ | BbhTemporalSequences195CLfind the time window when Emily could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT395 |
| ✓ | BbhTemporalSequences196CLfind the time window when Emily could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT318 |
| ✓ | BbhTemporalSequences197CLfind the time window when Sean could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT361 |
| ✓ | BbhTemporalSequences198CLfind the time window when Elizabeth could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT373 |
| ✓ | BbhTemporalSequences199CLfind the time window when William could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT394 |
| ✓ | BbhTemporalSequences200CLfind the time window when Thomas could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT386 |
| ✓ | BbhTemporalSequences201CLfind the time window when Susan could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT346 |
| ✓ | BbhTemporalSequences202CLfind the time window when Kimberly could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 160 OUT481 |
| ✓ | BbhTemporalSequences203CLfind the time window when Leslie could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT397 |
| ✓ | BbhTemporalSequences204CLfind the time window when William could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT612 |
| ✓ | BbhTemporalSequences205CLfind the time window when Andrew could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT379 |
| ✓ | BbhTemporalSequences206CLfind the time window when Jennifer could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT341 |
| ✓ | BbhTemporalSequences207CLfind the time window when Susan could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT408 |
| ✓ | BbhTemporalSequences208CLfind the time window when James could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT406 |
| ✓ | BbhTemporalSequences209CLfind the time window when Tiffany could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT390 |
| ✓ | BbhTemporalSequences210CLfind the time window when Andrew could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT500 |
| ✓ | BbhTemporalSequences211CLfind the time window when Thomas could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT315 |
| ✓ | BbhTemporalSequences212CLfind the time window when David could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT393 |
| ✓ | BbhTemporalSequences213CLfind the time window when Elizabeth could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT359 |
| ✓ | BbhTemporalSequences214CLfind the time window when Jason could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT457 |
| ✓ | BbhTemporalSequences215CLfind the time window when Sarah could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT427 |
| ✓ | BbhTemporalSequences216CLfind the time window when William could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT390 |
| ✓ | BbhTemporalSequences217CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT538 |
| ✓ | BbhTemporalSequences218CLfind the time window when William could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT368 |
| ✓ | BbhTemporalSequences219CLfind the time window when Sarah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT387 |
| ✓ | BbhTemporalSequences220CLfind the time window when Nancy could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT394 |
Reasoning - BBH temporal sequences (set 8)
29 passed |
100.0% |
IN33 017 OUT12 629 | |
| ✓ | BbhTemporalSequences221CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT376 |
| ✓ | BbhTemporalSequences222CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT348 |
| ✓ | BbhTemporalSequences223CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT334 |
| ✓ | BbhTemporalSequences224CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT470 |
| ✓ | BbhTemporalSequences225CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT439 |
| ✓ | BbhTemporalSequences226CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT438 |
| ✓ | BbhTemporalSequences227CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT409 |
| ✓ | BbhTemporalSequences228CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT495 |
| ✓ | BbhTemporalSequences229CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT581 |
| ✓ | BbhTemporalSequences230CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT470 |
| ✓ | BbhTemporalSequences231CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT504 |
| ✓ | BbhTemporalSequences232CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT425 |
| ✓ | BbhTemporalSequences233CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT448 |
| ✓ | BbhTemporalSequences234CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT336 |
| ✓ | BbhTemporalSequences235CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT424 |
| ✓ | BbhTemporalSequences236CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT517 |
| ✓ | BbhTemporalSequences237CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT699 |
| ✓ | BbhTemporalSequences238CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT351 |
| ✓ | BbhTemporalSequences239CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT455 |
| ✓ | BbhTemporalSequences240CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT481 |
| ✓ | BbhTemporalSequences241CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT389 |
| ✓ | BbhTemporalSequences242CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT404 |
| ✓ | BbhTemporalSequences243CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT403 |
| ✓ | BbhTemporalSequences244CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT352 |
| ✓ | BbhTemporalSequences245CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT521 |
| ✓ | BbhTemporalSequences246CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT434 |
| ✓ | BbhTemporalSequences247CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT382 |
| ✓ | BbhTemporalSequences248CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT378 |
| ✓ | BbhTemporalSequences249CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT366 |
Reasoning - BBH tracking shuffled objects (3) (set 1)
30 passed |
100.0% |
IN32 047 OUT8 970 | |
| ✓ | BbhTrackingThree10CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree11CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT292 |
| ✓ | BbhTrackingThree12CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT274 |
| ✓ | BbhTrackingThree13CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackingThree14CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree15CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT362 |
| ✓ | BbhTrackingThree16CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT315 |
| ✓ | BbhTrackingThree17CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT311 |
| ✓ | BbhTrackingThree18CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT291 |
| ✓ | BbhTrackingThree19CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT316 |
| ✓ | BbhTrackingThree20CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT372 |
| ✓ | BbhTrackingThree21CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT312 |
| ✓ | BbhTrackingThree22CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT345 |
| ✓ | BbhTrackingThree23CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree24CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT318 |
| ✓ | BbhTrackingThree25CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT313 |
| ✓ | BbhTrackingThree26CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackingThree27CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackingThree28CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT316 |
| ✓ | BbhTrackingThree29CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT411 |
| ✓ | BbhTrackingThree30CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT264 |
| ✓ | BbhTrackingThree31CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT317 |
| ✓ | BbhTrackingThree32CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT326 |
| ✓ | BbhTrackingThree33CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT278 |
| ✓ | BbhTrackingThree34CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT322 |
| ✓ | BbhTrackingThree35CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree36CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree37CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT270 |
| ✓ | BbhTrackingThree8CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT274 |
| ✓ | BbhTrackingThree9CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT275 |
Reasoning - BBH tracking shuffled objects (3) (set 2)
30 passed |
100.0% |
IN32 058 OUT9 003 | |
| ✓ | BbhTrackingThree38CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree39CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT433 |
| ✓ | BbhTrackingThree40CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree41CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT299 |
| ✓ | BbhTrackingThree42CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT287 |
| ✓ | BbhTrackingThree43CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree44CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT324 |
| ✓ | BbhTrackingThree45CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT268 |
| ✓ | BbhTrackingThree46CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT283 |
| ✓ | BbhTrackingThree47CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT304 |
| ✓ | BbhTrackingThree48CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT312 |
| ✓ | BbhTrackingThree49CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree50CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT324 |
| ✓ | BbhTrackingThree51CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree52CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree53CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT316 |
| ✓ | BbhTrackingThree54CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT276 |
| ✓ | BbhTrackingThree55CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree56CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT316 |
| ✓ | BbhTrackingThree57CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree58CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT340 |
| ✓ | BbhTrackingThree59CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT318 |
| ✓ | BbhTrackingThree60CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT324 |
| ✓ | BbhTrackingThree61CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT471 |
| ✓ | BbhTrackingThree62CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT316 |
| ✓ | BbhTrackingThree63CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT306 |
| ✓ | BbhTrackingThree64CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT298 |
| ✓ | BbhTrackingThree65CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT259 |
| ✓ | BbhTrackingThree66CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree67CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
Reasoning - BBH tracking shuffled objects (3) (set 3)
30 passed |
100.0% |
IN32 093 OUT9 040 | |
| ✓ | BbhTrackingThree68CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT270 |
| ✓ | BbhTrackingThree69CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT291 |
| ✓ | BbhTrackingThree70CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree71CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT283 |
| ✓ | BbhTrackingThree72CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT275 |
| ✓ | BbhTrackingThree73CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT317 |
| ✓ | BbhTrackingThree74CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree75CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackingThree76CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT271 |
| ✓ | BbhTrackingThree77CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT346 |
| ✓ | BbhTrackingThree78CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree79CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT310 |
| ✓ | BbhTrackingThree80CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT366 |
| ✓ | BbhTrackingThree81CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT312 |
| ✓ | BbhTrackingThree82CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT316 |
| ✓ | BbhTrackingThree83CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT329 |
| ✓ | BbhTrackingThree84CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT355 |
| ✓ | BbhTrackingThree85CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree86CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT331 |
| ✓ | BbhTrackingThree87CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT290 |
| ✓ | BbhTrackingThree88CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT304 |
| ✓ | BbhTrackingThree89CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT267 |
| ✓ | BbhTrackingThree90CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT276 |
| ✓ | BbhTrackingThree91CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT305 |
| ✓ | BbhTrackingThree92CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT290 |
| ✓ | BbhTrackingThree93CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT345 |
| ✓ | BbhTrackingThree94CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT253 |
| ✓ | BbhTrackingThree95CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT371 |
| ✓ | BbhTrackingThree96CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT332 |
| ✓ | BbhTrackingThree97CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT322 |
Reasoning - BBH tracking shuffled objects (3) (set 4)
29 passed |
100.0% |
IN30 961 OUT8 754 | |
| ✓ | BbhTrackingThree100CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree101CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT349 |
| ✓ | BbhTrackingThree102CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT278 |
| ✓ | BbhTrackingThree103CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree104CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT259 |
| ✓ | BbhTrackingThree105CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT282 |
| ✓ | BbhTrackingThree106CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree107CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT273 |
| ✓ | BbhTrackingThree108CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT275 |
| ✓ | BbhTrackingThree109CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT253 |
| ✓ | BbhTrackingThree110CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT316 |
| ✓ | BbhTrackingThree111CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT297 |
| ✓ | BbhTrackingThree112CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT292 |
| ✓ | BbhTrackingThree113CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT294 |
| ✓ | BbhTrackingThree114CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT277 |
| ✓ | BbhTrackingThree115CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT270 |
| ✓ | BbhTrackingThree116CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree117CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT291 |
| ✓ | BbhTrackingThree118CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT541 |
| ✓ | BbhTrackingThree119CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT257 |
| ✓ | BbhTrackingThree120CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT344 |
| ✓ | BbhTrackingThree121CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT305 |
| ✓ | BbhTrackingThree122CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT496 |
| ✓ | BbhTrackingThree123CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree124CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT313 |
| ✓ | BbhTrackingThree125CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT311 |
| ✓ | BbhTrackingThree126CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT311 |
| ✓ | BbhTrackingThree98CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree99CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT292 |
Reasoning - BBH tracking shuffled objects (3) (set 5)
32 passed |
100.0% |
IN34 243 OUT9 915 | |
| ✓ | BbhTrackThree127CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackThree128CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT290 |
| ✓ | BbhTrackThree129CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT314 |
| ✓ | BbhTrackThree130CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT283 |
| ✓ | BbhTrackThree131CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT353 |
| ✓ | BbhTrackThree132CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackThree133CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT270 |
| ✓ | BbhTrackThree134CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT278 |
| ✓ | BbhTrackThree135CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT412 |
| ✓ | BbhTrackThree136CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT310 |
| ✓ | BbhTrackThree137CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT295 |
| ✓ | BbhTrackThree138CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree139CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT311 |
| ✓ | BbhTrackThree140CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT304 |
| ✓ | BbhTrackThree141CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT309 |
| ✓ | BbhTrackThree142CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT325 |
| ✓ | BbhTrackThree143CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree144CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT317 |
| ✓ | BbhTrackThree145CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree146CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT273 |
| ✓ | BbhTrackThree147CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT246 |
| ✓ | BbhTrackThree148CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT299 |
| ✓ | BbhTrackThree149CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT360 |
| ✓ | BbhTrackThree150CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackThree151CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT314 |
| ✓ | BbhTrackThree152CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackThree153CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT267 |
| ✓ | BbhTrackThree154CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT714 |
| ✓ | BbhTrackThree155CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT298 |
| ✓ | BbhTrackThree156CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT306 |
| ✓ | BbhTrackThree157CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT291 |
| ✓ | BbhTrackThree158CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT333 |
Reasoning - BBH tracking shuffled objects (3) (set 6)
32 passed |
100.0% |
IN34 151 OUT9 638 | |
| ✓ | BbhTrackThree159CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree160CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT313 |
| ✓ | BbhTrackThree161CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT305 |
| ✓ | BbhTrackThree162CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree163CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT268 |
| ✓ | BbhTrackThree164CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree165CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree166CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT312 |
| ✓ | BbhTrackThree167CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT350 |
| ✓ | BbhTrackThree168CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT288 |
| ✓ | BbhTrackThree169CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT329 |
| ✓ | BbhTrackThree170CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT312 |
| ✓ | BbhTrackThree171CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT316 |
| ✓ | BbhTrackThree172CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackThree173CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT307 |
| ✓ | BbhTrackThree174CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT304 |
| ✓ | BbhTrackThree175CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT365 |
| ✓ | BbhTrackThree176CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT275 |
| ✓ | BbhTrackThree177CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT290 |
| ✓ | BbhTrackThree178CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree179CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT325 |
| ✓ | BbhTrackThree180CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT356 |
| ✓ | BbhTrackThree181CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT264 |
| ✓ | BbhTrackThree182CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT292 |
| ✓ | BbhTrackThree183CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT332 |
| ✓ | BbhTrackThree184CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT331 |
| ✓ | BbhTrackThree185CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT297 |
| ✓ | BbhTrackThree186CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT341 |
| ✓ | BbhTrackThree187CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT321 |
| ✓ | BbhTrackThree188CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT305 |
| ✓ | BbhTrackThree189CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackThree190CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT306 |
Reasoning - BBH tracking shuffled objects (3) (set 7)
32 passed |
100.0% |
IN34 170 OUT9 386 | |
| ✓ | BbhTrackThree191CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT306 |
| ✓ | BbhTrackThree192CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree193CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT312 |
| ✓ | BbhTrackThree194CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT253 |
| ✓ | BbhTrackThree195CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT277 |
| ✓ | BbhTrackThree196CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT310 |
| ✓ | BbhTrackThree197CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT354 |
| ✓ | BbhTrackThree198CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT314 |
| ✓ | BbhTrackThree199CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT375 |
| ✓ | BbhTrackThree200CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT343 |
| ✓ | BbhTrackThree201CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree202CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT270 |
| ✓ | BbhTrackThree203CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT288 |
| ✓ | BbhTrackThree204CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT280 |
| ✓ | BbhTrackThree205CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT379 |
| ✓ | BbhTrackThree206CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT280 |
| ✓ | BbhTrackThree207CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT324 |
| ✓ | BbhTrackThree208CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT307 |
| ✓ | BbhTrackThree209CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT258 |
| ✓ | BbhTrackThree210CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT305 |
| ✓ | BbhTrackThree211CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT334 |
| ✓ | BbhTrackThree212CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT253 |
| ✓ | BbhTrackThree213CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree214CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackThree215CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree216CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT282 |
| ✓ | BbhTrackThree217CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT300 |
| ✓ | BbhTrackThree218CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT292 |
| ✓ | BbhTrackThree219CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT291 |
| ✓ | BbhTrackThree220CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT262 |
| ✓ | BbhTrackThree221CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree222CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
Reasoning - BBH tracking shuffled objects (3) (set 8)
27 passed |
100.0% |
IN28 834 OUT8 528 | |
| ✓ | BbhTrackThree223CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree224CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT519 |
| ✓ | BbhTrackThree225CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT331 |
| ✓ | BbhTrackThree226CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT399 |
| ✓ | BbhTrackThree227CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT258 |
| ✓ | BbhTrackThree228CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT273 |
| ✓ | BbhTrackThree229CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT277 |
| ✓ | BbhTrackThree230CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT289 |
| ✓ | BbhTrackThree231CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT319 |
| ✓ | BbhTrackThree232CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree233CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT348 |
| ✓ | BbhTrackThree234CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT326 |
| ✓ | BbhTrackThree235CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT317 |
| ✓ | BbhTrackThree236CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT334 |
| ✓ | BbhTrackThree237CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT289 |
| ✓ | BbhTrackThree238CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT306 |
| ✓ | BbhTrackThree239CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT305 |
| ✓ | BbhTrackThree240CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT309 |
| ✓ | BbhTrackThree241CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT308 |
| ✓ | BbhTrackThree242CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree243CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT316 |
| ✓ | BbhTrackThree244CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT345 |
| ✓ | BbhTrackThree245CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree246CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT324 |
| ✓ | BbhTrackThree247CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT380 |
| ✓ | BbhTrackThree248CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT312 |
| ✓ | BbhTrackThree249CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT292 |
Reasoning - BBH tracking shuffled objects (5) (set 1)
33 passed |
100.0% |
IN36 717 OUT19 220 | |
| ✓ | BbhTrackingShuffledFive10CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT436 |
| ✓ | BbhTrackingShuffledFive11CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT402 |
| ✓ | BbhTrackingShuffledFive12CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT922 |
| ✓ | BbhTrackingShuffledFive13CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT521 |
| ✓ | BbhTrackingShuffledFive14CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 128 |
| ✓ | BbhTrackingShuffledFive15CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT415 |
| ✓ | BbhTrackingShuffledFive16CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT634 |
| ✓ | BbhTrackingShuffledFive17CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT626 |
| ✓ | BbhTrackingShuffledFive18CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT537 |
| ✓ | BbhTrackingShuffledFive19CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT501 |
| ✓ | BbhTrackingShuffledFive20CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT611 |
| ✓ | BbhTrackingShuffledFive21CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT620 |
| ✓ | BbhTrackingShuffledFive22CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT607 |
| ✓ | BbhTrackingShuffledFive23CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT439 |
| ✓ | BbhTrackingShuffledFive24CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT517 |
| ✓ | BbhTrackingShuffledFive25CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT519 |
| ✓ | BbhTrackingShuffledFive26CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT492 |
| ✓ | BbhTrackingShuffledFive27CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT568 |
| ✓ | BbhTrackingShuffledFive28CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT495 |
| ✓ | BbhTrackingShuffledFive29CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT542 |
| ✓ | BbhTrackingShuffledFive30CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT614 |
| ✓ | BbhTrackingShuffledFive31CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT877 |
| ✓ | BbhTrackingShuffledFive32CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT477 |
| ✓ | BbhTrackingShuffledFive33CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT638 |
| ✓ | BbhTrackingShuffledFive34CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT494 |
| ✓ | BbhTrackingShuffledFive35CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT610 |
| ✓ | BbhTrackingShuffledFive36CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT464 |
| ✓ | BbhTrackingShuffledFive37CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT640 |
| ✓ | BbhTrackingShuffledFive38CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT494 |
| ✓ | BbhTrackingShuffledFive39CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT650 |
| ✓ | BbhTrackingShuffledFive40CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT379 |
| ✓ | BbhTrackingShuffledFive8CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT648 |
| ✓ | BbhTrackingShuffledFive9CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT703 |
Reasoning - BBH tracking shuffled objects (5) (set 2)
33 passed |
100.0% |
IN36 623 OUT18 043 | |
| ✓ | BbhTrackingShuffledFive41CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT566 |
| ✓ | BbhTrackingShuffledFive42CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT518 |
| ✓ | BbhTrackingShuffledFive43CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT623 |
| ✓ | BbhTrackingShuffledFive44CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT653 |
| ✓ | BbhTrackingShuffledFive45CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT459 |
| ✓ | BbhTrackingShuffledFive46CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT625 |
| ✓ | BbhTrackingShuffledFive47CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT524 |
| ✓ | BbhTrackingShuffledFive48CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT604 |
| ✓ | BbhTrackingShuffledFive49CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT487 |
| ✓ | BbhTrackingShuffledFive50CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT573 |
| ✓ | BbhTrackingShuffledFive51CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT535 |
| ✓ | BbhTrackingShuffledFive52CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT521 |
| ✓ | BbhTrackingShuffledFive53CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT666 |
| ✓ | BbhTrackingShuffledFive54CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT378 |
| ✓ | BbhTrackingShuffledFive55CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT458 |
| ✓ | BbhTrackingShuffledFive56CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT589 |
| ✓ | BbhTrackingShuffledFive57CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT503 |
| ✓ | BbhTrackingShuffledFive58CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT615 |
| ✓ | BbhTrackingShuffledFive59CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT491 |
| ✓ | BbhTrackingShuffledFive60CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT618 |
| ✓ | BbhTrackingShuffledFive61CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT663 |
| ✓ | BbhTrackingShuffledFive62CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT462 |
| ✓ | BbhTrackingShuffledFive63CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT538 |
| ✓ | BbhTrackingShuffledFive64CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT410 |
| ✓ | BbhTrackingShuffledFive65CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT591 |
| ✓ | BbhTrackingShuffledFive66CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT527 |
| ✓ | BbhTrackingShuffledFive67CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT733 |
| ✓ | BbhTrackingShuffledFive68CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT435 |
| ✓ | BbhTrackingShuffledFive69CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT558 |
| ✓ | BbhTrackingShuffledFive70CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT512 |
| ✓ | BbhTrackingShuffledFive71CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT519 |
| ✓ | BbhTrackingShuffledFive72CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT436 |
| ✓ | BbhTrackingShuffledFive73CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT653 |
Reasoning - BBH tracking shuffled objects (5) (set 3)
32 passed |
100.0% |
IN35 590 OUT17 358 | |
| ✓ | BbhTrackingShuffledFive100CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT534 |
| ✓ | BbhTrackingShuffledFive101CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT590 |
| ✓ | BbhTrackingShuffledFive102CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT519 |
| ✓ | BbhTrackingShuffledFive103CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT529 |
| ✓ | BbhTrackingShuffledFive104CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT438 |
| ✓ | BbhTrackingShuffledFive105CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT457 |
| ✓ | BbhTrackingShuffledFive74CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT715 |
| ✓ | BbhTrackingShuffledFive75CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT533 |
| ✓ | BbhTrackingShuffledFive76CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT459 |
| ✓ | BbhTrackingShuffledFive77CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT373 |
| ✓ | BbhTrackingShuffledFive78CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT532 |
| ✓ | BbhTrackingShuffledFive79CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT661 |
| ✓ | BbhTrackingShuffledFive80CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT538 |
| ✓ | BbhTrackingShuffledFive81CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT574 |
| ✓ | BbhTrackingShuffledFive82CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT587 |
| ✓ | BbhTrackingShuffledFive83CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT465 |
| ✓ | BbhTrackingShuffledFive84CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT645 |
| ✓ | BbhTrackingShuffledFive85CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT540 |
| ✓ | BbhTrackingShuffledFive86CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT653 |
| ✓ | BbhTrackingShuffledFive87CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT546 |
| ✓ | BbhTrackingShuffledFive88CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT673 |
| ✓ | BbhTrackingShuffledFive89CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT543 |
| ✓ | BbhTrackingShuffledFive90CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT498 |
| ✓ | BbhTrackingShuffledFive91CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT482 |
| ✓ | BbhTrackingShuffledFive92CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT494 |
| ✓ | BbhTrackingShuffledFive93CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT701 |
| ✓ | BbhTrackingShuffledFive94CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT544 |
| ✓ | BbhTrackingShuffledFive95CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT464 |
| ✓ | BbhTrackingShuffledFive96CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT378 |
| ✓ | BbhTrackingShuffledFive97CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT609 |
| ✓ | BbhTrackingShuffledFive98CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT602 |
| ✓ | BbhTrackingShuffledFive99CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT482 |
Reasoning - BBH tracking shuffled objects (5) (set 4)
31 passed |
100.0% |
IN34 491 OUT17 921 | |
| ✓ | BbhTrackingShuffledFive106CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT694 |
| ✓ | BbhTrackingShuffledFive107CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT548 |
| ✓ | BbhTrackingShuffledFive108CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT470 |
| ✓ | BbhTrackingShuffledFive109CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT549 |
| ✓ | BbhTrackingShuffledFive110CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT530 |
| ✓ | BbhTrackingShuffledFive111CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT507 |
| ✓ | BbhTrackingShuffledFive112CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT760 |
| ✓ | BbhTrackingShuffledFive113CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT505 |
| ✓ | BbhTrackingShuffledFive114CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT621 |
| ✓ | BbhTrackingShuffledFive115CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT660 |
| ✓ | BbhTrackingShuffledFive116CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT659 |
| ✓ | BbhTrackingShuffledFive117CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT651 |
| ✓ | BbhTrackingShuffledFive118CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT712 |
| ✓ | BbhTrackingShuffledFive119CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT458 |
| ✓ | BbhTrackingShuffledFive120CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT663 |
| ✓ | BbhTrackingShuffledFive121CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT920 |
| ✓ | BbhTrackingShuffledFive122CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT520 |
| ✓ | BbhTrackingShuffledFive123CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT664 |
| ✓ | BbhTrackingShuffledFive124CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT505 |
| ✓ | BbhTrackingShuffledFive125CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT503 |
| ✓ | BbhTrackingShuffledFive126CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT650 |
| ✓ | BbhTrackingShuffledFive127CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT643 |
| ✓ | BbhTrackingShuffledFive128CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT564 |
| ✓ | BbhTrackingShuffledFive129CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT383 |
| ✓ | BbhTrackingShuffledFive130CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT471 |
| ✓ | BbhTrackingShuffledFive131CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT436 |
| ✓ | BbhTrackingShuffledFive132CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT458 |
| ✓ | BbhTrackingShuffledFive133CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT595 |
| ✓ | BbhTrackingShuffledFive134CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT456 |
| ✓ | BbhTrackingShuffledFive135CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT600 |
| ✓ | BbhTrackingShuffledFive136CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT566 |
Reasoning - BBH tracking shuffled objects (5) (set 5)
30 passed2 failed |
93.8% |
IN35 563 OUT17 137 | |
| ✓ | BbhTrackFive137CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT492 |
| ✓ | BbhTrackFive138CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT534 |
| ✓ | BbhTrackFive139CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT645 |
| ✓ | BbhTrackFive140CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT534 |
| ✓ | BbhTrackFive141CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT489 |
| ✓ | BbhTrackFive142CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT458 |
| ✓ | BbhTrackFive143CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT651 |
| ✓ | BbhTrackFive144CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT630 |
| ✓ | BbhTrackFive145CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT543 |
| ✓ | BbhTrackFive146CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT645 |
| ✓ | BbhTrackFive147CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT833 |
| ✓ | BbhTrackFive148CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT506 |
| ✓ | BbhTrackFive149CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT383 |
| ✓ | BbhTrackFive150CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT389 |
| ✓ | BbhTrackFive151CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT647 |
| ✓ | BbhTrackFive152CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT499 |
| ✓ | BbhTrackFive153CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT533 |
| ✓ | BbhTrackFive154CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT520 |
| ✓ | BbhTrackFive155CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT491 |
| × | BbhTrackFive156CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("")
Context: BbhTrackFive156CN: expected , got [(E)] |
Failed | IN1 105 OUT458 |
| × | BbhTrackFive157CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("")
Context: BbhTrackFive157CN: expected , got [(A)] |
Failed | IN1 117 OUT503 |
| ✓ | BbhTrackFive158CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT542 |
| ✓ | BbhTrackFive159CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT436 |
| ✓ | BbhTrackFive160CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT527 |
| ✓ | BbhTrackFive161CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT512 |
| ✓ | BbhTrackFive162CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT436 |
| ✓ | BbhTrackFive163CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT438 |
| ✓ | BbhTrackFive164CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT541 |
| ✓ | BbhTrackFive165CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT493 |
| ✓ | BbhTrackFive166CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT639 |
| ✓ | BbhTrackFive167CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT589 |
| ✓ | BbhTrackFive168CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT601 |
Reasoning - BBH tracking shuffled objects (5) (set 6)
32 passed |
100.0% |
IN35 607 OUT17 680 | |
| ✓ | BbhTrackFive169CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT555 |
| ✓ | BbhTrackFive170CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT509 |
| ✓ | BbhTrackFive171CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT636 |
| ✓ | BbhTrackFive172CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT540 |
| ✓ | BbhTrackFive173CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT534 |
| ✓ | BbhTrackFive174CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT553 |
| ✓ | BbhTrackFive175CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT463 |
| ✓ | BbhTrackFive176CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT612 |
| ✓ | BbhTrackFive177CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT552 |
| ✓ | BbhTrackFive178CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT404 |
| ✓ | BbhTrackFive179CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT647 |
| ✓ | BbhTrackFive180CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT503 |
| ✓ | BbhTrackFive181CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT528 |
| ✓ | BbhTrackFive182CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT536 |
| ✓ | BbhTrackFive183CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT462 |
| ✓ | BbhTrackFive184CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT510 |
| ✓ | BbhTrackFive185CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT648 |
| ✓ | BbhTrackFive186CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT662 |
| ✓ | BbhTrackFive187CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT549 |
| ✓ | BbhTrackFive188CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT624 |
| ✓ | BbhTrackFive189CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT507 |
| ✓ | BbhTrackFive190CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT507 |
| ✓ | BbhTrackFive191CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT464 |
| ✓ | BbhTrackFive192CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT505 |
| ✓ | BbhTrackFive193CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT474 |
| ✓ | BbhTrackFive194CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 114 |
| ✓ | BbhTrackFive195CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT507 |
| ✓ | BbhTrackFive196CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT561 |
| ✓ | BbhTrackFive197CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT474 |
| ✓ | BbhTrackFive198CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT563 |
| ✓ | BbhTrackFive199CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT552 |
| ✓ | BbhTrackFive200CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT425 |
Reasoning - BBH tracking shuffled objects (5) (set 7)
32 passed |
100.0% |
IN35 568 OUT18 266 | |
| ✓ | BbhTrackFive201CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT451 |
| ✓ | BbhTrackFive202CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT503 |
| ✓ | BbhTrackFive203CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT624 |
| ✓ | BbhTrackFive204CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT492 |
| ✓ | BbhTrackFive205CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT464 |
| ✓ | BbhTrackFive206CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT520 |
| ✓ | BbhTrackFive207CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT829 |
| ✓ | BbhTrackFive208CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT796 |
| ✓ | BbhTrackFive209CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT598 |
| ✓ | BbhTrackFive210CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT489 |
| ✓ | BbhTrackFive211CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT985 |
| ✓ | BbhTrackFive212CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT470 |
| ✓ | BbhTrackFive213CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT506 |
| ✓ | BbhTrackFive214CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT620 |
| ✓ | BbhTrackFive215CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT521 |
| ✓ | BbhTrackFive216CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT499 |
| ✓ | BbhTrackFive217CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT666 |
| ✓ | BbhTrackFive218CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT519 |
| ✓ | BbhTrackFive219CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT672 |
| ✓ | BbhTrackFive220CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT606 |
| ✓ | BbhTrackFive221CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT505 |
| ✓ | BbhTrackFive222CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT383 |
| ✓ | BbhTrackFive223CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT506 |
| ✓ | BbhTrackFive224CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT537 |
| ✓ | BbhTrackFive225CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT571 |
| ✓ | BbhTrackFive226CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT617 |
| ✓ | BbhTrackFive227CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT532 |
| ✓ | BbhTrackFive228CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT471 |
| ✓ | BbhTrackFive229CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT525 |
| ✓ | BbhTrackFive230CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT509 |
| ✓ | BbhTrackFive231CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT708 |
| ✓ | BbhTrackFive232CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT572 |
Reasoning - BBH tracking shuffled objects (5) (set 8)
32 passed |
100.0% |
IN36 272 OUT22 823 | |
| ✓ | BbhTrackFive233CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT478 |
| ✓ | BbhTrackFive234CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT659 |
| ✓ | BbhTrackFive235CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT834 |
| ✓ | BbhTrackFive236CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT458 |
| ✓ | BbhTrackFive237CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT528 |
| ✓ | BbhTrackFive238CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT384 |
| ✓ | BbhTrackFive239CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT616 |
| ✓ | BbhTrackFive240CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT503 |
| ✓ | BbhTrackFive241CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT662 |
| ✓ | BbhTrackFive242CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT600 |
| ✓ | BbhTrackFive243CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT855 |
| ✓ | BbhTrackFive244CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT582 |
| ✓ | BbhTrackFive245CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT690 |
| ✓ | BbhTrackFive246CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT511 |
| ✓ | BbhTrackFive247CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT653 |
| ✓ | BbhTrackFive248CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT509 |
| ✓ | BbhTrackFive249CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT814 |
| ✓ | BbhTrackSeven10CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT750 |
| ✓ | BbhTrackSeven11CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT838 |
| ✓ | BbhTrackSeven12CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT891 |
| ✓ | BbhTrackSeven13CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT885 |
| ✓ | BbhTrackSeven14CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT1 070 |
| ✓ | BbhTrackSeven15CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT715 |
| ✓ | BbhTrackSeven16CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT797 |
| ✓ | BbhTrackSeven17CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT817 |
| ✓ | BbhTrackSeven18CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT587 |
| ✓ | BbhTrackSeven19CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT965 |
| ✓ | BbhTrackSeven20CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT856 |
| ✓ | BbhTrackSeven21CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT884 |
| ✓ | BbhTrackSeven22CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT942 |
| ✓ | BbhTrackSeven8CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT594 |
| ✓ | BbhTrackSeven9CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT896 |
Reasoning - BBH tracking shuffled objects (7) (set 1)
30 passed |
100.0% |
IN34 796 OUT23 138 | |
| ✓ | BbhTrackSeven23CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT606 |
| ✓ | BbhTrackSeven24CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT568 |
| ✓ | BbhTrackSeven25CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT893 |
| ✓ | BbhTrackSeven26CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT663 |
| ✓ | BbhTrackSeven27CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT866 |
| ✓ | BbhTrackSeven28CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT899 |
| ✓ | BbhTrackSeven29CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT968 |
| ✓ | BbhTrackSeven30CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT925 |
| ✓ | BbhTrackSeven31CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT987 |
| ✓ | BbhTrackSeven32CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT714 |
| ✓ | BbhTrackSeven33CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT750 |
| ✓ | BbhTrackSeven34CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT970 |
| ✓ | BbhTrackSeven35CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT947 |
| ✓ | BbhTrackSeven36CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT728 |
| ✓ | BbhTrackSeven37CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT588 |
| ✓ | BbhTrackSeven38CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT775 |
| ✓ | BbhTrackSeven39CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT514 |
| ✓ | BbhTrackSeven40CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT830 |
| ✓ | BbhTrackSeven41CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT817 |
| ✓ | BbhTrackSeven42CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT825 |
| ✓ | BbhTrackSeven43CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT590 |
| ✓ | BbhTrackSeven44CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT642 |
| ✓ | BbhTrackSeven45CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT924 |
| ✓ | BbhTrackSeven46CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT743 |
| ✓ | BbhTrackSeven47CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT930 |
| ✓ | BbhTrackSeven48CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT849 |
| ✓ | BbhTrackSeven49CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT601 |
| ✓ | BbhTrackSeven50CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT608 |
| ✓ | BbhTrackSeven51CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 175 OUT594 |
| ✓ | BbhTrackSeven52CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT824 |
Reasoning - BBH tracking shuffled objects (7) (set 2)
32 passed |
100.0% |
IN37 146 OUT25 616 | |
| ✓ | BbhTrackingSeven53COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT588 |
| ✓ | BbhTrackingSeven54COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 178 OUT1 041 |
| ✓ | BbhTrackingSeven55COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT775 |
| ✓ | BbhTrackingSeven56COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT544 |
| ✓ | BbhTrackingSeven57COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT829 |
| ✓ | BbhTrackingSeven58COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT734 |
| ✓ | BbhTrackingSeven59COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT986 |
| ✓ | BbhTrackingSeven60COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT816 |
| ✓ | BbhTrackingSeven61COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT905 |
| ✓ | BbhTrackingSeven62COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT679 |
| ✓ | BbhTrackingSeven63COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT592 |
| ✓ | BbhTrackingSeven64COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT736 |
| ✓ | BbhTrackingSeven65COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT635 |
| ✓ | BbhTrackingSeven66COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT564 |
| ✓ | BbhTrackingSeven67COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT596 |
| ✓ | BbhTrackingSeven68COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 007 |
| ✓ | BbhTrackingSeven69COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT976 |
| ✓ | BbhTrackingSeven70COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT846 |
| ✓ | BbhTrackingSeven71COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT983 |
| ✓ | BbhTrackingSeven72COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT887 |
| ✓ | BbhTrackingSeven73COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT755 |
| ✓ | BbhTrackingSeven74COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 004 |
| ✓ | BbhTrackingSeven75COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT735 |
| ✓ | BbhTrackingSeven76COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 281 |
| ✓ | BbhTrackingSeven77COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT609 |
| ✓ | BbhTrackingSeven78COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT854 |
| ✓ | BbhTrackingSeven79COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT608 |
| ✓ | BbhTrackingSeven80COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT750 |
| ✓ | BbhTrackingSeven81COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT622 |
| ✓ | BbhTrackingSeven82COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT890 |
| ✓ | BbhTrackingSeven83COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT859 |
| ✓ | BbhTrackingSeven84COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT930 |
Reasoning - BBH tracking shuffled objects (7) (set 3)
32 passed |
100.0% |
IN37 233 OUT26 165 | |
| ✓ | BbhTrackingSeven100COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT899 |
| ✓ | BbhTrackingSeven101COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 186 OUT1 052 |
| ✓ | BbhTrackingSeven102COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT740 |
| ✓ | BbhTrackingSeven103COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT1 004 |
| ✓ | BbhTrackingSeven104COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT747 |
| ✓ | BbhTrackingSeven105COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT530 |
| ✓ | BbhTrackingSeven106COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 006 |
| ✓ | BbhTrackingSeven107COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT579 |
| ✓ | BbhTrackingSeven108COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT863 |
| ✓ | BbhTrackingSeven109COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT897 |
| ✓ | BbhTrackingSeven110COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT778 |
| ✓ | BbhTrackingSeven111COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT729 |
| ✓ | BbhTrackingSeven112COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT760 |
| ✓ | BbhTrackingSeven113COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT996 |
| ✓ | BbhTrackingSeven114COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT869 |
| ✓ | BbhTrackingSeven115COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT737 |
| ✓ | BbhTrackingSeven116COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT751 |
| ✓ | BbhTrackingSeven85COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT649 |
| ✓ | BbhTrackingSeven86COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT784 |
| ✓ | BbhTrackingSeven87COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT600 |
| ✓ | BbhTrackingSeven88COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT919 |
| ✓ | BbhTrackingSeven89COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT598 |
| ✓ | BbhTrackingSeven90COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 025 |
| ✓ | BbhTrackingSeven91COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT991 |
| ✓ | BbhTrackingSeven92COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT862 |
| ✓ | BbhTrackingSeven93COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT933 |
| ✓ | BbhTrackingSeven94COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT950 |
| ✓ | BbhTrackingSeven95COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT749 |
| ✓ | BbhTrackingSeven96COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT745 |
| ✓ | BbhTrackingSeven97COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT973 |
| ✓ | BbhTrackingSeven98COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT654 |
| ✓ | BbhTrackingSeven99COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT796 |
Reasoning - BBH tracking shuffled objects (7) (set 4)
32 passed |
100.0% |
IN37 229 OUT25 654 | |
| ✓ | BbhTrackingSeven117COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 023 |
| ✓ | BbhTrackingSeven118COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT638 |
| ✓ | BbhTrackingSeven119COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT857 |
| ✓ | BbhTrackingSeven120COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT892 |
| ✓ | BbhTrackingSeven121COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT993 |
| ✓ | BbhTrackingSeven122COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT606 |
| ✓ | BbhTrackingSeven123COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT594 |
| ✓ | BbhTrackingSeven124COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT817 |
| ✓ | BbhTrackingSeven125COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 005 |
| ✓ | BbhTrackingSeven126COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT638 |
| ✓ | BbhTrackingSeven127COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT622 |
| ✓ | BbhTrackingSeven128COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT696 |
| ✓ | BbhTrackingSeven129COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT863 |
| ✓ | BbhTrackingSeven130COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT891 |
| ✓ | BbhTrackingSeven131COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT905 |
| ✓ | BbhTrackingSeven132COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT664 |
| ✓ | BbhTrackingSeven133COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT741 |
| ✓ | BbhTrackingSeven134COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT507 |
| ✓ | BbhTrackingSeven135COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT742 |
| ✓ | BbhTrackingSeven136COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT592 |
| ✓ | BbhTrackingSeven137COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT801 |
| ✓ | BbhTrackingSeven138COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT778 |
| ✓ | BbhTrackingSeven139COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 184 OUT1 029 |
| ✓ | BbhTrackingSeven140COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT1 073 |
| ✓ | BbhTrackingSeven141COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 045 |
| ✓ | BbhTrackingSeven142COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT1 040 |
| ✓ | BbhTrackingSeven143COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT933 |
| ✓ | BbhTrackingSeven144COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT909 |
| ✓ | BbhTrackingSeven145COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT708 |
| ✓ | BbhTrackingSeven146COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT747 |
| ✓ | BbhTrackingSeven147COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT702 |
| ✓ | BbhTrackingSeven148COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT603 |
Reasoning - BBH tracking shuffled objects (7) (set 5)
31 passed |
100.0% |
IN36 047 OUT24 981 | |
| ✓ | BbhTrackingSeven149COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT888 |
| ✓ | BbhTrackingSeven150COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT892 |
| ✓ | BbhTrackingSeven151COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT672 |
| ✓ | BbhTrackingSeven152COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT846 |
| ✓ | BbhTrackingSeven153COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT897 |
| ✓ | BbhTrackingSeven154COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT1 003 |
| ✓ | BbhTrackingSeven155COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT838 |
| ✓ | BbhTrackingSeven156COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT769 |
| ✓ | BbhTrackingSeven157COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT894 |
| ✓ | BbhTrackingSeven158COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 117 |
| ✓ | BbhTrackingSeven159COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT842 |
| ✓ | BbhTrackingSeven160COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT748 |
| ✓ | BbhTrackingSeven161COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT699 |
| ✓ | BbhTrackingSeven162COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT920 |
| ✓ | BbhTrackingSeven163COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT693 |
| ✓ | BbhTrackingSeven164COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT760 |
| ✓ | BbhTrackingSeven165COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT744 |
| ✓ | BbhTrackingSeven166COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT593 |
| ✓ | BbhTrackingSeven167COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT910 |
| ✓ | BbhTrackingSeven168COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT770 |
| ✓ | BbhTrackingSeven169COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT886 |
| ✓ | BbhTrackingSeven170COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT848 |
| ✓ | BbhTrackingSeven171COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT916 |
| ✓ | BbhTrackingSeven172COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT948 |
| ✓ | BbhTrackingSeven173COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT537 |
| ✓ | BbhTrackingSeven174COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT561 |
| ✓ | BbhTrackingSeven175COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT588 |
| ✓ | BbhTrackingSeven176COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT918 |
| ✓ | BbhTrackingSeven177COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT818 |
| ✓ | BbhTrackingSeven178COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT551 |
| ✓ | BbhTrackingSeven179COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT915 |
Reasoning - BBH tracking shuffled objects (7) (set 6)
31 passed |
100.0% |
IN36 055 OUT24 863 | |
| ✓ | BbhTrackingSeven180COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT756 |
| ✓ | BbhTrackingSeven181COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT795 |
| ✓ | BbhTrackingSeven182COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 044 |
| ✓ | BbhTrackingSeven183COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT895 |
| ✓ | BbhTrackingSeven184COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT746 |
| ✓ | BbhTrackingSeven185COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT893 |
| ✓ | BbhTrackingSeven186COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT609 |
| ✓ | BbhTrackingSeven187COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT875 |
| ✓ | BbhTrackingSeven188COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT905 |
| ✓ | BbhTrackingSeven189COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT816 |
| ✓ | BbhTrackingSeven190COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT948 |
| ✓ | BbhTrackingSeven191COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT593 |
| ✓ | BbhTrackingSeven192COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT767 |
| ✓ | BbhTrackingSeven193COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT599 |
| ✓ | BbhTrackingSeven194COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT777 |
| ✓ | BbhTrackingSeven195COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT580 |
| ✓ | BbhTrackingSeven196COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT732 |
| ✓ | BbhTrackingSeven197COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT923 |
| ✓ | BbhTrackingSeven198COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT559 |
| ✓ | BbhTrackingSeven199COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT911 |
| ✓ | BbhTrackingSeven200COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT886 |
| ✓ | BbhTrackingSeven201COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 030 |
| ✓ | BbhTrackingSeven202COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT600 |
| ✓ | BbhTrackingSeven203COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT979 |
| ✓ | BbhTrackingSeven204COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT604 |
| ✓ | BbhTrackingSeven205COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT786 |
| ✓ | BbhTrackingSeven206COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 089 |
| ✓ | BbhTrackingSeven207COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT631 |
| ✓ | BbhTrackingSeven208COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT842 |
| ✓ | BbhTrackingSeven209COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT1 021 |
| ✓ | BbhTrackingSeven210COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT672 |
Reasoning - BBH tracking shuffled objects (7) (set 7)
39 passed |
100.0% |
IN45 365 OUT32 368 | |
| ✓ | BbhTrackingSeven211CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT820 |
| ✓ | BbhTrackingSeven212CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT961 |
| ✓ | BbhTrackingSeven213CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT730 |
| ✓ | BbhTrackingSeven214CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT700 |
| ✓ | BbhTrackingSeven215CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT595 |
| ✓ | BbhTrackingSeven216CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT730 |
| ✓ | BbhTrackingSeven217CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT888 |
| ✓ | BbhTrackingSeven218CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT756 |
| ✓ | BbhTrackingSeven219CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT902 |
| ✓ | BbhTrackingSeven220CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT831 |
| ✓ | BbhTrackingSeven221CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT809 |
| ✓ | BbhTrackingSeven222CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT831 |
| ✓ | BbhTrackingSeven223CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT848 |
| ✓ | BbhTrackingSeven224CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT846 |
| ✓ | BbhTrackingSeven225CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT967 |
| ✓ | BbhTrackingSeven226CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT803 |
| ✓ | BbhTrackingSeven227CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 047 |
| ✓ | BbhTrackingSeven228CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT861 |
| ✓ | BbhTrackingSeven229CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT923 |
| ✓ | BbhTrackingSeven230CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT739 |
| ✓ | BbhTrackingSeven231CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT932 |
| ✓ | BbhTrackingSeven232CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT633 |
| ✓ | BbhTrackingSeven233CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT747 |
| ✓ | BbhTrackingSeven234CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT972 |
| ✓ | BbhTrackingSeven235CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT696 |
| ✓ | BbhTrackingSeven236CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT679 |
| ✓ | BbhTrackingSeven237CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT740 |
| ✓ | BbhTrackingSeven238CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT854 |
| ✓ | BbhTrackingSeven239CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT778 |
| ✓ | BbhTrackingSeven240CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT794 |
| ✓ | BbhTrackingSeven241CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT1 017 |
| ✓ | BbhTrackingSeven242CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT827 |
| ✓ | BbhTrackingSeven243CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT715 |
| ✓ | BbhTrackingSeven244CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT831 |
| ✓ | BbhTrackingSeven245CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT998 |
| ✓ | BbhTrackingSeven246CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT907 |
| ✓ | BbhTrackingSeven247CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 039 |
| ✓ | BbhTrackingSeven248CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT890 |
| ✓ | BbhTrackingSeven249CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT732 |
Reasoning - BBH web of lies (set 1)
35 passed |
100.0% |
IN34 992 OUT9 345 | |
| ✓ | BbhWebOfLies10CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT174 |
| ✓ | BbhWebOfLies11CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT266 |
| ✓ | BbhWebOfLies12CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT243 |
| ✓ | BbhWebOfLies13CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT226 |
| ✓ | BbhWebOfLies14CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT288 |
| ✓ | BbhWebOfLies15CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT365 |
| ✓ | BbhWebOfLies16CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT203 |
| ✓ | BbhWebOfLies17CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT218 |
| ✓ | BbhWebOfLies18CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT294 |
| ✓ | BbhWebOfLies19CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT216 |
| ✓ | BbhWebOfLies20CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT254 |
| ✓ | BbhWebOfLies21CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT242 |
| ✓ | BbhWebOfLies22CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT212 |
| ✓ | BbhWebOfLies23CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT290 |
| ✓ | BbhWebOfLies24CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT308 |
| ✓ | BbhWebOfLies25CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT280 |
| ✓ | BbhWebOfLies26CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT269 |
| ✓ | BbhWebOfLies27CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT214 |
| ✓ | BbhWebOfLies28CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT264 |
| ✓ | BbhWebOfLies29CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT277 |
| ✓ | BbhWebOfLies30CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT263 |
| ✓ | BbhWebOfLies31CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT313 |
| ✓ | BbhWebOfLies32CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT285 |
| ✓ | BbhWebOfLies33CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT284 |
| ✓ | BbhWebOfLies34CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT259 |
| ✓ | BbhWebOfLies35CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT271 |
| ✓ | BbhWebOfLies36CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT363 |
| ✓ | BbhWebOfLies37CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT238 |
| ✓ | BbhWebOfLies38CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT341 |
| ✓ | BbhWebOfLies39CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT327 |
| ✓ | BbhWebOfLies40CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT290 |
| ✓ | BbhWebOfLies41CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT224 |
| ✓ | BbhWebOfLies42CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT305 |
| ✓ | BbhWebOfLies8CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT232 |
| ✓ | BbhWebOfLies9CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT247 |
Reasoning - BBH web of lies (set 2)
32 passed |
100.0% |
IN31 992 OUT8 887 | |
| ✓ | BbhWebOfLies43CRtruth-teller/liar chain — Does Audrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT281 |
| ✓ | BbhWebOfLies44CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT276 |
| ✓ | BbhWebOfLies45CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT251 |
| ✓ | BbhWebOfLies46CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT274 |
| ✓ | BbhWebOfLies47CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT310 |
| ✓ | BbhWebOfLies48CRtruth-teller/liar chain — Does Maybelle tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT301 |
| ✓ | BbhWebOfLies49CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT236 |
| ✓ | BbhWebOfLies50CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT219 |
| ✓ | BbhWebOfLies51CRtruth-teller/liar chain — Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT240 |
| ✓ | BbhWebOfLies52CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT382 |
| ✓ | BbhWebOfLies53CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT255 |
| ✓ | BbhWebOfLies54CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT268 |
| ✓ | BbhWebOfLies55CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT278 |
| ✓ | BbhWebOfLies56CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT290 |
| ✓ | BbhWebOfLies57CRtruth-teller/liar chain — Does Shalonda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT257 |
| ✓ | BbhWebOfLies58CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT212 |
| ✓ | BbhWebOfLies59CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT272 |
| ✓ | BbhWebOfLies60CRtruth-teller/liar chain — Does Sima tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT262 |
| ✓ | BbhWebOfLies61CRtruth-teller/liar chain — Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT360 |
| ✓ | BbhWebOfLies62CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT307 |
| ✓ | BbhWebOfLies63CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT241 |
| ✓ | BbhWebOfLies64CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT287 |
| ✓ | BbhWebOfLies65CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT200 |
| ✓ | BbhWebOfLies66CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT245 |
| ✓ | BbhWebOfLies67CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT293 |
| ✓ | BbhWebOfLies68CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT351 |
| ✓ | BbhWebOfLies69CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT322 |
| ✓ | BbhWebOfLies70CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT286 |
| ✓ | BbhWebOfLies71CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT281 |
| ✓ | BbhWebOfLies72CRtruth-teller/liar chain — Does Vernell tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT286 |
| ✓ | BbhWebOfLies73CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT250 |
| ✓ | BbhWebOfLies74CRtruth-teller/liar chain — Does Jaymie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT314 |
Reasoning - BBH web of lies (set 3)
32 passed |
100.0% |
IN31 996 OUT8 295 | |
| ✓ | BbhWebOfLies100CRtruth-teller/liar chain — Does Fidel tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT314 |
| ✓ | BbhWebOfLies101CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT263 |
| ✓ | BbhWebOfLies102CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT235 |
| ✓ | BbhWebOfLies103CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT283 |
| ✓ | BbhWebOfLies104CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT234 |
| ✓ | BbhWebOfLies105CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT232 |
| ✓ | BbhWebOfLies106CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT279 |
| ✓ | BbhWebOfLies75CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT324 |
| ✓ | BbhWebOfLies76CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT223 |
| ✓ | BbhWebOfLies77CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT213 |
| ✓ | BbhWebOfLies78CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT318 |
| ✓ | BbhWebOfLies79CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT279 |
| ✓ | BbhWebOfLies80CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT315 |
| ✓ | BbhWebOfLies81CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT191 |
| ✓ | BbhWebOfLies82CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT289 |
| ✓ | BbhWebOfLies83CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT208 |
| ✓ | BbhWebOfLies84CRtruth-teller/liar chain — Does Michael tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT228 |
| ✓ | BbhWebOfLies85CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT262 |
| ✓ | BbhWebOfLies86CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT299 |
| ✓ | BbhWebOfLies87CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT214 |
| ✓ | BbhWebOfLies88CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT217 |
| ✓ | BbhWebOfLies89CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT248 |
| ✓ | BbhWebOfLies90CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT215 |
| ✓ | BbhWebOfLies91CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT316 |
| ✓ | BbhWebOfLies92CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT209 |
| ✓ | BbhWebOfLies93CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT248 |
| ✓ | BbhWebOfLies94CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT273 |
| ✓ | BbhWebOfLies95CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT253 |
| ✓ | BbhWebOfLies96CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT272 |
| ✓ | BbhWebOfLies97CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT348 |
| ✓ | BbhWebOfLies98CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT253 |
| ✓ | BbhWebOfLies99CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT240 |
Reasoning - BBH web of lies (set 4)
32 passed |
100.0% |
IN32 016 OUT8 544 | |
| ✓ | BbhWebOfLies107CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT312 |
| ✓ | BbhWebOfLies108CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT281 |
| ✓ | BbhWebOfLies109CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT223 |
| ✓ | BbhWebOfLies110CRtruth-teller/liar chain — Does Kristian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT200 |
| ✓ | BbhWebOfLies111CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT249 |
| ✓ | BbhWebOfLies112CRtruth-teller/liar chain — Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT262 |
| ✓ | BbhWebOfLies113CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT249 |
| ✓ | BbhWebOfLies114CRtruth-teller/liar chain — Does Millie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT220 |
| ✓ | BbhWebOfLies115CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT229 |
| ✓ | BbhWebOfLies116CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT367 |
| ✓ | BbhWebOfLies117CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT270 |
| ✓ | BbhWebOfLies118CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT357 |
| ✓ | BbhWebOfLies119CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT236 |
| ✓ | BbhWebOfLies120CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT361 |
| ✓ | BbhWebOfLies121CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT220 |
| ✓ | BbhWebOfLies122CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT292 |
| ✓ | BbhWebOfLies123CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT245 |
| ✓ | BbhWebOfLies124CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT276 |
| ✓ | BbhWebOfLies125CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT263 |
| ✓ | BbhWebOfLies126CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT218 |
| ✓ | BbhWebOfLies127CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT305 |
| ✓ | BbhWebOfLies128CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT319 |
| ✓ | BbhWebOfLies129CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT249 |
| ✓ | BbhWebOfLies130CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT295 |
| ✓ | BbhWebOfLies131CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT288 |
| ✓ | BbhWebOfLies132CRtruth-teller/liar chain — Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT237 |
| ✓ | BbhWebOfLies133CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT329 |
| ✓ | BbhWebOfLies134CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT243 |
| ✓ | BbhWebOfLies135CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT286 |
| ✓ | BbhWebOfLies136CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT220 |
| ✓ | BbhWebOfLies137CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT210 |
| ✓ | BbhWebOfLies138CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT233 |
Reasoning - BBH web of lies (set 5)
32 passed |
100.0% |
IN32 010 OUT8 596 | |
| ✓ | BbhWebOfLies139CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT292 |
| ✓ | BbhWebOfLies140CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT289 |
| ✓ | BbhWebOfLies141CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT352 |
| ✓ | BbhWebOfLies142CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT354 |
| ✓ | BbhWebOfLies143CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT196 |
| ✓ | BbhWebOfLies144CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT269 |
| ✓ | BbhWebOfLies145CRtruth-teller/liar chain — Does Fidel tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT297 |
| ✓ | BbhWebOfLies146CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT221 |
| ✓ | BbhWebOfLies147CRtruth-teller/liar chain — Does Millie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT252 |
| ✓ | BbhWebOfLies148CRtruth-teller/liar chain — Does Audrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT283 |
| ✓ | BbhWebOfLies149CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT272 |
| ✓ | BbhWebOfLies150CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT252 |
| ✓ | BbhWebOfLies151CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT281 |
| ✓ | BbhWebOfLies152CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT272 |
| ✓ | BbhWebOfLies153CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT247 |
| ✓ | BbhWebOfLies154CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT216 |
| ✓ | BbhWebOfLies155CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT233 |
| ✓ | BbhWebOfLies156CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT272 |
| ✓ | BbhWebOfLies157CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT276 |
| ✓ | BbhWebOfLies158CRtruth-teller/liar chain — Does Shalonda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT341 |
| ✓ | BbhWebOfLies159CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT296 |
| ✓ | BbhWebOfLies160CRtruth-teller/liar chain — Does Ryan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT237 |
| ✓ | BbhWebOfLies161CRtruth-teller/liar chain — Does Willian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT220 |
| ✓ | BbhWebOfLies162CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT295 |
| ✓ | BbhWebOfLies163CRtruth-teller/liar chain — Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT263 |
| ✓ | BbhWebOfLies164CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT238 |
| ✓ | BbhWebOfLies165CRtruth-teller/liar chain — Does Raymond tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT228 |
| ✓ | BbhWebOfLies166CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT240 |
| ✓ | BbhWebOfLies167CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT262 |
| ✓ | BbhWebOfLies168CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT275 |
| ✓ | BbhWebOfLies169CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT269 |
| ✓ | BbhWebOfLies170CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT306 |
Reasoning - BBH web of lies (set 6)
30 passed |
100.0% |
IN29 986 OUT7 907 | |
| ✓ | BbhWebOfLies171CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT281 |
| ✓ | BbhWebOfLies172CRtruth-teller/liar chain — Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT301 |
| ✓ | BbhWebOfLies173CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT278 |
| ✓ | BbhWebOfLies174CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT200 |
| ✓ | BbhWebOfLies175CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT316 |
| ✓ | BbhWebOfLies176CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT279 |
| ✓ | BbhWebOfLies177CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT297 |
| ✓ | BbhWebOfLies178CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT275 |
| ✓ | BbhWebOfLies179CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT239 |
| ✓ | BbhWebOfLies180CRtruth-teller/liar chain — Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT294 |
| ✓ | BbhWebOfLies181CRtruth-teller/liar chain — Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT231 |
| ✓ | BbhWebOfLies182CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT313 |
| ✓ | BbhWebOfLies183CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT291 |
| ✓ | BbhWebOfLies184CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT289 |
| ✓ | BbhWebOfLies185CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT309 |
| ✓ | BbhWebOfLies186CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT195 |
| ✓ | BbhWebOfLies187CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT257 |
| ✓ | BbhWebOfLies188CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT288 |
| ✓ | BbhWebOfLies189CRtruth-teller/liar chain — Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT224 |
| ✓ | BbhWebOfLies190CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT212 |
| ✓ | BbhWebOfLies191CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT308 |
| ✓ | BbhWebOfLies192CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT218 |
| ✓ | BbhWebOfLies193CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT235 |
| ✓ | BbhWebOfLies194CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT292 |
| ✓ | BbhWebOfLies195CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT259 |
| ✓ | BbhWebOfLies196CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT206 |
| ✓ | BbhWebOfLies197CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT198 |
| ✓ | BbhWebOfLies198CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT285 |
| ✓ | BbhWebOfLies199CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT313 |
| ✓ | BbhWebOfLies200CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT224 |
Reasoning - BBH web of lies (set 7)
32 passed |
100.0% |
IN32 002 OUT8 411 | |
| ✓ | BbhWebOfLies201CSJamey tells the truth. Michael says Jamey lies. Sherrie says Michael lies. Bernita says Sherrie lies. Amberly says Bernita tells the truth. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT267 |
| ✓ | BbhWebOfLies202CSShaunda tells the truth. Jim says Shaunda lies. Alexis says Jim lies. Phoebe says Alexis tells the truth. Sherrie says Phoebe tells the truth. Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT245 |
| ✓ | BbhWebOfLies203CSMillicent lies. Ka says Millicent lies. Sima says Ka tells the truth. Dallas says Sima tells the truth. Jamey says Dallas lies. Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT264 |
| ✓ | BbhWebOfLies204CSChristie tells the truth. Fidel says Christie lies. Jerry says Fidel tells the truth. Maybelle says Jerry lies. Delfina says Maybelle tells the truth. Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT248 |
| ✓ | BbhWebOfLies205CSSima lies. Maybelle says Sima tells the truth. Audrie says Maybelle lies. Leda says Audrie tells the truth. Shaunda says Leda lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT289 |
| ✓ | BbhWebOfLies206CSVernell lies. Conception says Vernell lies. Kandi says Conception lies. Michael says Kandi tells the truth. Osvaldo says Michael tells the truth. Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT211 |
| ✓ | BbhWebOfLies207CSShaunda tells the truth. Delbert says Shaunda tells the truth. Bernita says Delbert lies. Yoland says Bernita tells the truth. Christie says Yoland tells the truth. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT288 |
| ✓ | BbhWebOfLies208CSInga tells the truth. Shalonda says Inga tells the truth. Phoebe says Shalonda tells the truth. Crista says Phoebe lies. Alejandro says Crista lies. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT273 |
| ✓ | BbhWebOfLies209CSFidel lies. Millicent says Fidel tells the truth. Crista says Millicent lies. Leda says Crista lies. Tamika says Leda tells the truth. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT286 |
| ✓ | BbhWebOfLies210CSAndree tells the truth. Christie says Andree tells the truth. Fidel says Christie lies. Antwan says Fidel tells the truth. Alejandro says Antwan lies. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT260 |
| ✓ | BbhWebOfLies211CSFletcher tells the truth. Willian says Fletcher tells the truth. Jamey says Willian lies. Leda says Jamey lies. Jaymie says Leda tells the truth. Does Jaymie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT278 |
| ✓ | BbhWebOfLies212CSTamika lies. Jerry says Tamika tells the truth. Delbert says Jerry tells the truth. Leda says Delbert tells the truth. Willian says Leda lies. Does Willian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT210 |
| ✓ | BbhWebOfLies213CSTeressa lies. Sal says Teressa tells the truth. Delbert says Sal tells the truth. Tamika says Delbert tells the truth. Gwenn says Tamika tells the truth. Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT214 |
| ✓ | BbhWebOfLies214CSJim lies. Delfina says Jim tells the truth. Shaunda says Delfina lies. Fidel says Shaunda lies. Sal says Fidel tells the truth. Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT273 |
| ✓ | BbhWebOfLies215CSMillie lies. Michael says Millie tells the truth. Vina says Michael lies. Conception says Vina tells the truth. Sima says Conception tells the truth. Does Sima tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT269 |
| ✓ | BbhWebOfLies216CSKandi lies. Millie says Kandi tells the truth. Lorine says Millie tells the truth. Michaela says Lorine tells the truth. Amberly says Michaela lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT262 |
| ✓ | BbhWebOfLies217CSMichael lies. Jerry says Michael tells the truth. Delbert says Jerry tells the truth. Sima says Delbert lies. Alexis says Sima lies. Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT255 |
| ✓ | BbhWebOfLies218CSSal lies. Kristian says Sal lies. Helene says Kristian lies. Inga says Helene tells the truth. Michael says Inga tells the truth. Does Michael tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT253 |
| ✓ | BbhWebOfLies219CSMaybelle lies. Crista says Maybelle tells the truth. Bernita says Crista lies. Kandi says Bernita tells the truth. Delbert says Kandi tells the truth. Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT355 |
| ✓ | BbhWebOfLies220CSKa lies. Rashida says Ka tells the truth. Sherrie says Rashida tells the truth. Tamika says Sherrie tells the truth. Elanor says Tamika tells the truth. Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT226 |
| ✓ | BbhWebOfLies221CSAntwan tells the truth. Audrie says Antwan tells the truth. Inga says Audrie lies. Shenna says Inga tells the truth. Christie says Shenna lies. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT255 |
| ✓ | BbhWebOfLies222CSTeressa lies. Michael says Teressa tells the truth. Fidel says Michael tells the truth. Leda says Fidel lies. Lorine says Leda lies. Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT275 |
| ✓ | BbhWebOfLies223CSAntwan tells the truth. Elanor says Antwan tells the truth. Bernita says Elanor tells the truth. Osvaldo says Bernita lies. Christie says Osvaldo tells the truth. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT226 |
| ✓ | BbhWebOfLies224CSInga tells the truth. Elanor says Inga lies. Millicent says Elanor tells the truth. Willian says Millicent tells the truth. Alejandro says Willian tells the truth. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT231 |
| ✓ | BbhWebOfLies225CSSherrie lies. Elanor says Sherrie tells the truth. Michael says Elanor tells the truth. Sima says Michael tells the truth. Shaunda says Sima lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT227 |
| ✓ | BbhWebOfLies226CSTamika lies. Conception says Tamika tells the truth. Kristian says Conception lies. Alejandro says Kristian tells the truth. Shaunda says Alejandro lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT218 |
| ✓ | BbhWebOfLies227CSHelene lies. Delfina says Helene lies. Christie says Delfina tells the truth. Jerry says Christie lies. Tamika says Jerry tells the truth. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT293 |
| ✓ | BbhWebOfLies228CSConception lies. Shenna says Conception tells the truth. Maybelle says Shenna tells the truth. Michael says Maybelle lies. Amberly says Michael lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT310 |
| ✓ | BbhWebOfLies229CSKristian tells the truth. Jamey says Kristian tells the truth. Millicent says Jamey lies. Fidel says Millicent lies. Tamika says Fidel lies. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT288 |
| ✓ | BbhWebOfLies230CSJamey lies. Sal says Jamey tells the truth. Teressa says Sal lies. Maybelle says Teressa tells the truth. Ka says Maybelle lies. Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT327 |
| ✓ | BbhWebOfLies231CSLorine lies. Amberly says Lorine lies. Fletcher says Amberly lies. Millie says Fletcher tells the truth. Antwan says Millie lies. Does Antwan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT301 |
| ✓ | BbhWebOfLies232CSAlexis tells the truth. Elanor says Alexis tells the truth. Sherrie says Elanor lies. Audrie says Sherrie tells the truth. Inga says Audrie tells the truth. Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT234 |
Reasoning - BBH web of lies (set 8)
31 passed1 failed |
96.9% |
IN31 729 OUT12 298 | |
| ✓ | BbhWebOfLies233CSFidel tells the truth. Teressa says Fidel tells the truth. Willian says Teressa tells the truth. Conception says Willian tells the truth. Bernita says Conception lies. Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT353 |
| ✓ | BbhWebOfLies234CSJerry tells the truth. Lorine says Jerry lies. Shenna says Lorine lies. Gwenn says Shenna lies. Shaunda says Gwenn lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT241 |
| ✓ | BbhWebOfLies235CSInga lies. Audrie says Inga lies. Rashida says Audrie lies. Willian says Rashida tells the truth. Amberly says Willian lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT290 |
| ✓ | BbhWebOfLies236CSKa tells the truth. Inga says Ka lies. Bernita says Inga lies. Osvaldo says Bernita tells the truth. Michaela says Osvaldo lies. Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT264 |
| ✓ | BbhWebOfLies237CSJaymie lies. Jamey says Jaymie tells the truth. Sal says Jamey lies. Amberly says Sal lies. Jerry says Amberly tells the truth. Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT312 |
| ✓ | BbhWebOfLies238CSMillie lies. Andree says Millie lies. Michaela says Andree lies. Leda says Michaela tells the truth. Shenna says Leda lies. Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT293 |
| ✓ | BbhWebOfLies239CSShaunda tells the truth. Maybelle says Shaunda tells the truth. Lorine says Maybelle lies. Millie says Lorine tells the truth. Vina says Millie tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT243 |
| ✓ | BbhWebOfLies240CSFletcher lies. Tamika says Fletcher tells the truth. Ka says Tamika lies. Sima says Ka tells the truth. Ryan says Sima tells the truth. Does Ryan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT220 |
| ✓ | BbhWebOfLies241CSShaunda lies. Andree says Shaunda tells the truth. Jaymie says Andree lies. Inga says Jaymie lies. Millicent says Inga tells the truth. Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT332 |
| ✓ | BbhWebOfLies242CSRashida lies. Fletcher says Rashida lies. Antwan says Fletcher lies. Willian says Antwan tells the truth. Vina says Willian tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT340 |
| ✓ | BbhWebOfLies243CSJerry lies. Kandi says Jerry lies. Fletcher says Kandi lies. Ryan says Fletcher lies. Delfina says Ryan lies. Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT245 |
| ✓ | BbhWebOfLies244CSShaunda tells the truth. Phoebe says Shaunda tells the truth. Ryan says Phoebe tells the truth. Christie says Ryan tells the truth. Vina says Christie tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT233 |
| ✓ | BbhWebOfLies245CSAlexis tells the truth. Amberly says Alexis tells the truth. Andree says Amberly tells the truth. Vina says Andree tells the truth. Helene says Vina tells the truth. Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT207 |
| ✓ | BbhWebOfLies246CSGwenn lies. Maybelle says Gwenn tells the truth. Raymond says Maybelle tells the truth. Jamey says Raymond tells the truth. Andree says Jamey lies. Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT515 |
| ✓ | BbhWebOfLies247CSKandi tells the truth. Ryan says Kandi lies. Michaela says Ryan tells the truth. Sima says Michaela lies. Andree says Sima tells the truth. Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT239 |
| ✓ | BbhWebOfLies248CSFletcher tells the truth. Alexis says Fletcher lies. Jerry says Alexis lies. Elanor says Jerry tells the truth. Jim says Elanor lies. Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT265 |
| ✓ | BbhWebOfLies249CSSherrie tells the truth. Ryan says Sherrie lies. Bernita says Ryan tells the truth. Tamika says Bernita lies. Jerry says Tamika lies. Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT222 |
| × | BbhWordSorting10CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"arapaho bacteria bela bock burley")
Context: BbhWordSorting10CS: expected arapaho bacteria bela bock burley, got [bacteria arapaho bela bock burley] |
Failed | IN973 OUT405 |
| ✓ | BbhWordSorting11CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT887 |
| ✓ | BbhWordSorting12CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT847 |
| ✓ | BbhWordSorting13CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT438 |
| ✓ | BbhWordSorting14CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT314 |
| ✓ | BbhWordSorting15CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT797 |
| ✓ | BbhWordSorting16CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT1 043 |
| ✓ | BbhWordSorting17CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT546 |
| ✓ | BbhWordSorting18CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT872 |
| ✓ | BbhWordSorting19CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT147 |
| ✓ | BbhWordSorting20CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT118 |
| ✓ | BbhWordSorting21CSalphabetically sort 7 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT326 |
| ✓ | BbhWordSorting22CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT395 |
| ✓ | BbhWordSorting8CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT172 |
| ✓ | BbhWordSorting9CSalphabetically sort 6 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT177 |
Reasoning - BBH word sorting (set 1)
29 passed3 failed |
90.6% |
IN31 451 OUT17 756 | |
| ✓ | BbhWordSorting23CSalphabetically sort 7 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT376 |
| ✓ | BbhWordSorting24CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 236 |
| × | BbhWordSorting25CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"acclaim champ clothbound commodity conclusion delirious dyestuff exempt gadwall hayes hood hypothalamus jigsaw lozenge pipeline plentiful sarcastic seashell sensory teen")
Context: BbhWordSorting25CS: expected acclaim champ clothbound commodity conclusion delirious dyestuff exempt gadwall hayes hood hypothalamus jigsaw lozenge pipeline plentiful sarcastic seashell sensory teen, got [acclaim champ clothbound commodity conclusion delirious dyestuff exempt gadwall hayes hood hypothalamus jigsaw lozenge pipeline plentiful sarcastic sensory seashell teen] |
Failed | IN999 OUT1 038 |
| ✓ | BbhWordSorting26CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT468 |
| ✓ | BbhWordSorting27CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT155 |
| ✓ | BbhWordSorting28CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT788 |
| ✓ | BbhWordSorting29CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT550 |
| ✓ | BbhWordSorting30CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT653 |
| ✓ | BbhWordSorting31CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT807 |
| ✓ | BbhWordSorting32CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT116 |
| × | BbhWordSorting33CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"adonis birdseed citizen contaminant convair extensive fateful frighten judaica scrubby soothe southeastern stormy suppose trillion trundle")
Context: BbhWordSorting33CS: expected adonis birdseed citizen contaminant convair extensive fateful frighten judaica scrubby soothe southeastern stormy suppose trillion trundle, got [adonis birdseed citizen contaminant convair extensive fateful frighten judaica scrubby southeastern soothe stormy suppose trillion trundle] |
Failed | IN992 OUT1 357 |
| ✓ | BbhWordSorting34CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT159 |
| ✓ | BbhWordSorting35CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 152 |
| ✓ | BbhWordSorting36CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT493 |
| ✓ | BbhWordSorting37CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 274 |
| ✓ | BbhWordSorting38CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT95 |
| ✓ | BbhWordSorting39CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT98 |
| ✓ | BbhWordSorting40CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 113 |
| ✓ | BbhWordSorting41CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT475 |
| ✓ | BbhWordSorting42CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT121 |
| ✓ | BbhWordSorting43CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT315 |
| × | BbhWordSorting44CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"abstract borough brown cortex cosec delphinium diminutive fleabane foot guy hair highfalutin ipsilateral longish mobster richfield trapezoidal ugh wintertime")
Context: BbhWordSorting44CS: expected abstract borough brown cortex cosec delphinium diminutive fleabane foot guy hair highfalutin ipsilateral longish mobster richfield trapezoidal ugh wintertime, got [abstract borough brown cosec cortex delphinium diminutive fleabane foot guy hair highfalutin ipsilateral longish mobster richfield trapezoidal ugh wintertime] |
Failed | IN1 001 OUT900 |
| ✓ | BbhWordSorting45CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT467 |
| ✓ | BbhWordSorting46CSalphabetically sort 6 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT317 |
| ✓ | BbhWordSorting47CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT311 |
| ✓ | BbhWordSorting48CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT474 |
| ✓ | BbhWordSorting49CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT802 |
| ✓ | BbhWordSorting50CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT116 |
| ✓ | BbhWordSorting51CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT635 |
| ✓ | BbhWordSorting52CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT182 |
| ✓ | BbhWordSorting53CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT171 |
| ✓ | BbhWordSorting54CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT542 |
Reasoning - BBH word sorting (set 2)
30 passed2 failed |
93.8% |
IN31 667 OUT24 750 | |
| ✓ | BbhWordSorting55CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT946 |
| ✓ | BbhWordSorting56CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT400 |
| ✓ | BbhWordSorting57CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 012 OUT944 |
| ✓ | BbhWordSorting58CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT1 270 |
| ✓ | BbhWordSorting59CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 096 |
| ✓ | BbhWordSorting60CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT539 |
| ✓ | BbhWordSorting61CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT834 |
| ✓ | BbhWordSorting62CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT939 |
| ✓ | BbhWordSorting63CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT592 |
| ✓ | BbhWordSorting64CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT336 |
| ✓ | BbhWordSorting65CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT487 |
| ✓ | BbhWordSorting66CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 032 |
| ✓ | BbhWordSorting67CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 212 |
| ✓ | BbhWordSorting68CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT213 |
| ✓ | BbhWordSorting69CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 236 |
| ✓ | BbhWordSorting70CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT1 241 |
| ✓ | BbhWordSorting71CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT773 |
| ✓ | BbhWordSorting72CSalphabetically sort 8 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT375 |
| ✓ | BbhWordSorting73CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT646 |
| × | BbhWordSorting74CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"allot chauncey clergymen coachmen coddington companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats")
Context: BbhWordSorting74CS: expected allot chauncey clergymen coachmen coddington companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats, got [allot chauncey coachmen clergymen coddington companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats] |
Failed | IN1 003 OUT1 096 |
| ✓ | BbhWordSorting75CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT199 |
| ✓ | BbhWordSorting76CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT716 |
| ✓ | BbhWordSorting77CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT133 |
| ✓ | BbhWordSorting78CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT733 |
| ✓ | BbhWordSorting79CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT761 |
| ✓ | BbhWordSorting80CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT1 232 |
| ✓ | BbhWordSorting81CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT804 |
| ✓ | BbhWordSorting82CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT168 |
| ✓ | BbhWordSorting83CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT499 |
| ✓ | BbhWordSorting84CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT441 |
| ✓ | BbhWordSorting85CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT609 |
| × | BbhWordSorting86CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman")
Context: BbhWordSorting86CS: expected betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman, got [ANSEWR: betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman] |
Failed | IN1 002 OUT2 248 |
Reasoning - BBH word sorting (set 3)
28 passed2 failed |
93.3% |
IN29 651 OUT20 026 | |
| ✓ | BbhWordSorting100CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT182 |
| ✓ | BbhWordSorting101CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 009 |
| × | BbhWordSorting102CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"adsorption align anastasia anastomotic apache award bobbin burrow calumny epaulet execrable hostelry hun macedon omnipotent putty roughshod smooth spontaneity")
Context: BbhWordSorting102CS: expected adsorption align anastasia anastomotic apache award bobbin burrow calumny epaulet execrable hostelry hun macedon omnipotent putty roughshod smooth spontaneity, got [adsorption align apache anastasia anastomotic award bobbin burrow calumny epaulet execrable hostelry hun macedon omnipotent putty roughshod smooth spontaneity] |
Failed | IN1 006 OUT1 371 |
| ✓ | BbhWordSorting103CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT414 |
| ✓ | BbhWordSorting104CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT605 |
| ✓ | BbhWordSorting105CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 163 |
| ✓ | BbhWordSorting106CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT824 |
| ✓ | BbhWordSorting107CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT752 |
| ✓ | BbhWordSorting108CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT480 |
| × | BbhWordSorting109CSalphabetically sort 8 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"blutwurst buckaroo closeup intelligent laguerre thesaurus vertebral wily")
Context: BbhWordSorting109CS: expected blutwurst buckaroo closeup intelligent laguerre thesaurus vertebral wily, got [ANSEWR: blutwurst buckaroo closeup intelligent laguerre thesaurus vertebral wily] |
Failed | IN983 OUT423 |
| ✓ | BbhWordSorting110CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT351 |
| ✓ | BbhWordSorting111CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT619 |
| ✓ | BbhWordSorting112CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT660 |
| ✓ | BbhWordSorting113CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT918 |
| ✓ | BbhWordSorting114CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT827 |
| ✓ | BbhWordSorting115CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT934 |
| ✓ | BbhWordSorting116CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT997 |
| ✓ | BbhWordSorting87CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT630 |
| ✓ | BbhWordSorting88CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT123 |
| ✓ | BbhWordSorting89CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 115 |
| ✓ | BbhWordSorting90CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT919 |
| ✓ | BbhWordSorting91CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT750 |
| ✓ | BbhWordSorting92CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT144 |
| ✓ | BbhWordSorting93CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT963 |
| ✓ | BbhWordSorting94CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT506 |
| ✓ | BbhWordSorting95CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT594 |
| ✓ | BbhWordSorting96CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT825 |
| ✓ | BbhWordSorting97CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT712 |
| ✓ | BbhWordSorting98CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT94 |
| ✓ | BbhWordSorting99CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT122 |
Reasoning - BBH word sorting (set 4)
26 passed1 failed |
96.3% |
IN26 570 OUT16 643 | |
| ✓ | BbhWordSorting117CTSort the following words alphabetically: List: marshmallow doge alpenstock ocean accept angus drool jutish resistive chromium vociferous castigate prize octennial stonewort concision elizabethan1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 276 |
| ✓ | BbhWordSorting118CTSort the following words alphabetically: List: jocund flagellate bodyguard flotation commensal involve miff ineradicable postprocess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT789 |
| ✓ | BbhWordSorting119CTSort the following words alphabetically: List: gloriana milord rendezvous schnabel allegoric loge lollipop salle collate prolix mast euphony1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT586 |
| ✓ | BbhWordSorting120CTSort the following words alphabetically: List: pea yonkers darkle quadrant moldboard instant olsen hookup erudite lip1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT408 |
| ✓ | BbhWordSorting121CTSort the following words alphabetically: List: regret uptake satisfy behold fumarole motif forth alcohol escutcheon pease hackberry walkie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT400 |
| ✓ | BbhWordSorting122CTSort the following words alphabetically: List: charcuterie muddle diatom impenetrable footfall crucifix greenberg synchronous spoken1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT602 |
| ✓ | BbhWordSorting123CTSort the following words alphabetically: List: sibyl calligraph goat inverness form threadbare1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT374 |
| ✓ | BbhWordSorting124CTSort the following words alphabetically: List: cashew apprehension ensemble1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT140 |
| ✓ | BbhWordSorting125CTSort the following words alphabetically: List: seashore haughty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT104 |
| ✓ | BbhWordSorting126CTSort the following words alphabetically: List: postcondition protoplasmic musicology helical uptrend vasoconstriction diacritic beefsteak beware birthplace bicycle junctor state obstinate banshee sap1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 091 |
| ✓ | BbhWordSorting127CTSort the following words alphabetically: List: whisper eyesight titmice mirage fletch cry biennial creedal miltonic fraudulent j1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT554 |
| ✓ | BbhWordSorting128CTSort the following words alphabetically: List: multitudinous adopt wage worcestershire pacifist afghan glimmer friday1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT385 |
| ✓ | BbhWordSorting129CTSort the following words alphabetically: List: lethe bilk perturb tactual1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT115 |
| × | BbhWordSorting130CTSort the following words alphabetically: List: liggett sterno pintail dilate frolic piggish forth parameter slaughterhouse lacunae mcintosh bloodstain lazy unesco protector beachhead algonquin1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("algonquin beachhead bloodstain dilate forth frolic lacunae lazy liggett mcintosh parameter piggish pintail protector slaughterhouse sterno unesco")
Context: BbhWordSorting130CT: expected sorted list, got [algonquin beachhead bloodstain dilate forth frolic lacunae lazy liggett mcintosh parameter piggish pintail slaughterhouse sterno unesco] |
Failed | IN998 OUT994 |
| ✓ | BbhWordSorting131CTSort the following words alphabetically: List: erudite marriageable stahl coach inadvisable vicksburg virgo brownian eosine magnesium flax1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT635 |
| ✓ | BbhWordSorting132CTSort the following words alphabetically: List: clifford brindle siskin gloat florist triploidy willard sacramento1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT485 |
| ✓ | BbhWordSorting133CTSort the following words alphabetically: List: confrontation quantitative proserpine proofread daddy hirsute1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT499 |
| ✓ | BbhWordSorting134CTSort the following words alphabetically: List: allyn carbonaceous majorca investigatory supersede cetacean pathogenic johann tung paradigmatic pray1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT691 |
| ✓ | BbhWordSorting135CTSort the following words alphabetically: List: calyx emphases vii elgin perquisite implacable blackbird afro jura mayapple whit blame1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT607 |
| ✓ | BbhWordSorting136CTSort the following words alphabetically: List: cast palazzi inhibit arragon tabletop danbury abbe adposition emplace gavin income falsetto onerous1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT670 |
| ✓ | BbhWordSorting137CTSort the following words alphabetically: List: ulcerate ballroom sparse intrepid arcing prorogue dodge benefit coupon ladle buggy shagging bassoon hypothermia decide affirmative airframe junior nineveh schmitt1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 142 |
| ✓ | BbhWordSorting138CTSort the following words alphabetically: List: statuette whiteface lauderdale encumber atonic specify olivia adipic neutrino antique catch persona athlete sovereignty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 215 |
| ✓ | BbhWordSorting139CTSort the following words alphabetically: List: hyperboloidal borough1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT109 |
| ✓ | BbhWordSorting140CTSort the following words alphabetically: List: yves butt scribe seaworthy manumitted clapeyron lift blackguard reconnaissance agile woodcut lightfooted sawbelly meager geophysical wiseacre mathieu cognoscenti purposive flamboyant1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT1 101 |
| ✓ | BbhWordSorting141CTSort the following words alphabetically: List: duane harriet impressible whippet glamour cowbell vietnamese fest switchboard anaglyph texture1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT443 |
| ✓ | BbhWordSorting142CTSort the following words alphabetically: List: plural dose allstate stalin dyad multitudinous powderpuff1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT542 |
| ✓ | BbhWordSorting143CTSort the following words alphabetically: List: crossbill bushland capacitive meager ricochet battery enigma wysiwyg wallet lipton jane wacke contingent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT686 |
Reasoning - BBH word sorting (set 5)
26 passed1 failed |
96.3% |
IN26 547 OUT15 504 | |
| ✓ | BbhWordSorting144CTSort the following words alphabetically: List: corpulent diagnose code scalp bombproof fluorine blythe swipe honeybee damn pore maharaja cytolysis solicit1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT828 |
| ✓ | BbhWordSorting145CTSort the following words alphabetically: List: fugue chicanery mountain1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT132 |
| ✓ | BbhWordSorting146CTSort the following words alphabetically: List: sideboard procrustean seepage buxton olympic tty typescript marque callus unitary cameron contribute precise verify shelf extensible methanol1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT798 |
| ✓ | BbhWordSorting147CTSort the following words alphabetically: List: sociolinguistic nigeria odysseus demystify proton voltaire penny contaminate bighorn stirrup1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT555 |
| ✓ | BbhWordSorting148CTSort the following words alphabetically: List: greasy lapidary mark1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT134 |
| ✓ | BbhWordSorting149CTSort the following words alphabetically: List: choreograph retardant multifarious pro carbonic equestrian wishful wigwam countersink libya authenticate metal have corvallis nitric petition obfuscatory1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 129 |
| ✓ | BbhWordSorting150CTSort the following words alphabetically: List: oakland seaborg jacobi membrane trapezoidal allis marmot toggle anthology1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT789 |
| ✓ | BbhWordSorting151CTSort the following words alphabetically: List: monotonous slick provocation functor hideaway holeable assimilable lola calypso nebuchadnezzar dogwood pacifism bivariate bought1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 279 |
| ✓ | BbhWordSorting152CTSort the following words alphabetically: List: livre substitution ix frisian panoramic buckley1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT264 |
| ✓ | BbhWordSorting153CTSort the following words alphabetically: List: no spokane cartilaginous that'd science1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT276 |
| ✓ | BbhWordSorting154CTSort the following words alphabetically: List: splashy rectitude novelty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT120 |
| ✓ | BbhWordSorting155CTSort the following words alphabetically: List: sediment varsity towel fracture1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT132 |
| ✓ | BbhWordSorting156CTSort the following words alphabetically: List: confidential ursa fiction heterozygous faery overture horehound1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT310 |
| ✓ | BbhWordSorting157CTSort the following words alphabetically: List: mckinley pesticide school radiometer marque prig relish trianon inspiration artful cancelled castrate glaucous ear croon tioga rothschild excite endpoint citadel1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 127 |
| ✓ | BbhWordSorting158CTSort the following words alphabetically: List: iniquity yoghurt behind volcanism hornpipe mcconnell scorn zip toroidal mollie sandy wellwisher inmate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT619 |
| ✓ | BbhWordSorting159CTSort the following words alphabetically: List: extempore gotten1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT102 |
| ✓ | BbhWordSorting160CTSort the following words alphabetically: List: yang relinquish merry maladjust olefin canaan ranch batavia1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT440 |
| ✓ | BbhWordSorting161CTSort the following words alphabetically: List: yuh edt cheddar oblivion from poignant pang1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT292 |
| ✓ | BbhWordSorting162CTSort the following words alphabetically: List: zan natalie coplanar stevenson1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT127 |
| ✓ | BbhWordSorting163CTSort the following words alphabetically: List: bottle kenney chapati okinawa boon bedtime1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT317 |
| ✓ | BbhWordSorting164CTSort the following words alphabetically: List: contrariety hoop muong dimorphic crochet forthwith hurty titrate prodigy polytope guaranteeing atavism emanate iniquity grind more katie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 098 |
| ✓ | BbhWordSorting165CTSort the following words alphabetically: List: sloane molten equip hillman max cochran practise advisee kuwait chancery hoofprint cardigan evildoer retinue wuhan ape apostate acrobacy crowbait1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT1 019 |
| ✓ | BbhWordSorting166CTSort the following words alphabetically: List: catawba almagest transfusable milt photolysis limpid gnome archenemy sagging councilwomen decrement jungian1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT536 |
| ✓ | BbhWordSorting167CTSort the following words alphabetically: List: inscribe upraise perceptive tragedian peculate cunard posterior crude danubian1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT613 |
| × | BbhWordSorting168CTSort the following words alphabetically: List: madeira offprint trafficked immemorial subject micro clinic sprue marx va anaheim eaten1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("anaheim clinic eaten immemorial madeira marx micro offprint sprue subject trafficked va")
Context: BbhWordSorting168CT: expected sorted list, got [anaheim clinic eaten immemorial madiera marx micro offprint sprue subject trafficked va] |
Failed | IN982 OUT439 |
| ✓ | BbhWordSorting169CTSort the following words alphabetically: List: reedy bully multiple edt sell referring lexicographer tit flogging perceptive minor bosporus rancorous forfeit pizza pungent sedition cork1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT898 |
| ✓ | BbhWordSorting170CTSort the following words alphabetically: List: thirteenth tinfoil thimble snuff ernie pow celebrity abuilding indiscoverable chigger burgher synchronous yap medlar exorcism newsboy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 131 |
Reasoning - BBH word sorting (set 6)
22 passed5 failed |
81.5% |
IN26 694 OUT19 027 | |
| ✓ | BbhWordSorting171CTSort the following words alphabetically: List: saprophytic confident pam stowaway stupor foundling compton1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT500 |
| ✓ | BbhWordSorting172CTSort the following words alphabetically: List: epiphyte swift sail anchor barre dimension faunal integrate eleanor edgy singular patrician orthodoxy parasol pendant buckle concatenate masochist1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT977 |
| ✓ | BbhWordSorting173CTSort the following words alphabetically: List: butterscotch veterinarian kochab molybdate extoller amos farrell academia glance predatory beautiful ford fiducial metzler europium circuitous diatom monomer1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT933 |
| ✓ | BbhWordSorting174CTSort the following words alphabetically: List: mater rica sigmund dean eosine inebriate gibson vassar mulligatawny formula1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT456 |
| ✓ | BbhWordSorting175CTSort the following words alphabetically: List: shakespearian laudatory1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT120 |
| ✓ | BbhWordSorting176CTSort the following words alphabetically: List: fasciculate judicature presto1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT89 |
| ✓ | BbhWordSorting177CTSort the following words alphabetically: List: thereupon wireman crag scrawny electrolytic wife cutover dickson clytemnestra took diocletian lipton inhuman stalk workplace marginal1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 147 |
| ✓ | BbhWordSorting178CTSort the following words alphabetically: List: bruckner admiration technion bandgap heron postprocessor maidenhair fungus allusion gambit accessory proscenium cruise1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT976 |
| × | BbhWordSorting179CTSort the following words alphabetically: List: grammarian chivalry polkadot gastronomic ferocity brenda knutson endothelial phenotype irreducible bilinear rockaway derivate eaten cacao scurrilous1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("bilinear brenda cacao chivalry derivate eaten endothelial ferocity gastronomic grammarian irreducible knutson phenotype polkadot rockaway scurrilous")
Context: BbhWordSorting179CT: expected sorted list, got [bilinear brenda cacao chivalry derivate endothelial eaten ferocity gastronomic grammarian irreducible knutson phenotype polkadot rockaway scurrilous] |
Failed | IN998 OUT744 |
| ✓ | BbhWordSorting180CTSort the following words alphabetically: List: hyperboloid buckle liquidate alkali nagasaki breach falsetto parmesan mirth1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT541 |
| ✓ | BbhWordSorting181CTSort the following words alphabetically: List: hornblower dissipate amanita canticle annoy besiege straight notre propylene sepia california pasture encephalitis boggle crocodilian dexter snipe amatory dizzy psychiatric1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 083 |
| × | BbhWordSorting182CTSort the following words alphabetically: List: peaceable residue sulfanilamide kingsley eft caliber headboard stamp phillip upholster irishman lobby nary gastrointestinal fragile faulkner capricious ouzo phylum1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("caliber capricious eft faulkner fragile gastrointestinal headboard irishman kingsley lobby nary ouzo peaceable phillip phylum residue stamp sulfanilamide upholster")
Context: BbhWordSorting182CT: expected sorted list, got [capricious eft faulkner fragile gastrointestinal headboard irishman kingsley lobby nary ouzo peaceable phillip phylum residue stamp sulfanilamide upholster] |
Failed | IN1 001 OUT981 |
| ✓ | BbhWordSorting183CTSort the following words alphabetically: List: multiplet catsup hypothalamus scherzo bridesmaid average therewith invocable coddle charm inequity region abbas trumpery memorandum shutout integral dogfish phloem inconvertible1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 005 OUT973 |
| ✓ | BbhWordSorting184CTSort the following words alphabetically: List: windowsill appoint biharmonic moustache baneberry wiry dyne pirate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT454 |
| ✓ | BbhWordSorting185CTSort the following words alphabetically: List: zoroaster skimpy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT104 |
| × | BbhWordSorting186CTSort the following words alphabetically: List: almost delphic semitic unbeknownst astute deadlock libertarian antic levitate execution sanction scathe storehouse dandelion affable sweeney fortunate leverage horntail apache1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("affable almost antic apache astute dandelion deadlock delphic execution fortunate horntail leverage levitate libertarian sanction scathe semitic storehouse sweeney unbeknownst")
Context: BbhWordSorting186CT: expected sorted list, got [affable almost antic apache astute deadlock dandelion delphic execution fortunate horntail libertarian leverage levitate sanction scathe semitic storehouse sweeney unbeknownst] |
Failed | IN999 OUT962 |
| ✓ | BbhWordSorting187CTSort the following words alphabetically: List: percent maser roadside we're caching distort phillip emboss epistemology gherkin somewhat spicy injustice winston savoyard delicious defend indicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 069 |
| ✓ | BbhWordSorting188CTSort the following words alphabetically: List: anarchic liven shortcut total diatom guelph usage acoustic yarmulke phenomenology neo bureaucracy trammel leftward immovable fabricate syndrome suggestive provide1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT734 |
| × | BbhWordSorting189CTSort the following words alphabetically: List: them abdominal address fomalhaut pay optimal hanoverian yang berry tattoo islamabad swathe jordan bounty stearic stigmata involve effusive tornado1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("abdominal address berry bounty effusive fomalhaut hanoverian involve islamabad jordan optimal pay stearic stigmata swathe tattoo them tornado yang")
Context: BbhWordSorting189CT: expected sorted list, got [abdominal address berry bounty effusive fomalhaut hanoverian islamabad involve jordan optimal pay stearic stigmata swathe tattoo them tornado yang] |
Failed | IN995 OUT893 |
| ✓ | BbhWordSorting190CTSort the following words alphabetically: List: maraud downtrodden hurst propane morphine parsonage gadgetry inertial gamin1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT513 |
| ✓ | BbhWordSorting191CTSort the following words alphabetically: List: foamflower lawful cayenne chandigarh only excursion declassify equanimity aseptic inclement october debauch kruger groupoid scorch dingy alterate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 011 |
| ✓ | BbhWordSorting192CTSort the following words alphabetically: List: acidify antagonism asteria1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT141 |
| ✓ | BbhWordSorting193CTSort the following words alphabetically: List: elton olympic iodinate captious ineligible sherman1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT319 |
| ✓ | BbhWordSorting194CTSort the following words alphabetically: List: alleviate duopoly mattress gland benelux townsmen buoyant klaxon hardbound tomography felice gunk1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 119 |
| ✓ | BbhWordSorting195CTSort the following words alphabetically: List: solvate nat usable kajar downey vision detest aitken barycentric1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT363 |
| × | BbhWordSorting196CTSort the following words alphabetically: List: griffith common nanosecond loosestrife decoy swabby aerospace denmark yates frill spitz enviable screechy allure sow spermatozoa saute jibe exclusive1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("aerospace allure common decoy denmark enviable exclusive frill griffith jibe loosestrife nanosecond saute screechy sow spermatozoa spitz swabby yates")
Context: BbhWordSorting196CT: expected sorted list, got [aerospace allure common decoy denmark enviable exclusive frill griffith jibe loosestrife nanosecond saute screechy spermatozoa spitz sow swabby yates] |
Failed | IN1 003 OUT907 |
| ✓ | BbhWordSorting197CTSort the following words alphabetically: List: reverie giantess muddy mast callous bate dnieper prank cortez staunch satisfy dogging moran climb garrison1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT918 |
Reasoning - BBH word sorting (set 7)
25 passed1 failed |
96.2% |
IN25 598 OUT15 014 | |
| ✓ | BbhWordSorting198CTSort the following words alphabetically: List: avoid whimsey grassland elegy pitfall error gaillardia hostile caramel conclave stepchild accept carbuncle compressor utopia drib spectra rosa embower1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT791 |
| × | BbhWordSorting199CTSort the following words alphabetically: List: rowley midshipman saline furl geld locomote exhumation anarchic bstj gradual riddance j liniment pantheist elution profess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("anarchic bstj elution exhumation furl geld gradual j liniment locomote midshipman pantheist profess riddance rowley saline")
Context: BbhWordSorting199CT: expected sorted list, got [anarchic bstj elution exhumation furl geld gradual j liminent locomote midshipman pantheist profess riddance rowley saline] |
Failed | IN995 OUT535 |
| ✓ | BbhWordSorting200CTSort the following words alphabetically: List: hangout drafty supposable sushi proscenium anticonvulsant you'd magnificent scrub embolden hasty pewee carrageen disseminate glamour discovery registrar acuity1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 099 |
| ✓ | BbhWordSorting201CTSort the following words alphabetically: List: sweatband murray1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT105 |
| ✓ | BbhWordSorting202CTSort the following words alphabetically: List: resignation dormouse bathos walt amethyst obtuse1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT292 |
| ✓ | BbhWordSorting203CTSort the following words alphabetically: List: paginate amicable they've histology extreme indoeuropean sonority tramway turvy coo deerstalker browne henchman pelvis calumny1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT664 |
| ✓ | BbhWordSorting204CTSort the following words alphabetically: List: transgressor sidewinder tacky casualty shouldn't courtier leprosy avoidance gibbon merge1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT554 |
| ✓ | BbhWordSorting205CTSort the following words alphabetically: List: intercept tamp geopolitic loci newsmen umber credit atmospheric lunge siren swart chess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT643 |
| ✓ | BbhWordSorting206CTSort the following words alphabetically: List: wink envious scotia planetaria pooh emancipate army1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT407 |
| ✓ | BbhWordSorting207CTSort the following words alphabetically: List: condescend rampant percolate coltish date rochester placid significant1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT612 |
| ✓ | BbhWordSorting208CTSort the following words alphabetically: List: infantrymen knob yaounde aqueous deregulate lysergic gala1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT286 |
| ✓ | BbhWordSorting209CTSort the following words alphabetically: List: promulgate altercate foraminifera sophocles raft wrongdoer syllabus jive cornerstone gossamer courtroom insist dusenberg sal1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT667 |
| ✓ | BbhWordSorting210CTSort the following words alphabetically: List: jacob sleight globule consonant musician1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT184 |
| ✓ | BbhWordSorting211CTSort the following words alphabetically: List: nascent muddy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT97 |
| ✓ | BbhWordSorting212CTSort the following words alphabetically: List: automate abner shampoo gurgle none exquisite abramson amity fruitful shorten waterproof1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT810 |
| ✓ | BbhWordSorting213CTSort the following words alphabetically: List: search marcello depreciate unary frame numbly sprout clench agamemnon herbivorous eject zaire lien forum1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 112 |
| ✓ | BbhWordSorting214CTSort the following words alphabetically: List: saul fresco teen chopstick footwork smalley anaplasmosis palisade orthography storey bumble pilate clue ingot fiesta1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT743 |
| ✓ | BbhWordSorting215CTSort the following words alphabetically: List: avalanche chaplin squabble coachman ipsilateral clonic cameroon fortran cram medea referring kennan canal pyridine ussr postpone1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 218 |
| ✓ | BbhWordSorting216CTSort the following words alphabetically: List: peculiar margaret contravention ingratiate dreg superintendent drapery bizarre sequential1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT499 |
| ✓ | BbhWordSorting217CTSort the following words alphabetically: List: sweatshirt bone ia wagoneer doleful homeobox hindustan convergent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT514 |
| ✓ | BbhWordSorting218CTSort the following words alphabetically: List: pewter gross gujarati cortland diffusible nomograph stag broadcast galvanometer transition incestuous sketchbook scout larynx1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT567 |
| ✓ | BbhWordSorting219CTSort the following words alphabetically: List: portsmouth impalpable portrait gouda brainy succumb fatuous hoof lisbon erudite malaria enigma isaacson cony stronghold servomechanism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT883 |
| ✓ | BbhWordSorting220CTSort the following words alphabetically: List: testate orchestra permafrost leasehold shiva1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT228 |
| ✓ | BbhWordSorting221CTSort the following words alphabetically: List: logarithm robertson inanimate uganda micron incurring wilcox lumber sammy souvenir newman profuse maladapt cone butch airlift homeowner1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT763 |
| ✓ | BbhWordSorting222CTSort the following words alphabetically: List: chrysalis wallaby1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT109 |
| ✓ | BbhWordSorting223CTSort the following words alphabetically: List: dredge checksum huckster baronial spotlight circumstance eulogy comment felicia dartmouth monochromator neuroanatomic emittance1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT632 |
Reasoning - BBH word sorting (set 8)
25 passed1 failed |
96.2% |
IN25 656 OUT15 983 | |
| ✓ | BbhWordSorting224CTSort the following words alphabetically: List: breezy germinal biplane canada nutrient archipelago peloponnese thigh conspiracy constructor accrue dobbin lost hamburger insubstantial malleable ted laramie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT948 |
| ✓ | BbhWordSorting225CTSort the following words alphabetically: List: wagging cabdriver astronomic pivot loch coherent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT252 |
| ✓ | BbhWordSorting226CTSort the following words alphabetically: List: inability nucleus vague possible invocable janice erg1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT329 |
| ✓ | BbhWordSorting227CTSort the following words alphabetically: List: gag grecian ada nobody militia wildcat right abc blend cankerworm hanukkah pierson ron falcon quad flamboyant lobster kruger austere indicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT899 |
| ✓ | BbhWordSorting228CTSort the following words alphabetically: List: babysat curvaceous hugh cutaneous spoke consul stationarity regiment1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT661 |
| ✓ | BbhWordSorting229CTSort the following words alphabetically: List: woebegone nimble complementary afternoon immaculate laughlin kind hesitate loire septuagenarian sandia toggle dixie horsepower sierra shuffleboard mechanism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT801 |
| ✓ | BbhWordSorting230CTSort the following words alphabetically: List: linchpin arrear eavesdropping fasciculate shin postdoctoral maritime ibn henry hermaphrodite sticky jane brookside incorrigible vehicular herodotus1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT825 |
| ✓ | BbhWordSorting231CTSort the following words alphabetically: List: sinter cortex kangaroo pillage threshold insane mcleod roundabout stipulate marionette trammel incident1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT528 |
| ✓ | BbhWordSorting232CTSort the following words alphabetically: List: soulful dnieper labile lease vehicular1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT232 |
| ✓ | BbhWordSorting233CTSort the following words alphabetically: List: cornerstone green brindle prowl nonce ballard scrappy credulous des pitney difluoride toward predilect nostalgic rape mixup horseplay jew curio1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 061 |
| ✓ | BbhWordSorting234CTSort the following words alphabetically: List: covenant thence davenport sonorant scoreboard densitometer noisy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT398 |
| ✓ | BbhWordSorting235CTSort the following words alphabetically: List: inhabitation bona cicero purport gaussian silver responsible rastus zig exonerate portfolio coastal asset ruanda dusky handlebar1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT659 |
| ✓ | BbhWordSorting236CTSort the following words alphabetically: List: implementor bangui nudge cavalier refractory islamabad contextual abramson waite railway carlisle magistrate silvery picnicking dustbin emacs1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 021 |
| ✓ | BbhWordSorting237CTSort the following words alphabetically: List: strove hendrickson apostasy jacm indonesia vibrate afloat lanthanide grosbeak scavenge quark chattel conner bechtel melancholy ferment1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT699 |
| ✓ | BbhWordSorting238CTSort the following words alphabetically: List: spongy separate gabriel jackson quail hendrickson washy backslide shelter calvert anastomosis variety commando pizzicato sticktight syndicate hollister1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT699 |
| × | BbhWordSorting239CTSort the following words alphabetically: List: behold oxalic maybe hew steel termcap pray stiffen dissipate misogyny format dew1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("behold dew dissipate format hew maybe misogyny oxalic pray steel stiffen termcap")
Context: BbhWordSorting239CT: expected sorted list, got [behold dew format hew maybe misogyny oxalic pray steel stiffen termcap] |
Failed | IN982 OUT434 |
| ✓ | BbhWordSorting240CTSort the following words alphabetically: List: dove rodriguez sonant monaco strap horticulture claret play paschal bizarre valuate colonist baden deplore1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT562 |
| ✓ | BbhWordSorting241CTSort the following words alphabetically: List: patrolman arabesque novel spearmint arkansan bend brassiere diagnosable ytterbium sheep bedroom yen planeload deterrent fossiliferous alveolar trident fluke curvilinear1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT1 249 |
| ✓ | BbhWordSorting242CTSort the following words alphabetically: List: papaw perfidious frugal labial feel sonar blunderbuss box dinnertime oresteia1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT433 |
| ✓ | BbhWordSorting243CTSort the following words alphabetically: List: haddock colossus sheepskin honeymoon candid pragmatic agrarian people applicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT586 |
| ✓ | BbhWordSorting244CTSort the following words alphabetically: List: catechism daddy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT97 |
| ✓ | BbhWordSorting245CTSort the following words alphabetically: List: holocene muriel r&d pathetic talon mare staircase bradshaw aperture sigh1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT429 |
| ✓ | BbhWordSorting246CTSort the following words alphabetically: List: bust midshipmen musicale urinary chalk rocket resiny horror tangential sapient dumb serf pathway urea goucher fatty masonry dentistry cowboy roadrunner1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT827 |
| ✓ | BbhWordSorting247CTSort the following words alphabetically: List: sanhedrin scratchy helical beau venezuela awash bessie extricable indoeuropean vice pendulum cream animism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT758 |
| ✓ | BbhWordSorting248CTSort the following words alphabetically: List: him correspond seashore herpes1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT222 |
| ✓ | BbhWordSorting249CTSort the following words alphabetically: List: worcestershire bonito fritter dreamboat haggard nose whodunit1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT374 |