Benchmark summary
The model builds a Supaplex-like digger puzzle game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Supaplex-like digger puzzle
12 passed3 failed |
80.0% |
IN999 048 OUT61 826 | |
| ✓ | SupaplexThe model builds a Supaplex-style game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.2 AI queries
24 tool calls |
Passed | IN999 048 OUT61 826 |
| ✓ | Hidden_CollectsInfotronsMurphy collects an infotron by moving onto it.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ColumnFallsTogetherA column of objects falls together in one physics step, because the rows are processed from the bottom up.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_EnemiesAndMurphyAn enemy kills Murphy by stepping onto him or when he steps onto it; a falling zonk kills an enemy.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_EnemiesWalkEnemies take a step every 8 frames, only into empty cells.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ExitThe exit lets Murphy in only after all needed infotrons are collected; then the game is won.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_FallingZonkKillsA falling zonk kills Murphy; a zonk that is resting on him does not.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_GameOverAndRestartWhile the game is over nothing changes; Enter starts a new game with the start level.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: enemyX.Count == 2 && Hidden_At(6, 1) == 3 && Hidden_At(17, 10) == 5 && Hidden_At(5, 8) == 4
Context: the start level
Values:
enemyX.Count = 2 (line 964) |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the level is the given map, Murphy, the enemies and the infotron counters have their starting values.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: level[y * 20 + x] == expected
Context: cell (2, 1) is 0, expected 1
Values:
expected = 1 (line 673) |
Failed | IN0 OUT0 |
| ✓ | Hidden_MurphyMovesAndEatsBaseMurphy moves one cell when an arrow key is held, eats the base, and waits 4 frames before the next move.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ObjectsFallZonks and infotrons fall one cell per physics step, every 4 frames, and stop on base.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ObjectsRollA round object on another round object rolls left if the left side is free, otherwise right, and stays if both sides are blocked.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_PushingZonksMurphy pushes a zonk left or right into an empty cell.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheLevelThe screen shows the level tiles, Murphy, the enemies and the text.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Frame.CountPixels(Color.Hex(0x795548)) == bases * 256
Context: base: 0 pixels, 138 cells (line 985) |
Failed | IN0 OUT0 |
| ✓ | Hidden_WhatStopsMurphyHardware stops Murphy, and he cannot push a zonk up, down or into something that is not empty.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |