Benchmark summary
The model builds a Supaplex-like digger puzzle game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Supaplex-like digger puzzle
7 passed8 failed |
46.7% |
IN1 405 767 OUT21 826 | |
| ✓ | SupaplexThe model builds a Supaplex-style game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.2 AI queries
29 tool calls |
Passed | IN1 405 767 OUT21 826 |
| ✓ | Hidden_CollectsInfotronsMurphy collects an infotron by moving onto it.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_ColumnFallsTogetherA column of objects falls together in one physics step, because the rows are processed from the bottom up.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(8, 2) == 0 && Hidden_At(8, 3) == 3 && Hidden_At(8, 4) == 3 && Hidden_At(8, 5) == 3
Context: all three moved down by one (line 911) |
Failed | IN0 OUT0 |
| × | Hidden_EnemiesAndMurphyAn enemy kills Murphy by stepping onto him or when he steps onto it; a falling zonk kills an enemy.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: gameOver
Context: Murphy steps onto the enemy
Values:
gameOver = false (line 1018) |
Failed | IN0 OUT0 |
| ✓ | Hidden_EnemiesWalkEnemies take a step every 8 frames, only into empty cells.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ExitThe exit lets Murphy in only after all needed infotrons are collected; then the game is won.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_FallingZonkKillsA falling zonk kills Murphy; a zonk that is resting on him does not.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: !gameOver && Hidden_At(5, 4) == 3
Context: after Murphy left, the zonk fell into his old cell
Values:
gameOver = false (line 928) |
Failed | IN0 OUT0 |
| × | Hidden_GameOverAndRestartWhile the game is over nothing changes; Enter starts a new game with the start level.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: gameOver
Values:
gameOver = false (line 1044) |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the level is the given map, Murphy, the enemies and the infotron counters have their starting values.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: level[y * 20 + x] == expected
Context: cell (2, 1) is 0, expected 1
Values:
expected = 1 (line 760) |
Failed | IN0 OUT0 |
| ✓ | Hidden_MurphyMovesAndEatsBaseMurphy moves one cell when an arrow key is held, eats the base, and waits 4 frames before the next move.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_ObjectsFallZonks and infotrons fall one cell per physics step, every 4 frames, and stop on base.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(5, 2) == 0 && Hidden_At(5, 3) == 3
Context: the zonk fell in frame 4 (line 860) |
Failed | IN0 OUT0 |
| × | Hidden_ObjectsRollA round object on another round object rolls left if the left side is free, otherwise right, and stays if both sides are blocked.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(4, 3) == 3 && Hidden_At(5, 3) == 0
Context: rolled left (line 895) |
Failed | IN0 OUT0 |
| ✓ | Hidden_PushingZonksMurphy pushes a zonk left or right into an empty cell.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheLevelThe screen shows the level tiles, Murphy, the enemies and the text.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Frame.CountPixels(Color.Hex(0x795548)) == bases * 256
Context: base: 0 pixels, 138 cells (line 1072) |
Failed | IN0 OUT0 |
| ✓ | Hidden_WhatStopsMurphyHardware stops Murphy, and he cannot push a zonk up, down or into something that is not empty.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |