Benchmark summary
The model builds a Supaplex-like digger puzzle game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Supaplex-like digger puzzle
14 failed1 broken |
0.0% |
IN124 130 OUT71 359 | |
| ! | SupaplexThe model builds a Supaplex-style game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
3 tool calls Assertion expression evaluated to false: app.Built
Context: AI: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.
App: not submitted; the last file version is graded (end: AI error: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.)
Work: 1 AI calls (4 model rounds with tool calls), 1 whole-file writes, 2 edits (191 lines), 0 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 0 reminders to hand in
Compile errors: (2,12) Expected '(' to declare a function or '=' to declare a global variable after 'int murphyX', found ','. Global variables must have an initializer.
Values:
app.Built = false |
Broken | IN124 130 OUT71 359 |
| × | Hidden_CollectsInfotronsMurphy collects an infotron by moving onto it.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ColumnFallsTogetherA column of objects falls together in one physics step, because the rows are processed from the bottom up.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EnemiesAndMurphyAn enemy kills Murphy by stepping onto him or when he steps onto it; a falling zonk kills an enemy.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EnemiesWalkEnemies take a step every 8 frames, only into empty cells.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ExitThe exit lets Murphy in only after all needed infotrons are collected; then the game is won.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_FallingZonkKillsA falling zonk kills Murphy; a zonk that is resting on him does not.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameOverAndRestartWhile the game is over nothing changes; Enter starts a new game with the start level.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the level is the given map, Murphy, the enemies and the infotron counters have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_MurphyMovesAndEatsBaseMurphy moves one cell when an arrow key is held, eats the base, and waits 4 frames before the next move.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ObjectsFallZonks and infotrons fall one cell per physics step, every 4 frames, and stop on base.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ObjectsRollA round object on another round object rolls left if the left side is free, otherwise right, and stays if both sides are blocked.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PushingZonksMurphy pushes a zonk left or right into an empty cell.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheLevelThe screen shows the level tiles, Murphy, the enemies and the text.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_WhatStopsMurphyHardware stops Murphy, and he cannot push a zonk up, down or into something that is not empty.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |