Benchmark summary
The model builds a Supaplex-like digger puzzle game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Supaplex-like digger puzzle
15 failed |
0.0% |
IN1 111 322 OUT76 587 | |
| × | SupaplexThe model builds a Supaplex-style game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.12 AI queries
7 tool calls Assertion expression evaluated to false: app.Built
Context: AI: completed
App: not submitted; the last file version is graded (end: stopped: no tool call in 3 answers in a row)
Work: 12 AI calls (19 model rounds with tool calls), 1 whole-file writes, 2 edits (7 lines), 3 CheckApp calls, 1 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 11 reminders to hand in
Compile errors: (82,19) Expected ';' after the statement, found 'f'.
Values:
app.Built = false |
Failed | IN1 111 322 OUT76 587 |
| × | Hidden_CollectsInfotronsMurphy collects an infotron by moving onto it.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ColumnFallsTogetherA column of objects falls together in one physics step, because the rows are processed from the bottom up.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EnemiesAndMurphyAn enemy kills Murphy by stepping onto him or when he steps onto it; a falling zonk kills an enemy.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EnemiesWalkEnemies take a step every 8 frames, only into empty cells.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ExitThe exit lets Murphy in only after all needed infotrons are collected; then the game is won.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_FallingZonkKillsA falling zonk kills Murphy; a zonk that is resting on him does not.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameOverAndRestartWhile the game is over nothing changes; Enter starts a new game with the start level.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the level is the given map, Murphy, the enemies and the infotron counters have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_MurphyMovesAndEatsBaseMurphy moves one cell when an arrow key is held, eats the base, and waits 4 frames before the next move.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ObjectsFallZonks and infotrons fall one cell per physics step, every 4 frames, and stop on base.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ObjectsRollA round object on another round object rolls left if the left side is free, otherwise right, and stays if both sides are blocked.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PushingZonksMurphy pushes a zonk left or right into an empty cell.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheLevelThe screen shows the level tiles, Murphy, the enemies and the text.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_WhatStopsMurphyHardware stops Murphy, and he cannot push a zonk up, down or into something that is not empty.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |