Benchmark summary
The model builds a Sokoban-like box pushing puzzle in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Sokoban-like box pushing puzzle
15 failed |
0.0% |
IN66 815 OUT7 638 | |
| × | SokobanThe model builds a Sokoban game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.3 AI queries
0 tool calls Assertion expression evaluated to false: app.Built
Context: AI: completed
App: none handed in (end: stopped: no tool call in 3 answers in a row)
Work: 3 AI calls (3 model rounds with tool calls), 0 whole-file writes, 0 edits (0 lines), 0 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 2 reminders to hand in
Values:
app.Built = false |
Failed | IN66 815 OUT7 638 |
| × | Hidden_AllLevelsAfter the last level the game is complete; Enter starts a new game and the best results stay.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_BestResultsA better result replaces the best one, a worse one does not, and R after completing a level plays it again.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_BestTieBreakThe best result compares moves first and pushes only when the moves are equal.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_BoxOrderA pushed box keeps its place in the box lists.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit, level 1 is loaded from its map, the counters are 0 and no level has a best result.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_LevelCompleteWhen every box is on a goal, the level is complete: arrows and undo do nothing, the result becomes the best one, and the screen says LEVEL COMPLETE.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_NextLevelEnter after a completed level starts the next level, and the frame of Enter does nothing else.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_OneMovePerPressHolding an arrow longer moves the worker once: a move needs a new press.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_PushingPushing moves exactly one box; a box does not move into a wall or into another box, and then the worker stays.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_RestartR starts the level again with the counters at 0 and an empty undo history.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenThe screen shows the walls, the floor, the goals, the boxes in their two colors, the worker and the texts.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_UndoUndo takes back one move at a time, pushes included, down to the level start; then it does nothing.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_UndoSkipsBlockedMovesUndo does not count blocked moves, and after undo new moves can be undone too.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_WalkingAndWallsAn arrow moves the worker one cell per press and counts the move; walls stop the worker and are not counted.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |