Benchmark summary
The model builds 2048, the sliding tile puzzle in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: 2048
11 failed |
0.0% |
IN340 531 OUT28 962 | |
| × | Game2048The model builds the game 2048 from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.4 AI queries
5 tool calls Assertion expression evaluated to false: app.Built
Context: AI: completed
App: not submitted; the last file version is graded (end: stopped: no tool call in 3 answers in a row)
Work: 4 AI calls (9 model rounds with tool calls), 1 whole-file writes, 2 edits (109 lines), 2 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 3 reminders to hand in
Compile errors: (208,48) Expected ')' to close the argument list, found 'int'.
Values:
app.Built = false |
Failed | IN340 531 OUT28 962 |
| × | Hidden_BestScoreThe best score follows the score and stays for the next game.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameOverWhen no move is left after a move, the game is over; arrows do nothing then, and Enter starts a new game and does nothing else in that frame.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the grid has exactly two tiles (2 or 4) and the counters have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_KeyPressesHolding an arrow makes one move; only a new press makes the next one. With two arrows at once, Left, Right, Up, Down is the order.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ManyMovesEvery move that changes the grid gives the slide result plus exactly one new tile on a cell that was left empty.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_MoveLeftLeft slides and merges every row from the left, each tile merging at most once, and adds the merged values to the score.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_MoveRightUpDownRight, Up and Down slide and merge from their own side.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_NoChangeNoTileA move that changes nothing adds no tile, no move and no score.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenThe screen shows the board, the cells in the colors of their values, and the texts.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_WinningMaking a 2048 tile wins: the win screen ignores arrows until Enter, then the game goes on and does not show the win screen again.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |