Benchmark summary
The model builds a Tetris-like falling blocks game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Tetris-like falling blocks
18 broken |
0.0% |
IN20 428 OUT19 888 | |
| ! | TetrisThe model builds a Tetris game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
0 tool calls Assertion expression evaluated to false: app.Built
Context: AI: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.
App: none handed in (end: AI error: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.)
Work: 1 AI calls (1 model rounds with tool calls), 0 whole-file writes, 0 edits (0 lines), 0 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 0 reminders to hand in
Values:
app.Built = false |
Broken | IN20 428 OUT19 888 |
| ! | Hidden_GameOverAndRestartWhen a new piece does not fit, the game is over and stops; Enter starts a new game.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_GetsFasterThe game gets faster: dropFrames never grows, is at most 20 after 10 rows, never below 2, and gravity really steps every dropFrames frames.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_GravityGravity moves the piece down by one row in the 30th frame and then every 30 frames.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_HardDropSpace drops the piece to the bottom, locks it, adds 2 points per row and brings the next piece.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_InitialStateAfter AppInit the board is empty, the first piece is at its start position, and score, lines and speed have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_LockingWithoutHardDropSoft drop and gravity lock a piece that cannot move down.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_MovesAndStopsAtTheWallsLeft and Right move the piece by one column and stop it at the walls.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_OneRowA full row is removed, the rows above move down, and score grows by 100.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_PieceCompletesRowsWithAGapA piece that completes two rows with a not full row between them removes exactly those two rows.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_PieceOrderThe pieces come in an order where any 14 pieces in a row contain all 7 types, and each new piece is the announced next one.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_PiecesAndRotationsEvery piece in every rotation has the cells of the task's table and rotation rule: a hard drop locks exactly those cells.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_RotationAtTheWallA rotation that does not fit is either ignored or moved by at most 2 columns to a free position in the same row.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_RowsInAMixedBoardFull rows that are not next to each other in a mixed board are removed, the rows above them move down exactly, and the screen shows the new board.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_ScreenAfterARowLocked cells are drawn in the colors of their types, also after a row is removed and its effect ends.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_ScreenShowsTheSceneThe screen shows the board area, the falling piece, the next piece and the texts.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_SeveralRowsRemoving 2, 3 or 4 rows at once gives 300, 500 or 800 points.0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |
| ! | Hidden_SoftDropHolding Down moves the piece down one row every frame; the gravity counter does not grow in those frames (step 5 is skipped).0 AI queries
0 tool callsHiddenTest Not run: the model's answer was cut off before an app existed (AiRunner marked it broken: a loop or the output token limit). |
Broken | IN0 OUT0 |