Benchmark summary
The model builds a Space Invaders-like fixed shooter game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Space Invaders-like fixed shooter
17 failed |
0.0% |
IN241 333 OUT40 644 | |
| × | SpaceInvadersThe model builds a Space Invaders game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.4 AI queries
3 tool calls Assertion expression evaluated to false: app.Built
Context: AI: completed
App: none handed in (end: stopped: no tool call in 3 answers in a row)
Work: 4 AI calls (7 model rounds with tool calls), 0 whole-file writes, 0 edits (0 lines), 0 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 3 reminders to hand in
Values:
app.Built = false |
Failed | IN241 333 OUT40 644 |
| × | Hidden_AliensDropBombsAliens drop bombs from the bottom center of the lowest live alien of a column, at least once every 90 frames, and never more than 3 at once.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_BombFallsSteadilyA bomb falls at a constant speed of 2 px to 4 px per frame and disappears below the screen.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_BombHitsTheCannonA bomb that reaches the cannon is removed and costs a life; with the last life the game is over.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_CannonMovesAndStopsHolding Left or Right moves the cannon by 3 px per frame and stops it at both edges.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_EdgesCountLiveAliensOnlyOnly live aliens count for the edges: without the outer columns, the formation goes further before it turns.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_FormationGetsFasterThe formation gets faster: moveFrames never grows, is at most 10 with 15 aliens left, never below 2, and the formation really moves every moveFrames frames.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_FormationMovesThe formation moves by 4 px in the 20th frame and then every 20 frames.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_FormationTurnsAtTheEdgeAt the right edge the formation moves down by 8 px and turns, and then moves left.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit, the cannon, the formation, the speed, the score, the lives and the shot have their starting values.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_InvasionEndsTheGameWhen the lowest live alien reaches y 210, the game is over; dead aliens do not count.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_PointsByRowAliens in row 0 give 30 points, in rows 1 and 2 20 points and in rows 3 and 4 10 points.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenAfterAKillAfter an alien dies and the explosion effect ends, the screen shows one alien less and the new score.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheSceneThe screen shows the aliens in their row colors, the cannon, the background and the texts.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ShotFliesAndLeavesSpace fires one shot from the middle of the cannon; it moves up by 6 px per frame, a second Space does nothing while it flies, and it disappears at the top.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ShotKillsTheLowestAlienA shot from below hits the lowest live alien of its column, kills it, adds its points and disappears.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_WinningAndRestartKilling the last alien wins the game and stops it; Enter starts a new game.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |