Benchmark summary
The model builds a Space Invaders-like fixed shooter game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Space Invaders-like fixed shooter
17 failed |
0.0% |
IN3 189 816 OUT40 182 | |
| × | SpaceInvadersThe model builds a Space Invaders game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.6 AI queries
44 tool calls Assertion expression evaluated to false: app.Built
Context: AI: Max tool rounds exceeded: 36.
App: not submitted; the last file version is graded (end: limit of 50 AI calls)
Work: 6 AI calls (50 model rounds with tool calls), 1 whole-file writes, 2 edits (60 lines), 40 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 5 reminders to hand in
Compile errors: (313,1) Expected an expression, found 'void'.
Values:
app.Built = false |
Failed | IN3 189 816 OUT40 182 |
| × | Hidden_AliensDropBombsAliens drop bombs from the bottom center of the lowest live alien of a column, at least once every 90 frames, and never more than 3 at once.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_BombFallsSteadilyA bomb falls at a constant speed of 2 px to 4 px per frame and disappears below the screen.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_BombHitsTheCannonA bomb that reaches the cannon is removed and costs a life; with the last life the game is over.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_CannonMovesAndStopsHolding Left or Right moves the cannon by 3 px per frame and stops it at both edges.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EdgesCountLiveAliensOnlyOnly live aliens count for the edges: without the outer columns, the formation goes further before it turns.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_FormationGetsFasterThe formation gets faster: moveFrames never grows, is at most 10 with 15 aliens left, never below 2, and the formation really moves every moveFrames frames.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_FormationMovesThe formation moves by 4 px in the 20th frame and then every 20 frames.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_FormationTurnsAtTheEdgeAt the right edge the formation moves down by 8 px and turns, and then moves left.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit, the cannon, the formation, the speed, the score, the lives and the shot have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InvasionEndsTheGameWhen the lowest live alien reaches y 210, the game is over; dead aliens do not count.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PointsByRowAliens in row 0 give 30 points, in rows 1 and 2 20 points and in rows 3 and 4 10 points.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenAfterAKillAfter an alien dies and the explosion effect ends, the screen shows one alien less and the new score.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheSceneThe screen shows the aliens in their row colors, the cannon, the background and the texts.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ShotFliesAndLeavesSpace fires one shot from the middle of the cannon; it moves up by 6 px per frame, a second Space does nothing while it flies, and it disappears at the top.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ShotKillsTheLowestAlienA shot from below hits the lowest live alien of its column, kills it, adds its points and disappears.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_WinningAndRestartKilling the last alien wins the game and stops it; Enter starts a new game.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |