Benchmark summary
The model builds an Asteroids-like space rocks shooter with power-ups in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Asteroids-like space rocks shooter
14 failed1 broken |
0.0% |
IN70 889 OUT71 952 | |
| ! | AsteroidsThe model builds an Asteroids game with power-ups from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
2 tool calls Assertion expression evaluated to false: app.Built
Context: AI: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.
App: not submitted; the last file version is graded (end: AI error: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.)
Work: 1 AI calls (3 model rounds with tool calls), 0 whole-file writes, 0 edits (0 lines), 0 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 0 reminders to hand in
Compile errors: The script needs a function 'void AppInit()' to run as an app.
Values:
app.Built = false |
Broken | IN70 889 OUT71 952 |
| × | Hidden_FiringSpace fires a bullet from the ship's nose; it flies at 6 px per frame, lives 40 frames, and the next shot comes 8 frames later; at most 4 bullets exist.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_HitDistanceA bullet hits a rock only when its center is closer than the rock's radius.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the ship, the counters and the rocks of wave 1 have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_NextWaveWhen no rock is left, the next wave starts with one large rock more, at least 60 px from the ship.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PowerUpsAppearAt least one power-up appears for every 10 destroyed rocks, at the position of a destroyed rock.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PowerUpsExpirePower-ups that are not picked up disappear after 300 frames.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_RapidFireAndShieldRapid fire shortens the cooldown to 3 frames, and the shield protects the ship from rocks.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_RocksSplitA bullet splits a large rock into two medium rocks at its position, a medium rock into two small ones; small rocks vanish; points are 20, 50 and 100.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheFieldThe screen shows the rocks, a power-up, a bullet, the ship and the texts.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ShipHitA rock that touches the ship costs a life; the ship goes back to the center and is safe for 90 frames; without lives the game is over.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ThrustAndDriftUp pushes the ship by 0.15 per frame in its heading up to a speed of 4; without Up the velocity shrinks by 1 % per frame.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_TripleShotTriple shot fires three bullets 15 degrees apart for 300 frames.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_TurningLeft and Right turn the ship by 6 degrees per frame and keep the angle between 0 and 360.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_WrappingThe ship wraps around the edges of the screen.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |