Benchmark summary
The model builds a Scorched Earth-like turn-based artillery duel in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Scorched Earth-like artillery duel
12 failed |
0.0% |
IN3 429 922 OUT79 413 | |
| × | ScorchedEarthThe model builds a Scorched Earth artillery game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.9 AI queries
41 tool calls Assertion expression evaluated to false: app.Built
Context: AI: Max tool rounds exceeded: 2.
App: not submitted; the last file version is graded (end: limit of 50 AI calls)
Work: 9 AI calls (50 model rounds with tool calls), 1 whole-file writes, 4 edits (60 lines), 27 CheckApp calls, 8 refused rewrites, 1 refused repeated edits, 0 rejected submissions, 8 reminders to hand in
Compile errors: (135,5) Found 'if' outside any function: statements must be inside a function body. A '}' above probably closes a function too early, or a function header is missing.
Values:
app.Built = false |
Failed | IN3 429 922 OUT79 413 |
| × | Hidden_AimingLeft and Right change the player's angle by 1 per frame within 0 .. 180, Up and Down the power within 10 .. 100.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_BallisticsSpace fires the shell from the end of the turret; it does not move in that frame and then flies by gravity and wind exactly.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ComputerTurnsComputer tanks fire within 60 frames of their turn, from their turret, within the angle and power limits; the wind changes by at most 3 per turn.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_CraterA shell that hits the terrain makes a crater by the exact rule, and the turn goes to the next tank.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_DamageAn explosion damages tanks by distance: (int)(m * (1 - distance / r)); the player's score grows by the damage to others.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EdgesOfTheScreenA shell that leaves the screen at the side is gone without an explosion; above the top it keeps flying.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_FallingA tank whose ground is blown away falls onto the ground and loses half of the fall height.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the terrain, the tanks and the counters have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_LargeMissileA large missile makes a crater of radius 24 and costs ammunition; W switches the weapon only while there is ammunition.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenThe screen shows the terrain, the tanks, the shell and the texts.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_TurnsAndRoundEndTanks without hp die; the turn order skips dead tanks; the round ends when at most one tank is alive.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |