Benchmark summary
The model builds a Contra-like side-scrolling run-and-gun game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Contra-like run and gun
15 failed1 broken |
0.0% |
IN45 909 OUT59 522 | |
| ! | RunAndGunThe model builds a Contra-like run-and-gun game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.2 AI queries
0 tool calls Assertion expression evaluated to false: app.Built
Context: AI: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.
App: not submitted; the last file version is graded (end: AI error: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.)
Work: 2 AI calls (2 model rounds with tool calls), 0 whole-file writes, 0 edits (0 lines), 0 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 1 reminders to hand in
Compile errors: (1,14) Expected '(' to declare a function or '=' to declare a global variable after 'extern Color', found 'Color'. Global variables must have an initializer.
Values:
app.Built = false |
Broken | IN45 909 OUT59 522 |
| × | Hidden_AimingThe player shoots in 8 directions: the aim comes from the held arrows, the bullet starts at the muzzle and flies at 5 px per frame.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_CrouchingDown on the ground crouches: the player does not run, still turns, and has a lower box.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_DyingAn enemy bullet or a soldier kills the player unless invulnerable; with the last life the game is over; Enter starts a new game.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EnemiesActSoldiers keep appearing at the right edge, never more than 4 at once; enemy bullets start at a soldier's chest, a turret's center or the boss, with speed 3.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_FireRateShots come every 10 frames while X is held, bullets fly 5 px per frame and leave the screen, and at most 6 player bullets exist.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the level, the player, the enemies, the power-ups and the counters have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_JumpA jump starts with playerVY -7 and follows gravity 0.4 per frame (at most 6) until the feet land on the ground again.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PitFalling into a pit costs a life even while invulnerable; the player comes back over ground with 120 frames of invulnerability.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PlatformsPlatforms are passed from below and stop a falling player on their top.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_RunningAndCameraThe player runs 2 px per frame, cannot leave the left edge of the screen, and the camera follows only forward.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheLevelThe screen shows the level, the player, a turret and the power-up relative to the camera.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_SoldiersWalkSoldiers walk 1 px per frame, fall into pits, and are removed behind the left edge.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_TargetsBullets kill soldiers (100 points), turrets need 8 hits (500 points), and the boss needs 30 hits; killing the boss wins the game.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_TurretFireA turret on the screen fires aimed bullets every 60 to 120 frames.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_WeaponsThe spread gun fires three bullets 15 degrees apart, the rapid gun every 4 frames; picking up a power-up sets the weapon.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |