Benchmark summary
The model builds a bubble shooter game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Bubble shooter
18 failed |
0.0% |
IN572 608 OUT55 092 | |
| × | PuzzleBobbleThe model builds a Puzzle Bobble game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.6 AI queries
6 tool calls Assertion expression evaluated to false: app.Built
Context: AI: completed
App: not submitted; the last file version is graded (end: stopped: no tool call in 3 answers in a row)
Work: 6 AI calls (12 model rounds with tool calls), 1 whole-file writes, 1 edits (79 lines), 4 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 5 reminders to hand in
Compile errors: (538,1) Found an extra '}' outside any function: the braces above do not match (a '}' too many, or a function closed too early).
Values:
app.Built = false |
Failed | IN572 608 OUT55 092 |
| × | Hidden_AimingLeft and Right turn the launcher by 2 degrees per frame while held, within 15-165 degrees.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_AngledShotsShots at an angle reach the ceiling in the cell nearest to where they stop.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_CeilingAndTiesA bubble that reaches the ceiling goes into the nearest empty cell; on a tie the smaller row, then the smaller column wins.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_CeilingDropAfter every 8th shot that does not clear the field the ceiling drops by one row; the field moves down 14 pixels and the shots reach the new ceiling sooner.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_FallingAfter a pop, bubbles that no longer hang from row 0 fall for 20 points each; bubbles still connected to row 0 stay, whatever their color.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameOverWhen a bubble's center reaches y 192 the game is over; then only Enter works, and it starts a new game in a frame that does nothing else.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameOverByBubbleA bubble that attaches in a row with center y 192 or more ends the game.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_HitBubblesA bubble that hits grid bubbles goes into the nearest empty cell.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit, level 1 is loaded from its map, the launcher points up and its two colors are on the field.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_LevelClearWhen the field is empty after a shot, the level is clear and the shot counter stays; Enter starts the next level and keeps the score.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_NextColorThe next bubble always gets a color that is on the field when the shot starts.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_OddRowNeighborsNeighbors on odd rows are shifted: (1, 3) touches (2, 3) and (2, 4), and (3, 3) touches (2, 3) and (2, 4).0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PopThree or more connected bubbles of the shot's color pop for 10 points each; two stay.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenThe screen shows the field, the lowered ceiling, the deadline, the bubbles at their cells, the launcher, the next bubble, the aim and the texts.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ShotSpace starts a shot from (160, 220) that moves 8 pixels per frame from the next frame on; the launcher colors move on, and Space does nothing while a bubble flies.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_StopDistanceA bubble stops next to a grid bubble only when it comes closer than 14 pixels; passing at 14-16 pixels does not stop it.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_WallBounceThe side walls reflect the bubble; the bubble stays between x = 104 and x = 216.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |