Benchmark summary
The model builds a bubble shooter game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Bubble shooter
7 passed11 failed |
38.9% |
IN2 117 303 OUT21 578 | |
| ✓ | PuzzleBobbleThe model builds a Puzzle Bobble game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
47 tool calls |
Passed | IN2 117 303 OUT21 578 |
| ✓ | Hidden_AimingLeft and Right turn the launcher by 2 degrees per frame while held, within 15-165 degrees.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_AngledShotsShots at an angle reach the ceiling in the cell nearest to where they stop.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(0, hiddenCols[hiddenI]) == 0 && Hidden_Count() == 1
Context: angle 75: expected row 0, column 7, the field is (line 869) |
Failed | IN0 OUT0 |
| × | Hidden_CeilingAndTiesA bubble that reaches the ceiling goes into the nearest empty cell; on a tie the smaller row, then the smaller column wins.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(0, 3) == 1 && Hidden_Count() == 2
Context: straight up into row 0, column 3 (tie with column 4): 0,7=4 (line 851) |
Failed | IN0 OUT0 |
| × | Hidden_CeilingDropAfter every 8th shot that does not clear the field the ceiling drops by one row; the field moves down 14 pixels and the shots reach the new ceiling sooner.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(0, 0) == 2 && shotsSinceDrop == 0 && ceilingDrops == 1
Context: the 8th shot drops the ceiling: shots 0, drops 1, field 0,7=3
Values:
shotsSinceDrop = 0
ceilingDrops = 1 (line 1029) |
Failed | IN0 OUT0 |
| ✓ | Hidden_FallingAfter a pop, bubbles that no longer hang from row 0 fall for 20 points each; bubbles still connected to row 0 stay, whatever their color.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_GameOverWhen a bubble's center reaches y 192 the game is over; then only Enter works, and it starts a new game in a frame that does nothing else.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: angle == 90 && !shotActive
Context: the Enter frame does nothing else: angle 92
Values:
angle = 92
shotActive = false (line 1067) |
Failed | IN0 OUT0 |
| × | Hidden_GameOverByBubbleA bubble that attaches in a row with center y 192 or more ends the game.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(11, 3) == 5
Context: the shot goes under the column into row 11: 0,3=1 1,3=3 2,3=1 3,3=3 4,3=1 5,3=3 6,3=1 7,3=3 8,3=1 9,3=3 10,3=1 (line 1079) |
Failed | IN0 OUT0 |
| × | Hidden_HitBubblesA bubble that hits grid bubbles goes into the nearest empty cell.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(1, hiddenCols[hiddenI]) == 2 && Hidden_Count() == 9
Context: angle 75 under a full row 0: expected row 1, column 6, the field is 0,0=0 0,1=1 0,2=0 0,3=1 0,4=0 0,5=1 0,6=0 0,7=1 (line 944) |
Failed | IN0 OUT0 |
| ✓ | Hidden_InitialStateAfter AppInit, level 1 is loaded from its map, the launcher points up and its two colors are on the field.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_LevelClearWhen the field is empty after a shot, the level is clear and the shot counter stays; Enter starts the next level and keeps the score.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: !levelClear && level == 1 && score == 30 && angle == 90 && shotsSinceDrop == 0
Context: level 2 starts, the score stays, the Enter frame does nothing else: level 1, score 30, angle 88
Values:
level = 1
score = 30
angle = 88
levelClear = false
shotsSinceDrop = 0 (line 1103) |
Failed | IN0 OUT0 |
| ✓ | Hidden_NextColorThe next bubble always gets a color that is on the field when the shot starts.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_OddRowNeighborsNeighbors on odd rows are shifted: (1, 3) touches (2, 3) and (2, 4), and (3, 3) touches (2, 3) and (2, 4).0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(3, 3) == 0 && Hidden_At(2, 1) == 0 && Hidden_At(2, 2) == 0 && Hidden_Count() == 8 && score == 0
Context: (3, 3) does not touch (2, 2), so the red bubbles stay: 0,3=1 0,4=1 1,2=2 1,3=2 2,1=0 2,2=0 2,4=2, score 10
Values:
score = 10 (line 995) |
Failed | IN0 OUT0 |
| × | Hidden_PopThree or more connected bubbles of the shot's color pop for 10 points each; two stay.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(0, 3) == 0 && Hidden_At(1, 3) == 0 && Hidden_Count() == 3 && score == 0
Context: two red bubbles stay: 0,0=1, score 20
Values:
score = 20 (line 956) |
Failed | IN0 OUT0 |
| ✓ | Hidden_ScreenThe screen shows the field, the lowered ceiling, the deadline, the bubbles at their cells, the launcher, the next bubble, the aim and the texts.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ShotSpace starts a shot from (160, 220) that moves 8 pixels per frame from the next frame on; the launcher colors move on, and Space does nothing while a bubble flies.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_StopDistanceA bubble stops next to a grid bubble only when it comes closer than 14 pixels; passing at 14-16 pixels does not stop it.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(0, 3) == 0
Context: angle 93 passes (2, 2) at about 15 pixels and reaches the ceiling: (line 921) |
Failed | IN0 OUT0 |
| × | Hidden_WallBounceThe side walls reflect the bubble; the bubble stays between x = 104 and x = 216.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_At(0, hiddenCols[hiddenI]) == 0 && Hidden_Count() == 1
Context: angle 59: expected row 0, column 3, the field is (line 892) |
Failed | IN0 OUT0 |