Benchmark summary
The model builds a Diamond Dash-like gem clicker (groups of 3 or more gems vanish, the rest fall) played with the mouse with a polished look in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Diamond Dash-like gem clicker
6 passed9 failed |
40.0% |
IN1 224 031 OUT19 925 | |
| ✓ | DiamondDashThe model builds a mouse game like Diamond Dash (click groups of 3 or more matching diamonds, they fall and refill, a 60-second round with combos) from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
19 tool calls |
Passed | IN1 224 031 OUT19 925 |
| × | Hidden_ClicksOutsideAndAtTheEdgesClicks outside the board do nothing; the board edges belong to the edge cells.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: score == 30 && grid[0] == 1 && grid[1] == 2 && grid[2] == 3
Context: pixel (40, 100) is cell (0, 0): score 30
Values:
score = 30
grid[0] = 1
grid[1] = 3
grid[2] = 1 (line 1096) |
Failed | IN0 OUT0 |
| × | Hidden_ComboA hit within the combo time raises the combo (at most 5) and multiplies the score; after the time runs out the combo starts again at 1.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: combo == 5 && score == 90 + 120 + 30 + 150
Context: the combo stays at 5: combo 5, score 240
Values:
combo = 5
score = 240 (line 1073) |
Failed | IN0 OUT0 |
| × | Hidden_EffectsEndThe effects of a hit and of a miss end within a second: afterwards the still board looks exactly as before the miss.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: after[i] == before[i]
Context: cell (0, 0) still looks different 1 second after the miss
Values:
after[i] = 1058766498
before[i] = 646625473 (line 1269) |
Failed | IN0 OUT0 |
| × | Hidden_FallAndRefillOrderThe diamonds above a removed group keep their order, and columns are refilled from left to right, each from its lowest empty cell up.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_GridText() == Hidden_RowsText(expected)
Context: grid 13131313/15242442/42431311/31332424/24241131/33131242/42424231/31313134/24242424, expected 15421234/32331412/14143234/32321412/14143234/34123412/12341234/34123412/12341234 (line 977) |
Failed | IN0 OUT0 |
| ✓ | Hidden_GroupsAreOrthogonalGroups connect only through left, right, up and down neighbors, and the whole group is removed however it winds.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_HitRemovesAGroupA click on a group of 3 removes it, the diamonds above fall down and the new ones come from refill, bottom first.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_GridText() == Hidden_RowsText(expected)
Context: grid 13131313/12424242/42123131/31334242/42441313/13131242/42424231/31313134/24242424, expected 12231234/34143412/12321234/34143412/12321234/34143412/12321234/34143412/12341234 (line 929) |
Failed | IN0 OUT0 |
| ✓ | Hidden_HoverHighlightThe mouse over a group of 3 or more highlights exactly the cells of that group; over a smaller group nothing changes on the board.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_InitialStateAfter AppInit a full board of colors 1 to 5 is ready, the refill list is empty and the counters have their starting values.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_MissLocksClicksA click on a group of 1 or 2 is a miss: nothing is removed, the combo ends and clicks are ignored for the next 30 frames.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_NewGameEnter starts a new game at any time, also after the time is up; a click in the same frame does nothing.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_OwnRefillWith an empty refill list the game picks the new colors itself, always from 1 to 5.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: score == before + 30
Context: round 1: the hit counts
Values:
score = 30 (line 1184) |
Failed | IN0 OUT0 |
| × | Hidden_ScoreBySizeThe score of a hit is 10 * n * (n - 2) for 3, 4 and a whole column of 9 diamonds.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_GridText() == "12341214/34123422/12341214/34123422/12341214/34123422/12341214/34123422/12341214"
Context: the column is refilled bottom first: 13131313/12424242/42313131/31342424/24241313/13131242/42424212/12121214/24242424 (line 1002) |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheDiamondsWhen the board is still, the middle pixel of every cell shows its diamond's color: same color, same pixel; different colors, different pixels; also after a hit.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: hiddenSame == (hiddenSeenValue[hiddenK] == grid[hiddenI])
Context: cell (2, 0) with color 3 has the middle pixel (27, 27, 47); a cell with color 1 has (27, 27, 47)
Values:
hiddenSame = true (line 863) |
Failed | IN0 OUT0 |
| × | Hidden_TimeRunsOutThe time counts down once per frame and is shown in whole seconds rounded up; at 0 the round ends and clicks do nothing.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Frame.HasText("TIME UP") && Frame.HasText("FINAL SCORE 470") && Frame.HasText("TIME 0")
Context: the end of the round on the screen (line 1130) |
Failed | IN0 OUT0 |