Benchmark summary
The model builds a Diamond Dash-like gem clicker (groups of 3 or more gems vanish, the rest fall) played with the mouse with a polished look in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Diamond Dash-like gem clicker
3 passed12 failed |
20.0% |
IN410 138 OUT19 806 | |
| ✓ | DiamondDashThe model builds a mouse game like Diamond Dash (click groups of 3 or more matching diamonds, they fall and refill, a 60-second round with combos) from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.4 AI queries
5 tool calls |
Passed | IN410 138 OUT19 806 |
| × | Hidden_ClicksOutsideAndAtTheEdgesClicks outside the board do nothing; the board edges belong to the edge cells.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: score == 30 && grid[0] == 1 && grid[1] == 2 && grid[2] == 3
Context: pixel (40, 100) is cell (0, 0): score 0
Values:
score = 0
grid[0] = 5
grid[1] = 5
grid[2] = 5 (line 447) |
Failed | IN0 OUT0 |
| × | Hidden_ComboA hit within the combo time raises the combo (at most 5) and multiplies the score; after the time runs out the combo starts again at 1.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: combo == 3 && comboFrames == 60 && score == 90
Context: a hit in the combo time: combo 2, comboFrames 29, score 0
Values:
combo = 2
comboFrames = 29
score = 0 (line 400) |
Failed | IN0 OUT0 |
| × | Hidden_EffectsEndThe effects of a hit and of a miss end within a second: afterwards the still board looks exactly as before the miss.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: lockFrames == 29 && Hidden_GridText() == grid0
Context: a miss: lockFrames 0
Values:
lockFrames = 0
grid0 = "12341234/34123412/12341234/34123412/12341234/34123412/12341234/34123412/55541234" (line 617) |
Failed | IN0 OUT0 |
| × | Hidden_FallAndRefillOrderThe diamonds above a removed group keep their order, and columns are refilled from left to right, each from its lowest empty cell up.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_GridText() == Hidden_RowsText(expected)
Context: grid 12341234/34123412/12341234/34123412/15555234/34153412/12341234/34123412/12341234, expected 15421234/32331412/14143234/32321412/14143234/34123412/12341234/34123412/12341234 (line 328) |
Failed | IN0 OUT0 |
| × | Hidden_GroupsAreOrthogonalGroups connect only through left, right, up and down neighbors, and the whole group is removed however it winds.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_GridText() == before && score == 0 && lockFrames == 29
Context: a diagonal neighbor does not join the group: score 0, lockFrames 0
Values:
before = "12541234/35123412/15341234/34123412/12341234/34123412/12341234/34123412/12341234"
score = 0
lockFrames = 0 (line 299) |
Failed | IN0 OUT0 |
| × | Hidden_HitRemovesAGroupA click on a group of 3 removes it, the diamonds above fall down and the new ones come from refill, bottom first.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_GridText() == Hidden_RowsText(expected)
Context: grid 12341234/34123412/12341234/34123412/12341234/34123412/12541234/34553412/12341234, expected 12231234/34143412/12321234/34143412/12321234/34143412/12321234/34143412/12341234 (line 280) |
Failed | IN0 OUT0 |
| × | Hidden_HoverHighlightThe mouse over a group of 3 or more highlights exactly the cells of that group; over a smaller group nothing changes on the board.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: inGroup ? hover[i] != still[i] : hover[i] == still[i]
Context: cell (2, 3) of the group is not highlighted (line 589) |
Failed | IN0 OUT0 |
| ✓ | Hidden_InitialStateAfter AppInit a full board of colors 1 to 5 is ready, the refill list is empty and the counters have their starting values.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_MissLocksClicksA click on a group of 1 or 2 is a miss: nothing is removed, the combo ends and clicks are ignored for the next 30 frames.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: lockFrames == 30 && combo == 0 && comboFrames == 0 && Hidden_GridText() == before && score == 0
Context: a miss on a single diamond: lockFrames 0, combo 3
Values:
lockFrames = 0
combo = 3
comboFrames = 39
before = "12341234/34123412/12341234/34123412/12341234/34123412/12341554/34123452/12341234"
score = 0 (line 369) |
Failed | IN0 OUT0 |
| ✓ | Hidden_NewGameEnter starts a new game at any time, also after the time is up; a click in the same frame does nothing.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_OwnRefillWith an empty refill list the game picks the new colors itself, always from 1 to 5.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: score == before + 30
Context: round 0: the hit counts
Values:
score = 0 (line 535) |
Failed | IN0 OUT0 |
| × | Hidden_ScoreBySizeThe score of a hit is 10 * n * (n - 2) for 3, 4 and a whole column of 9 diamonds.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: score == 80
Context: a square of four 5s gives 80: score 0
Values:
score = 0 (line 344) |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheDiamondsWhen the board is still, the middle pixel of every cell shows its diamond's color: same color, same pixel; different colors, different pixels; also after a hit.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: grid[64] == 3 && grid[0] == 5
Context: the hit changed the grid
Values:
grid[64] = 5
grid[0] = 1 (line 563) |
Failed | IN0 OUT0 |
| × | Hidden_TimeRunsOutThe time counts down once per frame and is shown in whole seconds rounded up; at 0 the round ends and clicks do nothing.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Frame.HasText("TIME UP") && Frame.HasText("FINAL SCORE 470") && Frame.HasText("TIME 0")
Context: the end of the round on the screen (line 481) |
Failed | IN0 OUT0 |