Benchmark summary
The model builds Mahjong solitaire (72 tiles in 3 layers) played with the mouse with a polished look in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Mahjong solitaire
11 passed2 failed |
84.6% |
IN1 997 387 OUT22 245 | |
| ✓ | MahjongThe model builds a Mahjong solitaire game played with the mouse from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
32 tool calls |
Passed | IN1 997 387 OUT22 245 |
| ✓ | Hidden_AvailablePairsavailablePairs counts the faces with two free tiles; with none left the game shows NO MOVES until a pair opens up.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_FreeTilesOnly free tiles can be selected: tiles with a tile on them or with both sides closed ignore clicks.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_NewGameAfter AppInit the layout lists match the task, every face is on exactly two tiles, nothing is removed or selected, and the start has a pair to remove.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_NewGameWithEnterEnter starts a new game at any time; a click in the same frame does nothing.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_PickinghoverTile is the tile under the mouse from the highest layer, with the rectangle edges exactly as in the task.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_RemoveAPairClicking a second free tile with the same face removes the pair and records it in history.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_ScreenFacesDifferent faces look different in the tiles' middle areas, and the same face looks the same.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: signatures[a] != signatures[b]
Context: faces 29 (tile 39) and 30 (tile 40) look the same
Values:
signatures[a] = 478892551765
signatures[b] = 478892551765 (line 1037) |
Failed | IN0 OUT0 |
| ✓ | Hidden_ScreenRemovedTilesA removed pair disappears from the screen: what is left there does not depend on the removed tiles' face, and undo draws them again.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ScreenSelectionAndHoverThe selected tile and a free tile under the mouse look different; a tile that is not free does not react to the mouse.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_SelectionA click on a free tile with another face moves the selection; clicks on blocked tiles or outside the pile keep it.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: selected == 12
Context: a tile that is not free keeps the selection: selected -1
Values:
selected = -1 (line 897) |
Failed | IN0 OUT0 |
| ✓ | Hidden_UndoU puts back the last removed pair and clears the selection; with an empty history it only clears the selection.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_WinningRemoving the last pair wins; U after the win puts that pair back.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |