Benchmark summary
The model builds Mahjong solitaire (72 tiles in 3 layers) played with the mouse with a polished look in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Mahjong solitaire
10 passed3 failed |
76.9% |
IN2 353 015 OUT16 695 | |
| ✓ | MahjongThe model builds a Mahjong solitaire game played with the mouse from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.7 AI queries
32 tool calls |
Passed | IN2 353 015 OUT16 695 |
| ✓ | Hidden_AvailablePairsavailablePairs counts the faces with two free tiles; with none left the game shows NO MOVES until a pair opens up.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_FreeTilesOnly free tiles can be selected: tiles with a tile on them or with both sides closed ignore clicks.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_NewGameAfter AppInit the layout lists match the task, every face is on exactly two tiles, nothing is removed or selected, and the start has a pair to remove.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: tileColumn.Count == 72 && tileRow.Count == 72 && tileLayer.Count == 72 && faces.Count == 72 && removed.Count == 72
Context: 72 tiles
Values:
tileColumn.Count = 72
tileRow.Count = 72
tileLayer.Count = 72
faces.Count = 72
removed.Count = 144 (line 697) |
Failed | IN0 OUT0 |
| × | Hidden_NewGameWithEnterEnter starts a new game at any time; a click in the same frame does nothing.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_Remaining() == 72 && history.Count == 0 && selected == -1 && !won
Context: a new game and no selection: selected -1
Values:
history.Count = 0
selected = -1
won = false (line 930) |
Failed | IN0 OUT0 |
| ✓ | Hidden_PickinghoverTile is the tile under the mouse from the highest layer, with the rectangle edges exactly as in the task.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_RemoveAPairClicking a second free tile with the same face removes the pair and records it in history.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_ScreenFacesDifferent faces look different in the tiles' middle areas, and the same face looks the same.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: signatures[a] != signatures[b]
Context: faces 9 (tile 23) and 10 (tile 24) look the same
Values:
signatures[a] = 820896267765
signatures[b] = 820896267765 (line 976) |
Failed | IN0 OUT0 |
| ✓ | Hidden_ScreenRemovedTilesA removed pair disappears from the screen: what is left there does not depend on the removed tiles' face, and undo draws them again.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ScreenSelectionAndHoverThe selected tile and a free tile under the mouse look different; a tile that is not free does not react to the mouse.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_SelectionA click on a free tile with another face moves the selection; clicks on blocked tiles or outside the pile keep it.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_UndoU puts back the last removed pair and clears the selection; with an empty history it only clears the selection.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_WinningRemoving the last pair wins; U after the win puts that pair back.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |