Benchmark summary
The model builds a checkers game against the computer in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Checkers
12 passed3 failed |
80.0% |
IN1 539 639 OUT35 031 | |
| ✓ | CheckersThe model builds a checkers game with an arrow-key cursor against the computer from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
31 tool calls |
Passed | IN1 539 639 OUT35 031 |
| ✓ | Hidden_ComputerMustCaptureThe computer always plays a legal complete turn, also when it must capture or continue a multi-jump.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_ComputerPlaysLegallyIn a played game every turn of the computer is one of the legal complete turns.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: hiddenStep.Count == 4
Context: the player has a legal step
Values:
hiddenStep.Count = 0 (line 1479) |
Failed | IN0 OUT0 |
| ✓ | Hidden_CursorMovesThe arrows move the cursor by one square and stop it at the edges of the board.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_DrawForty quiet turns in a row (only kings moved, nothing captured) end the game in a draw.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_GameOverAndRestartWhile the game is over only Enter works, and it starts a new game in a frame that does nothing else.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_InitialStateAfter AppInit the pieces stand on the dark squares of rows 0-2 and 5-7, and the cursor, the selection and the counters have their starting values.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_KingsKings move one square in all four directions and never fly over empty squares.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_MandatoryCaptureCaptures are mandatory: a piece that cannot capture cannot be selected while another one can; the jumped piece is removed.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_MultiJumpA man may capture backward, and a multi-jump must be continued with the same piece; Escape and other pieces are ignored meanwhile.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: !jumping && turn == 1 && moveNumber == 1 && selX == -1
Context: the turn ended after the last jump
Values:
jumping = true
turn = 0
moveNumber = 0
selX = 5 (line 1403) |
Failed | IN0 OUT0 |
| ✓ | Hidden_PromotionA man that reaches the last row becomes a king; a promotion by a capture ends the turn.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ScreenThe screen shows the squares, the pieces, the kings, the selection, the cursor and the texts.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_SelectAndMoveSpace selects a movable piece and moves it to a legal square; then the computer plays its turn in the next frame.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_SelectionRulesOnly pieces with a legal step can be selected; an illegal target keeps the selection; Escape clears it.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: selX == 2 && selY == 5
Context: another movable piece takes over the selection
Values:
selX = 4
selY = 5 (line 1338) |
Failed | IN0 OUT0 |
| ✓ | Hidden_WinAndLoseThe side without pieces loses, and the side without a legal step loses.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |