Benchmark summary
The model builds a checkers game against the computer in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Checkers
15 failed |
0.0% |
IN870 201 OUT66 634 | |
| × | CheckersThe model builds a checkers game with an arrow-key cursor against the computer from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.6 AI queries
10 tool calls Assertion expression evaluated to false: app.Built
Context: AI: completed
App: not submitted; the last file version is graded (end: stopped: no tool call in 3 answers in a row)
Work: 6 AI calls (16 model rounds with tool calls), 1 whole-file writes, 7 edits (1128 lines), 1 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 1 rejected submissions, 5 reminders to hand in
Compile errors: (491,49) Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
app.Built = false |
Failed | IN870 201 OUT66 634 |
| × | Hidden_ComputerMustCaptureThe computer always plays a legal complete turn, also when it must capture or continue a multi-jump.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ComputerPlaysLegallyIn a played game every turn of the computer is one of the legal complete turns.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_CursorMovesThe arrows move the cursor by one square and stop it at the edges of the board.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_DrawForty quiet turns in a row (only kings moved, nothing captured) end the game in a draw.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameOverAndRestartWhile the game is over only Enter works, and it starts a new game in a frame that does nothing else.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the pieces stand on the dark squares of rows 0-2 and 5-7, and the cursor, the selection and the counters have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_KingsKings move one square in all four directions and never fly over empty squares.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_MandatoryCaptureCaptures are mandatory: a piece that cannot capture cannot be selected while another one can; the jumped piece is removed.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_MultiJumpA man may capture backward, and a multi-jump must be continued with the same piece; Escape and other pieces are ignored meanwhile.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PromotionA man that reaches the last row becomes a king; a promotion by a capture ends the turn.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenThe screen shows the squares, the pieces, the kings, the selection, the cursor and the texts.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_SelectAndMoveSpace selects a movable piece and moves it to a legal square; then the computer plays its turn in the next frame.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_SelectionRulesOnly pieces with a legal step can be selected; an illegal target keeps the selection; Escape clears it.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_WinAndLoseThe side without pieces loses, and the side without a legal step loses.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |