Ai4Users.comLocal AI & LLM community
Sign in to contributeSign in

Qwen 3.6 27B NVFP4 Instruct | Benchmark #223

Described in:Checkers game against the computer built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Checkers game against the computer built in SC# as an agent

ModelQwen 3.6 27B QuantizationNVFP4 ModeInstruct

The model builds a checkers game against the computer in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 14/15 93.3% passed
14 passed1 failed15 total 1 AI query 8 tool calls TOKENS IN284 420 TOKENS OUT8 752

Results by difficulty

1 group
Level Unknown / other
14 / 15 93.3%
passed
1 AI query 8 tool calls IN284 420 OUT8 752

Results by category

1 group
Category Other
93.3% 14/15 passed
1 AI query 8 tool calls TOKENS IN284 420 TOKENS OUT8 752

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Building Games in SC#: Checkers 14 passed1 failed
93.3%
 IN284 420 OUT8 752
✓
CheckersThe model builds a checkers game with an arrow-key cursor against the computer from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query 8 tool calls
Play in browser
 Passed IN284 420 OUT8 752
✓
Hidden_ComputerMustCaptureThe computer always plays a legal complete turn, also when it must capture or continue a multi-jump.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_ComputerPlaysLegallyIn a played game every turn of the computer is one of the legal complete turns.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_CursorMovesThe arrows move the cursor by one square and stop it at the edges of the board.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_DrawForty quiet turns in a row (only kings moved, nothing captured) end the game in a draw.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_GameOverAndRestartWhile the game is over only Enter works, and it starts a new game in a frame that does nothing else.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_InitialStateAfter AppInit the pieces stand on the dark squares of rows 0-2 and 5-7, and the cursor, the selection and the counters have their starting values.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_KingsKings move one square in all four directions and never fly over empty squares.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_MandatoryCaptureCaptures are mandatory: a piece that cannot capture cannot be selected while another one can; the jumped piece is removed.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_MultiJumpA man may capture backward, and a multi-jump must be continued with the same piece; Escape and other pieces are ignored meanwhile.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_PromotionA man that reaches the last row becomes a king; a promotion by a capture ends the turn.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_ScreenThe screen shows the squares, the pieces, the kings, the selection, the cursor and the texts.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_SelectAndMoveSpace selects a movable piece and moves it to a legal square; then the computer plays its turn in the next frame.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
×
Hidden_SelectionRulesOnly pieces with a legal step can be selected; an illegal target keeps the selection; Escape clears it.0 AI queries 0 tool callsHiddenTest
Assertion expression evaluated to false: selX == 2 && selY == 5 Context: another movable piece takes over the selection Values: selX = 4 selY = 5 (line 1051)
 Failed IN0 OUT0
✓
Hidden_WinAndLoseThe side without pieces loses, and the side without a legal step loses.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0