Ai4Users.comLocal AI & LLM community
Sign in to contributeSign in

Qwen 3.8 27B NVFP4 Think: medium | Benchmark #178

Described in:Checkers game against the computer built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Checkers game against the computer built in SC# as an agent

ModelQwen 3.8 27B QuantizationNVFP4 ModeThink: medium

The model builds a checkers game against the computer in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 14/15 93.3% passed
14 passed1 failed15 total 1 AI query 23 tool calls TOKENS IN1 063 301 TOKENS OUT74 951

Results by difficulty

1 group
Level Unknown / other
14 / 15 93.3%
passed
1 AI query 23 tool calls IN1 063 301 OUT74 951

Results by category

1 group
Category Other
93.3% 14/15 passed
1 AI query 23 tool calls TOKENS IN1 063 301 TOKENS OUT74 951

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Building Games in SC#: Checkers 14 passed1 failed
93.3%
 IN1 063 301 OUT74 951
✓
CheckersThe model builds a checkers game with an arrow-key cursor against the computer from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query 23 tool calls
Play in browser
 Passed IN1 063 301 OUT74 951
✓
Hidden_ComputerMustCaptureThe computer always plays a legal complete turn, also when it must capture or continue a multi-jump.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_ComputerPlaysLegallyIn a played game every turn of the computer is one of the legal complete turns.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_CursorMovesThe arrows move the cursor by one square and stop it at the edges of the board.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_DrawForty quiet turns in a row (only kings moved, nothing captured) end the game in a draw.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_GameOverAndRestartWhile the game is over only Enter works, and it starts a new game in a frame that does nothing else.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_InitialStateAfter AppInit the pieces stand on the dark squares of rows 0-2 and 5-7, and the cursor, the selection and the counters have their starting values.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_KingsKings move one square in all four directions and never fly over empty squares.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_MandatoryCaptureCaptures are mandatory: a piece that cannot capture cannot be selected while another one can; the jumped piece is removed.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_MultiJumpA man may capture backward, and a multi-jump must be continued with the same piece; Escape and other pieces are ignored meanwhile.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_PromotionA man that reaches the last row becomes a king; a promotion by a capture ends the turn.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_ScreenThe screen shows the squares, the pieces, the kings, the selection, the cursor and the texts.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
✓
Hidden_SelectAndMoveSpace selects a movable piece and moves it to a legal square; then the computer plays its turn in the next frame.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0
×
Hidden_SelectionRulesOnly pieces with a legal step can be selected; an illegal target keeps the selection; Escape clears it.0 AI queries 0 tool callsHiddenTest
Assertion expression evaluated to false: selX == 2 && selY == 5 Context: another movable piece takes over the selection Values: selX = 4 selY = 5 (line 1154)
 Failed IN0 OUT0
✓
Hidden_WinAndLoseThe side without pieces loses, and the side without a legal step loses.0 AI queries 0 tool callsHiddenTest
 Passed IN0 OUT0