Qwen 3.8 27B NVFP4 Think: medium | Benchmark #176

Described in:Minesweeper-like game built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Minesweeper-like game built in SC# as an agent

ModelQwen 3.8 27B QuantizationNVFP4 ModeThink: medium

The model builds a Minesweeper-like mine field puzzle in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 16/16 100.0% passed
16 passed16 total 1 AI query 16 tool calls TOKENS IN601 293 TOKENS OUT89 817

Results by difficulty

1 group
Level Unknown / other
16 / 16 100.0%
passed
1 AI query 16 tool calls IN601 293 OUT89 817

Results by category

1 group
Category Other
100.0% 16/16 passed
1 AI query 16 tool calls TOKENS IN601 293 TOKENS OUT89 817

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Building Games in SC#: Minesweeper-like mine field puzzle 16 passed
100.0%
IN601 293 OUT89 817
✓
MinesweeperThe model builds a keyboard Minesweeper game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query 16 tool calls
Passed IN601 293 OUT89 817
✓
Hidden_ChordSpace on a number with exactly that many flagged neighbors reveals its other neighbors; with fewer flags nothing happens.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ChordWithAWrongFlagA chord with a wrong flag reveals a mine and loses the game; the neighbors are revealed in row order up to that mine.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_CursorMovesEach arrow press moves the cursor one cell, and the cursor stops at the edges.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FirstRevealIsSafeThe first reveal places exactly 40 mines, none on the revealed cell or its neighbors, so it opens an empty area.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FlagsF flags and unflags a hidden cell, does nothing on a revealed cell, and the counter shows 40 minus the flags, also below 0.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FloodRevealAn empty cell opens the whole empty area up to the numbers around it, exactly by the reveal rule.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FloodStopsAtFlagsFlagged cells are not opened by the flood, and they do not open anything.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_InitialStateAfter AppInit every cell is hidden, there are no mines yet, and the cursor, flags and timer have their starting values.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_LosingRevealing a mine loses the game: every mine is shown, the timer stops, and keys do nothing until Enter.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_RestartEnter starts a new game at any time; that frame does nothing else.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_RevealANumberRevealing a cell with a number opens only that cell; mines are not moved when a test placed them.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenAfterALossAfter a loss the screen shows the mines and the exploded one in its own color, and no cursor.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenShowsTheBoardThe screen shows hidden cells, revealed cells, a flag and the cursor at their places.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_TimerThe timer starts with the first reveal and counts frames; the screen shows whole seconds.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_WinningRevealing the last cell without a mine wins: the mines get flags, the counter shows 0 and the timer stops.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0