Qwen 3.8 27B NVFP4 Instruct | Benchmark #249

Described in:Klondike solitaire built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Klondike solitaire game built in SC# as an agent

ModelQwen 3.8 27B QuantizationNVFP4 ModeInstruct

The model builds Klondike solitaire played with the mouse (drag and drop) with a polished look in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 10/16 62.5% passed
10 passed6 failed16 total 7 AI queries 28 tool calls TOKENS IN1 787 280 TOKENS OUT18 212

Results by difficulty

1 group
Level Unknown / other
10 / 16 62.5%
passed
7 AI queries 28 tool calls IN1 787 280 OUT18 212

Results by category

1 group
Category Other
62.5% 10/16 passed
7 AI queries 28 tool calls TOKENS IN1 787 280 TOKENS OUT18 212

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Building Games in SC#: Klondike solitaire 10 passed6 failed
62.5%
IN1 787 280 OUT18 212
✓
SolitaireThe model builds a mouse-driven Klondike solitaire like the classic Windows game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.7 AI queries 28 tool calls
Passed IN1 787 280 OUT18 212
✓
Hidden_DealDeal() lays out the deck in the exact order: columns 0 to 6 with 1 to 7 cards, only top cards face up, the rest in the stock with deck[28] on top.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
×
Hidden_DragARunA press on the uncovered strip of a covered card drags it with all cards above it; a press on the top card drags only that card.0 AI queries 0 tool callsHiddenTest
RemoveAt index 4 is out of range for a list of length 4. (line 226)
Failed IN0 OUT0
×
Hidden_DragOntoAColumnDragging a card onto a card one rank higher of the other color moves it; the uncovered face-down card turns face up.0 AI queries 0 tool callsHiddenTest
List index -9 is out of range. The list has 7 element(s). (line 288)
Failed IN0 OUT0
✓
Hidden_EnterStartsANewGameEnter starts a new game at any time; a mouse press in that frame does nothing, and a drag in progress is dropped.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
×
Hidden_FoundationCardBackThe top card of a foundation can be dragged back onto a column when the column rule allows it.0 AI queries 0 tool callsHiddenTest
List index -8 is out of range. The list has 7 element(s). (line 288)
Failed IN0 OUT0
×
Hidden_FoundationsFoundations take one card of their own suit in rank order from the Ace; other drops go back.0 AI queries 0 tool callsHiddenTest
Assertion expression evaluated to false: Hidden_Same(foundations[1], [Hidden_Card(1, 1), Hidden_Card(2, 1)]) && Hidden_Same(tableau[0], [Hidden_Card(5, 0)]) && hiddenCount[0] == 0 Context: the 2 of diamonds follows Values: hiddenCount[0] = 1 (line 1046)
Failed IN0 OUT0
✓
Hidden_IllegalDropsGoBackIllegal drops and drops outside every pile put the cards back and are not moves; only the mouse point decides the target.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
×
Hidden_KingOnAnEmptyColumnOnly a King, alone or with the cards above it, can be dropped on an empty column.0 AI queries 0 tool callsHiddenTest
RemoveAt index 2 is out of range for a list of length 2. (line 226)
Failed IN0 OUT0
✓
Hidden_NewGameDealsAllCardsA new game (AppInit) deals all 52 different cards: 28 in the columns as Deal() does and 24 in the stock.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_RightClickA right click sends the waste top card or a column's top card to its foundation when legal; otherwise nothing happens.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenDragFollowsTheMouseWhile dragging, the dragged card is drawn under the mouse at the pressed offset; after the drop the table looks as before.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenShowsCardsFace-up cards, face-down cards and the empty table look different.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_StockAndWasteA click on the stock turns its top card to the waste; on the empty stock it turns the whole waste back in order; each is a move.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
×
Hidden_StripEdgesA press picks the card whose uncovered strip contains the mouse, exactly at the strip edges.0 AI queries 0 tool callsHiddenTest
RemoveAt index 6 is out of range for a list of length 6. (line 226)
Failed IN0 OUT0
✓
Hidden_WinningThe last card on the foundations wins: won is true, "YOU WIN" is shown and the mouse does nothing more.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0