Qwen 3.8 27B NVFP4 Instruct | Benchmark #139

Described in:Snake game built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Snake game built in SC# as an agent

ModelQwen 3.8 27B QuantizationNVFP4 ModeInstruct

The model builds the classic Snake game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 15/15 100.0% passed
15 passed15 total 1 AI query 20 tool calls TOKENS IN542 692 TOKENS OUT7 166

Results by difficulty

1 group
Level Unknown / other
15 / 15 100.0%
passed
1 AI query 20 tool calls IN542 692 OUT7 166

Results by category

1 group
Category Other
100.0% 15/15 passed
1 AI query 20 tool calls TOKENS IN542 692 TOKENS OUT7 166

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Building Games in SC#: Snake 15 passed
100.0%
IN542 692 OUT7 166
✓
SnakeThe model builds a Snake game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query 20 tool calls
Passed IN542 692 OUT7 166
✓
Hidden_ArrowKeysTurnAn arrow key turns the snake, and the next step goes in the new direction.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_BodyEndsTheGameStepping onto the snake's body ends the game.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_EatingGrowsEating the food makes the snake one cell longer, adds 10 points and places new food on a free cell.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FoodNeverOnTheSnakeNew food always appears on a board cell that is not part of the snake.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_GameOverAndRestartWhile the game is over nothing moves and arrow keys are ignored; Enter starts a new game.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_GetsFasterThe snake gets faster: stepFrames never grows, is at most 4 after 10 foods, never below 2, and the snake really steps every stepFrames frames.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_HeadMayFollowTheTailThe head may step onto the tail's cell, because the tail leaves it in the same step.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_InitialStateAfter AppInit, the snake, the direction, the speed, the score and the food have their starting values.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_KeyInTheStepFrameA key pressed in the frame of a step already turns that step.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_MovesEveryStepFramesThe snake takes its first step in the 6th frame and then one step every 6 frames, and the body follows the head.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ReverseIsIgnoredThe key opposite to the last step is ignored, also after another turn before the next step.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenAfterEatingAfter eating and after the eating effect ends, the screen shows the longer snake and the new score.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenShowsTheSceneThe screen shows the background, the food, the body, the head and the score text.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_WallEndsTheGameStepping out of the board ends the game, and the snake stays where it was.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0