Qwen 3.6 27B NVFP4 Instruct | Benchmark #187

Described in:Space Invaders-like game built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Space Invaders-like game built in SC# as an agent

ModelQwen 3.6 27B QuantizationNVFP4 ModeInstruct

The model builds a Space Invaders-like fixed shooter game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 17/17 100.0% passed
17 passed17 total 1 AI query 9 tool calls TOKENS IN294 060 TOKENS OUT8 549

Results by difficulty

1 group
Level Unknown / other
17 / 17 100.0%
passed
1 AI query 9 tool calls IN294 060 OUT8 549

Results by category

1 group
Category Other
100.0% 17/17 passed
1 AI query 9 tool calls TOKENS IN294 060 TOKENS OUT8 549

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Building Games in SC#: Space Invaders-like fixed shooter 17 passed
100.0%
IN294 060 OUT8 549
✓
SpaceInvadersThe model builds a Space Invaders game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query 9 tool calls
Passed IN294 060 OUT8 549
✓
Hidden_AliensDropBombsAliens drop bombs from the bottom center of the lowest live alien of a column, at least once every 90 frames, and never more than 3 at once.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_BombFallsSteadilyA bomb falls at a constant speed of 2 px to 4 px per frame and disappears below the screen.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_BombHitsTheCannonA bomb that reaches the cannon is removed and costs a life; with the last life the game is over.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_CannonMovesAndStopsHolding Left or Right moves the cannon by 3 px per frame and stops it at both edges.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_EdgesCountLiveAliensOnlyOnly live aliens count for the edges: without the outer columns, the formation goes further before it turns.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FormationGetsFasterThe formation gets faster: moveFrames never grows, is at most 10 with 15 aliens left, never below 2, and the formation really moves every moveFrames frames.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FormationMovesThe formation moves by 4 px in the 20th frame and then every 20 frames.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FormationTurnsAtTheEdgeAt the right edge the formation moves down by 8 px and turns, and then moves left.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_InitialStateAfter AppInit, the cannon, the formation, the speed, the score, the lives and the shot have their starting values.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_InvasionEndsTheGameWhen the lowest live alien reaches y 210, the game is over; dead aliens do not count.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_PointsByRowAliens in row 0 give 30 points, in rows 1 and 2 20 points and in rows 3 and 4 10 points.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenAfterAKillAfter an alien dies and the explosion effect ends, the screen shows one alien less and the new score.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenShowsTheSceneThe screen shows the aliens in their row colors, the cannon, the background and the texts.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ShotFliesAndLeavesSpace fires one shot from the middle of the cannon; it moves up by 6 px per frame, a second Space does nothing while it flies, and it disappears at the top.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ShotKillsTheLowestAlienA shot from below hits the lowest live alien of its column, kills it, adds its points and disappears.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_WinningAndRestartKilling the last alien wins the game and stops it; Enter starts a new game.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0