Qwen 3.8 27B NVFP4 Think: medium | Benchmark #240

Described in:Minecraft-like 3D game built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Minecraft-like 3D game built in SC# as an agent

ModelQwen 3.8 27B QuantizationNVFP4 ModeThink: medium

The model builds a Minecraft-like 3D block world with a polished look (pixel-art textures, hotbar, effects) in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 22/23 95.7% passed
22 passed1 failed23 total 3 AI queries 23 tool calls TOKENS IN1 458 174 TOKENS OUT73 762

Results by difficulty

1 group
Level Unknown / other
22 / 23 95.7%
passed
3 AI queries 23 tool calls IN1 458 174 OUT73 762

Results by category

1 group
Category Other
95.7% 22/23 passed
3 AI queries 23 tool calls TOKENS IN1 458 174 TOKENS OUT73 762

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Building Games in SC#: Minecraft-like block world (3D, pretty) 22 passed1 failed
95.7%
IN1 458 174 OUT73 762
✓
BlocksPrettyThe model builds a good-looking Minecraft-like 3D block game (textures, hotbar, effects) as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.3 AI queries 23 tool calls
Passed IN1 458 174 OUT73 762
✓
Hidden_BreakB breaks the targeted block, but not the bottom layer.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_FallAndLandA player in the air falls and lands exactly on the ground.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
×
Hidden_HeadHitsCeilingA block above the head stops a jump: the head stays just below it.0 AI queries 0 tool callsHiddenTest
Assertion expression evaluated to false: Hidden_Near(playerY, 1.2) && velocityY == 0 Context: player y 1.18, velocity -0.02 Values: velocityY = -0.02 (line 1043)
Failed IN0 OUT0
✓
Hidden_HudThe HUD shows the counters and the selected block type.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_JumpSpace jumps from the ground, the player goes up and lands again.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_JumpOntoStepA jump while walking gets the player onto a step one block high.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_LandOnHighestA fast fall into a stack lands on top of the highest block the box overlaps.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_MouseLookMoving the mouse right turns right and moving it up looks up; pitch keeps its limits.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_NoJumpInTheAirSpace in the air does not jump again: the jump never gets higher than one jump from the ground.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_PlaceP places the selected type in front of the targeted face, but never into the player.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_PlaceOnSlantedFacesA ray that reaches a block at a slant: the place cell is in front of the face the ray entered through (a side face when looking past the block, the top face when looking down at the ground), and P places the block there.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_RespawnFalling out of the world puts the player back at the start.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenShowsBlocksThe targeted block is on the screen in the middle, and looking up shows only the sky there.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_StartsStandingThe player starts in the middle on the grass and stays standing there.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_TargetThe block in front of the eye is the target, and the place cell is in front of the face looked at.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_TargetRangeAndDownBlocks further than 5 from the eye are not targeted; looking down targets the ground under the player.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_TurnAndLookLeft/Right turn by 3 degrees per frame, A/Z look up and down, and pitch stops at 89.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_WalkUp walks 0.15 per frame where the player looks, Down walks back, and turning changes the direction.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_WallStopsA wall two blocks high stops the player right in front of it.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_WallStopsAlongXA wall stops the player also when walking along X.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_WorldColumnsThe new world has the columns of the height formula: stone, two dirt cells, grass on top and air above.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_WorldEdgeThe player cannot leave the world at its edge.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0