Ai4Users - Qwen 3.8 27B NVFP4 Instruct

Described in:Pac-Man-like game built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Pac-Man-like game built in SC# as an agent

ModelQwen 3.8 27B QuantizationNVFP4 ModeInstruct

The model builds a Pac-Man-like maze chase game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 13/18 72.2% passed

13 passed5 failed18 total 1 AI query 49 tool calls TOKENS IN2 619 892 TOKENS OUT20 261

Results by difficulty

1 group

Level Unknown / other

13 / 18 72.2%

passed

1 AI query 49 tool calls IN2 619 892 OUT20 261

Results by category

1 group

Category Other

72.2% 13/18 passed

1 AI query 49 tool calls TOKENS IN2 619 892 TOKENS OUT20 261

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test		Status	Tokens
Building Games in SC#: Pac-Man-like maze chase 13 passed5 failed		72.2%	IN2 619 892 OUT20 261
✓	PacManThe model builds a Pac-Man game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query 49 tool calls Play in browser	Passed	IN2 619 892 OUT20 261
×	Hidden_ChaseReachesPacMan01In chase mode ghosts 0 and 1 reach Pac-Man when he stands still, and he loses a life.0 AI queries 0 tool callsHiddenTest Assertion expression evaluated to false: pacX == 9 && pacY == 15 && pacDir == 3 && ghostX[0] == 9 && ghostY[0] == 7 && ghostState[1] == 3 && ghostWait[3] == 270 && tick == 0 && modeClock == 0 Context: the start positions after the lost life Values: pacX = 9 pacY = 15 pacDir = 3 ghostX[0] = 9 ghostY[0] = 7 ghostState[1] = 3 ghostWait[3] = 269 tick = 0 modeClock = 0 (line 1200)	Failed	IN0 OUT0
×	Hidden_ChaseReachesPacMan23In chase mode ghosts 2 and 3 reach Pac-Man when he stands still, and he loses a life.0 AI queries 0 tool callsHiddenTest Assertion expression evaluated to false: pacX == 9 && pacY == 15 && pacDir == 3 && ghostX[0] == 9 && ghostY[0] == 7 && ghostState[1] == 3 && ghostWait[3] == 270 && tick == 0 && modeClock == 0 Context: the start positions after the lost life Values: pacX = 9 pacY = 15 pacDir = 3 ghostX[0] = 9 ghostY[0] = 7 ghostState[1] = 3 ghostWait[3] = 269 tick = 0 modeClock = 0 (line 1200)	Failed	IN0 OUT0
✓	Hidden_EatingGhostsEating frightened ghosts with one pill gives 200, 400, 800 and 1600 points; a new pill starts again at 200.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_EyesGoBackToTheHouseAn eaten ghost goes back as eyes along a shortest path, one step every 2 frames, waits 60 frames in the house and leaves it normal.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_GameOverAndRestartWhile the game is over nothing changes; Enter starts a new game with a full maze.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_GhostsLeaveTheHouseGhosts leave the house at frames 30, 150 and 270 of the level.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_GhostSpeedAndMovesFrightened ghosts step every 8 frames, normal ghosts every 5 frames; ghosts never step through walls or back.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_InitialStateAfter AppInit the maze is the given map and Pac-Man, the ghosts and the counters have their start values.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
×	Hidden_LevelCompleteWhen the last dot is eaten, the level is complete: the next level starts with a full maze and the score stays.0 AI queries 0 tool callsHiddenTest Assertion expression evaluated to false: pacX == 9 && pacY == 15 && pacDir == 3 && tick == 0 && ghostX[0] == 9 && ghostY[0] == 7 && ghostState[2] == 3 Context: the start positions Values: pacX = 9 pacY = 15 pacDir = 3 tick = 0 ghostX[0] = 8 ghostY[0] = 7 ghostState[2] = 3 (line 1445)	Failed	IN0 OUT0
×	Hidden_LosingALifeA normal ghost on Pac-Man's cell takes a life and everything goes to the start positions; the dots stay eaten.0 AI queries 0 tool callsHiddenTest Assertion expression evaluated to false: lives == 2 && !gameOver Context: lives 3 Values: lives = 3 gameOver = false (line 1379)	Failed	IN0 OUT0
×	Hidden_ModeScheduleThe mode follows the schedule of modeClock, and a change of mode makes the normal ghosts turn back.0 AI queries 0 tool callsHiddenTest Assertion expression evaluated to false: ghostDir[0] == 3 && ghostDir[1] == 0 Context: both ghosts reversed: ghostDir 3 and 3 Values: ghostDir[0] = 3 ghostDir[1] = 3 (line 1181)	Failed	IN0 OUT0
✓	Hidden_PacManMovesAndEatsPac-Man steps one cell every 4 frames, eats the dots on his way (10 points each) and stops at a wall.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_PowerPillA power pill gives 50 points, frightens the ghosts and makes them turn back; frightened mode ends after 180 frames.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_ScatterGoesToCornersIn scatter mode every ghost reaches its corner cell.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_ScreenShowsTheMazeThe screen shows the walls, the door, the dots, the pills, Pac-Man, the ghosts and the texts.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_TunnelAndTurningBackPac-Man turns back right away at his next step and goes through the tunnel from one side to the other.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0
✓	Hidden_TurnBufferingA pressed direction is remembered and taken at the first step where its cell is open.0 AI queries 0 tool callsHiddenTest	Passed	IN0 OUT0

Qwen 3.8 27B NVFP4 Instruct | Benchmark #179

Pac-Man-like game built in SC# as an agent

Results by difficulty

Results by category