Level
Unknown / other
0 / 18
0.0%
passed1 AI query
0 tool calls
IN0
OUT0
Benchmark summary
Pac-Man-like game built in SC# as an agent
ModelQwen 3.6 27B
QuantizationNVFP4
ModeThinking
The model builds a Pac-Man-like maze chase game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Score
0/18
0.0% passed
0 passed18 failed18 total
1 AI query
0 tool calls
TOKENS IN0
TOKENS OUT0
Results by difficulty
1 groupResults by category
1 groupCategory
Other
0.0%
0/18 passed
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Pac-Man-like maze chase
18 failed |
0.0% |
IN0 OUT0 | |
| × | PacManThe model builds a Pac-Man game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
0 tool calls Assertion expression evaluated to false: app.Built
Context: AI: AiRunner request failed with HTTP 404 (NotFound).
App: none handed in (end: AI error: AiRunner request failed with HTTP 404 (NotFound).)
Work: 1 AI calls (0 model rounds with tool calls), 0 whole-file writes, 0 edits (0 lines), 0 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 0 reminders to hand in
Values:
app.Built = false |
Failed | IN0 OUT0 |
| × | Hidden_ChaseReachesPacMan01In chase mode ghosts 0 and 1 reach Pac-Man when he stands still, and he loses a life.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ChaseReachesPacMan23In chase mode ghosts 2 and 3 reach Pac-Man when he stands still, and he loses a life.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_EatingGhostsEating frightened ghosts with one pill gives 200, 400, 800 and 1600 points; a new pill starts again at 200.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_EyesGoBackToTheHouseAn eaten ghost goes back as eyes along a shortest path, one step every 2 frames, waits 60 frames in the house and leaves it normal.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_GameOverAndRestartWhile the game is over nothing changes; Enter starts a new game with a full maze.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_GhostsLeaveTheHouseGhosts leave the house at frames 30, 150 and 270 of the level.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_GhostSpeedAndMovesFrightened ghosts step every 8 frames, normal ghosts every 5 frames; ghosts never step through walls or back.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the maze is the given map and Pac-Man, the ghosts and the counters have their start values.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_LevelCompleteWhen the last dot is eaten, the level is complete: the next level starts with a full maze and the score stays.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_LosingALifeA normal ghost on Pac-Man's cell takes a life and everything goes to the start positions; the dots stay eaten.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ModeScheduleThe mode follows the schedule of modeClock, and a change of mode makes the normal ghosts turn back.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_PacManMovesAndEatsPac-Man steps one cell every 4 frames, eats the dots on his way (10 points each) and stops at a wall.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_PowerPillA power pill gives 50 points, frightens the ghosts and makes them turn back; frightened mode ends after 180 frames.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ScatterGoesToCornersIn scatter mode every ghost reaches its corner cell.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheMazeThe screen shows the walls, the door, the dots, the pills, Pac-Man, the ghosts and the texts.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_TunnelAndTurningBackPac-Man turns back right away at his next step and goes through the tunnel from one side to the other.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |
| × | Hidden_TurnBufferingA pressed direction is remembered and taken at the first step where its cell is open.0 AI queries
0 tool callsHiddenTest Not run: no app was handed in. |
Failed | IN0 OUT0 |