Benchmark summary
The model builds a Pac-Man-like maze chase game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Pac-Man-like maze chase
18 failed |
0.0% |
IN268 765 OUT46 390 | |
| × | PacManThe model builds a Pac-Man game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.4 AI queries
2 tool calls Assertion expression evaluated to false: app.Built
Context: AI: completed
App: not submitted; the last file version is graded (end: stopped: no tool call in 3 answers in a row)
Work: 4 AI calls (6 model rounds with tool calls), 1 whole-file writes, 0 edits (0 lines), 1 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 3 reminders to hand in
Compile errors: (52,11) Expected a variable name, found '['.
Values:
app.Built = false |
Failed | IN268 765 OUT46 390 |
| × | Hidden_ChaseReachesPacMan01In chase mode ghosts 0 and 1 reach Pac-Man when he stands still, and he loses a life.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ChaseReachesPacMan23In chase mode ghosts 2 and 3 reach Pac-Man when he stands still, and he loses a life.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EatingGhostsEating frightened ghosts with one pill gives 200, 400, 800 and 1600 points; a new pill starts again at 200.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_EyesGoBackToTheHouseAn eaten ghost goes back as eyes along a shortest path, one step every 2 frames, waits 60 frames in the house and leaves it normal.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameOverAndRestartWhile the game is over nothing changes; Enter starts a new game with a full maze.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GhostsLeaveTheHouseGhosts leave the house at frames 30, 150 and 270 of the level.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GhostSpeedAndMovesFrightened ghosts step every 8 frames, normal ghosts every 5 frames; ghosts never step through walls or back.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the maze is the given map and Pac-Man, the ghosts and the counters have their start values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_LevelCompleteWhen the last dot is eaten, the level is complete: the next level starts with a full maze and the score stays.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_LosingALifeA normal ghost on Pac-Man's cell takes a life and everything goes to the start positions; the dots stay eaten.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ModeScheduleThe mode follows the schedule of modeClock, and a change of mode makes the normal ghosts turn back.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PacManMovesAndEatsPac-Man steps one cell every 4 frames, eats the dots on his way (10 points each) and stops at a wall.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PowerPillA power pill gives 50 points, frightens the ghosts and makes them turn back; frightened mode ends after 180 frames.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScatterGoesToCornersIn scatter mode every ghost reaches its corner cell.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheMazeThe screen shows the walls, the door, the dots, the pills, Pac-Man, the ghosts and the texts.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_TunnelAndTurningBackPac-Man turns back right away at his next step and goes through the tunnel from one side to the other.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_TurnBufferingA pressed direction is remembered and taken at the first step where its cell is open.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |