Benchmark summary
The model builds a keyboard app with a main menu, options, high scores, a pause menu and a name entry around a small game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Game menus and screen transitions
15 failed1 broken |
0.0% |
IN217 166 OUT150 278 | |
| ! | MenusAndScreensThe model builds an app with nested keyboard menus and screen transitions (around a small Snake game) from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
5 tool calls Assertion expression evaluated to false: app.Built
Context: AI: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.
App: not submitted; the last file version is graded (end: AI error: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.)
Work: 1 AI calls (6 model rounds with tool calls), 1 whole-file writes, 4 edits (521 lines), 0 CheckApp calls, 0 refused rewrites, 0 refused repeated edits, 0 rejected submissions, 0 reminders to hand in
Compile errors: (100,11) Expected a variable name, found '['.
Values:
app.Built = false |
Broken | IN217 166 OUT150 278 |
| × | Hidden_ControlsA direction takes a new key; a key used by another direction is rejected; Escape cancels; Reset to default is skipped while disabled.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_DifficultyThe difficulty screen opens on the current difficulty; choosing one returns to options, and the game uses its speed.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameBasicsThe game steps every stepFrames frames, grows on food, and a game that ends with points opens the name entry.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_GameScreenThe game screen shows the snake, its head and the food on their cells.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_HighScoreListHigh scores stay sorted, a new equal score goes after the old one, only the best 5 stay, and a score below them is not entered.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_InitialStateAfter AppInit the main menu is open with Start selected, and the settings and high scores have their starting values.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_MainMenuNavigationUp and Down move the selection with wrap-around; Escape on the main menu does nothing.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_NameEntryThe name entry changes letters with Up and Down (with wrap-around) and moves with Left and Right; Enter saves the score.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_OptionsAndBackOptions opens with its first item selected; Escape and Back return to the main menu with Options selected.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PauseAndResumeP opens the pause menu, nothing moves while paused, and Resume continues the game exactly where it was.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_PauseOptionsAndQuitOptions opened from the pause menu return to the pause menu; Quit to menu ends the game without a high score.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_QuitQuit shows GOODBYE and AppLoop returns false from then on.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_RemappedSteeringThe snake steers with the keys from controlKeys only.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_ResetConfirmationReset asks for confirmation with No selected; No keeps the scores, Yes deletes them and leaves only Back enabled.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |
| × | Hidden_VolumeLeft and Right change the volume within 0 to 10 only while Volume is selected; Back returns to options.0 AI queries
0 tool callsHiddenTest Not run: the app does not compile. |
Failed | IN0 OUT0 |