Benchmark summary
The model builds a Mario-like scrolling platformer in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Mario-like platformer
13 passed2 failed |
86.7% |
IN3 641 703 OUT55 027 | |
| ✓ | PlatformerThe model builds a scrolling platform game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query
49 tool calls |
Passed | IN3 641 703 OUT55 027 |
| ✓ | Hidden_BricksA big player breaks a brick from below for 50 points; a small player only bumps it.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_CameraThe camera follows the player's middle minus 160, never moves back, stops at 640, and the player cannot walk left out of the view.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_CoinBlockJumping into a question block from below uses it: a coin block gives a coin and 200 points.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_CoinsTouching a coin collects it for 200 points.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_EnemiesWalkEnemies walk at a constant speed of 0.5 to 1 px per frame, turn at walls and at the edge of a pit, and never fall or enter solid tiles.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: left && right
Context: the enemy turned around
Values:
left = false
right = false (line 1336) |
Failed | IN0 OUT0 |
| ✓ | Hidden_EnemyHitsTouching an enemy from the side costs a small player a life and makes a big player small for 60 safe frames.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_FlagTouching the flag wins: the game stops, and Enter starts a new game.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_InitialStateAfter AppInit the level is the given map, the coins and enemies come from it, and the player and counters have their starting values.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_JumpingA jump starts with -7, gravity adds 0.5 per frame; holding Up gives the full jump, releasing it early a lower one.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_MushroomA mushroom block lets a mushroom out on top; the mushroom moves at a constant speed and falls to the ground without entering solid tiles; eating it makes the player big.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: landed
Context: the mushroom fell to the ground
Values:
landed = false (line 1276) |
Failed | IN0 OUT0 |
| ✓ | Hidden_PitsAndGameOverFalling into a pit costs a life; with the last life the game is over, and Enter starts a new game.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_ScreenThe screen shows the tiles, the enemies, the player and the coins at their positions relative to the camera.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_StompFalling onto an enemy stomps it: it is removed, 100 points, and the player bounces up with -4.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| ✓ | Hidden_WalkingLeft and Right move the player by 2 px per frame; walls stop the player at their edge.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |