Benchmark summary
The model builds a Super Speed-like top-down racing game with laps, wrenches and an upgrade shop in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Building Games in SC#: Super Speed-like racing
3 passed15 failed |
16.7% |
IN2 450 613 OUT80 506 | |
| ✓ | RacingThe model builds a top-down racing game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.3 AI queries
39 tool calls |
Passed | IN2 450 613 OUT80 506 |
| × | Hidden_AcceleratesUp adds accel to the speed every frame, up to maxSpeed and never above.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_Near(carSpeed, 0.5)
Context: after 10 frames carSpeed 0 (line 830) |
Failed | IN0 OUT0 |
| × | Hidden_CarsCollideThe player's car cannot drive into a computer car: it does not move and its speed is halved.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: carX == 139 && Hidden_Near(carSpeed, 0.9875)
Context: blocked by the computer car: carX 139, carSpeed 2
Values:
carX = 139 (line 1102) |
Failed | IN0 OUT0 |
| × | Hidden_ComputerCarsThe computer cars stay on road tiles, keep 10 px away from other cars, point in the direction they move, and each completes a lap in time.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: diff < 0.001 || Math.Abs(diff - 360) < 0.001
Context: frame 196: computer car 1 moved toward 0 degrees but aiAngle is 301.8675346566048
Values:
diff = 301.8675346566048 (line 1142) |
Failed | IN0 OUT0 |
| ✓ | Hidden_ComputerCarsAvoidThePlayerComputer cars do not drive into the player's car: they keep 10 px away from it.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_GrassOn grass the speed is limited to half of maxSpeed.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_Near(carSpeed, 1.5)
Context: on grass carSpeed 3 (line 901) |
Failed | IN0 OUT0 |
| ✓ | Hidden_InitialStateAfter AppInit the track is the given map, and the car, the computer cars and the counters have their starting values.0 AI queries
0 tool callsHiddenTest |
Passed | IN0 OUT0 |
| × | Hidden_LapsA lap counts only when the car passes checkpoints 1, 2 and 3 in this order and then the start line.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: nextCheckpoint == 2
Context: checkpoint 1: nextCheckpoint 1
Values:
nextCheckpoint = 1 (line 945) |
Failed | IN0 OUT0 |
| × | Hidden_MovesInItsDirectionThe car moves by carSpeed in the direction of carAngle.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_Near(carY, 129.975) && Hidden_Near(carX, 40)
Context: angle 90 moves down: (40, 128) (line 861) |
Failed | IN0 OUT0 |
| × | Hidden_NextRaceEnter in the shop starts the next race: the track and the cars start again, wrenches and upgrades stay.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: raceNumber == 2 && !raceOver && !inShop && laps == 0 && nextCheckpoint == 1 && place == 0
Context: the next race
Values:
raceNumber = 1
raceOver = false
inShop = false
laps = 2
nextCheckpoint = 0
place = 0 (line 1045) |
Failed | IN0 OUT0 |
| × | Hidden_PositionThe position counts the computer cars that passed more checkpoints than the player, and the screen shows it.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: position == 3
Context: progress: player 6, computer cars 7, 6 (farther from checkpoint 3) and 8, so position 3, not 1
Values:
position = 1 (line 1069) |
Failed | IN0 OUT0 |
| × | Hidden_RaceEndAfter the third lap the race is over: the place counts the computer cars that already have 3 laps, and the prize is added.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: laps == 3 && raceOver && inShop
Context: the race is over
Values:
laps = 2
raceOver = false
inShop = false (line 987) |
Failed | IN0 OUT0 |
| × | Hidden_ScreenShowsTheTrackThe screen shows the track tiles, the start line, the wrenches and the cars.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: carPixels >= 60 && carPixels <= 100
Context: the player's car has 48 pixels, a 12 x 8 rectangle has 96
Values:
carPixels = 48 (line 1192) |
Failed | IN0 OUT0 |
| × | Hidden_ShopIn the shop the keys 1, 2 and 3 buy upgrades for 2 * (level + 1) wrenches, the values grow, and level 3 is the last.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: inShop && place == 1
Context: won the race
Values:
inShop = false
place = 0 (line 1008) |
Failed | IN0 OUT0 |
| × | Hidden_SlowsDownWithout keys the car slows down by accel / 2 per frame, Down brakes by 2 * accel, and the speed never goes below 0.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_Near(carSpeed, 1.975)
Context: rolling: carSpeed 2 (line 844) |
Failed | IN0 OUT0 |
| × | Hidden_SteeringLeft and Right turn a moving car by turnRate per frame and keep the angle between 0 and 360; a car that stands still does not turn.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_Near(carAngle, 22)
Context: Right: carAngle 10 (line 883) |
Failed | IN0 OUT0 |
| × | Hidden_WallsA wall stops the car: it does not move and its speed becomes 0.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: Hidden_Near(carX, 33) && Hidden_Near(carY, 128) && Hidden_Near(carSpeed, 0.9875)
Context: head-on: (33, 128), speed 2 (line 917) |
Failed | IN0 OUT0 |
| × | Hidden_WrenchesDriving over a wrench picks it up: the tile becomes road and wrenches grows by 1.0 AI queries
0 tool callsHiddenTest Assertion expression evaluated to false: wrenches == 1 && track[2 * 20 + 5] == 0
Context: wrenches 0, tile 3
Values:
wrenches = 0 (line 967) |
Failed | IN0 OUT0 |