Qwen 3.8 27B NVFP4 Think: medium | Benchmark #146

Described in:Tetris-like game built by Qwen 3.6 and Qwen 3.8 as agents

Benchmark summary

Tetris-like game built in SC# as an agent

ModelQwen 3.8 27B QuantizationNVFP4 ModeThink: medium

The model builds a Tetris-like falling blocks game in SC# as an agent: it writes the app, compiles it, runs its own tests and fixes the code. Hidden tests check the rules from the task; each hidden test is its own result.

Score 17/18 94.4% passed
17 passed1 failed18 total 1 AI query 14 tool calls TOKENS IN511 072 TOKENS OUT47 287

Results by difficulty

1 group
Level Unknown / other
17 / 18 94.4%
passed
1 AI query 14 tool calls IN511 072 OUT47 287

Results by category

1 group
Category Other
94.4% 17/18 passed
1 AI query 14 tool calls TOKENS IN511 072 TOKENS OUT47 287

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Building Games in SC#: Tetris-like falling blocks 17 passed1 failed
94.4%
IN511 072 OUT47 287
✓
TetrisThe model builds a Tetris game from the task as an agent; this item passes when the app compiles and can be exported, and each hidden test is its own item.1 AI query 14 tool calls
Passed IN511 072 OUT47 287
✓
Hidden_GameOverAndRestartWhen a new piece does not fit, the game is over and stops; Enter starts a new game.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_GetsFasterThe game gets faster: dropFrames never grows, is at most 20 after 10 rows, never below 2, and gravity really steps every dropFrames frames.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_GravityGravity moves the piece down by one row in the 30th frame and then every 30 frames.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_HardDropSpace drops the piece to the bottom, locks it, adds 2 points per row and brings the next piece.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_InitialStateAfter AppInit the board is empty, the first piece is at its start position, and score, lines and speed have their starting values.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_LockingWithoutHardDropSoft drop and gravity lock a piece that cannot move down.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_MovesAndStopsAtTheWallsLeft and Right move the piece by one column and stop it at the walls.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_OneRowA full row is removed, the rows above move down, and score grows by 100.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_PieceCompletesRowsWithAGapA piece that completes two rows with a not full row between them removes exactly those two rows.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_PieceOrderThe pieces come in an order where any 14 pieces in a row contain all 7 types, and each new piece is the announced next one.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_PiecesAndRotationsEvery piece in every rotation has the cells of the task's table and rotation rule: a hard drop locks exactly those cells.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_RotationAtTheWallA rotation that does not fit is either ignored or moved by at most 2 columns to a free position in the same row.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_RowsInAMixedBoardFull rows that are not next to each other in a mixed board are removed, the rows above them move down exactly, and the screen shows the new board.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenAfterARowLocked cells are drawn in the colors of their types, also after a row is removed and its effect ends.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_ScreenShowsTheSceneThe screen shows the board area, the falling piece, the next piece and the texts.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
✓
Hidden_SeveralRowsRemoving 2, 3 or 4 rows at once gives 300, 500 or 800 points.0 AI queries 0 tool callsHiddenTest
Passed IN0 OUT0
×
Hidden_SoftDropHolding Down moves the piece down one row every frame; the gravity counter does not grow in those frames (step 5 is skipped).0 AI queries 0 tool callsHiddenTest
Assertion expression evaluated to false: pieceY == 4 Context: no gravity step yet, pieceY 5 Values: pieceY = 5 (line 670)
Failed IN0 OUT0