Ai4Users.comLocal AI & LLM community
Sign in to contributeSign in

Qwen 3.8 Flash Next NVFP4 Think: medium | Benchmark #74

Described in:Qwen 3.8 Flash Next NVFP4: benchmark results and the broken thinking mode

Benchmark summary

SC# Coding & Agentic Benchmark by Ai4Users

ModelQwen 3.8 Flash Next QuantizationNVFP4 ModeThink: medium

A custom Ai4Users benchmark suite for SC#, covering code generation, reasoning, and stateful agentic workflows where models use tools, observe results, adapt their actions, recover from failures, and verify the final state.

Score 152/331 45.9% passed
152 passed176 failed3 broken331 total 857 AI queries 372 tool calls TOKENS IN8 520 134 TOKENS OUT1 758 976

Results by difficulty

4 groups
Level Easy
34 / 65 52.3%
passed
178 AI queries 0 tool calls IN1 632 754 OUT213 217
Level Medium
58 / 156 37.2%
passed
469 AI queries 0 tool calls IN4 090 871 OUT1 065 152
Level Hard
35 / 68 51.5%
passed
154 AI queries 118 tool calls IN1 721 343 OUT390 399
Level VeryHard
25 / 42 59.5%
passed
56 AI queries 254 tool calls IN1 075 166 OUT90 208

Results by category

10 groups
Category AgenticCoding
23.1% 3/13 passed
37 AI queries 0 tool calls TOKENS IN490 472 TOKENS OUT144 023
Category AgenticReasoning
83.3% 5/6 passed
6 AI queries 7 tool calls TOKENS IN150 739 TOKENS OUT12 500
Category AgenticToolUse
66.7% 38/57 passed
59 AI queries 365 tool calls TOKENS IN1 098 089 TOKENS OUT57 907
Category Algorithms
46.3% 50/108 passed
294 AI queries 0 tool calls TOKENS IN3 094 148 TOKENS OUT766 388
Category CodeGeneration
42.1% 112/266 passed
769 AI queries 8 tool calls TOKENS IN7 411 465 TOKENS OUT1 696 813
Category ContextProcessing
22.2% 2/9 passed
30 AI queries 0 tool calls TOKENS IN40 464 TOKENS OUT7 374
Category Database
0.0% 0/17 passed
85 AI queries 0 tool calls TOKENS IN97 170 TOKENS OUT11 781
Category GameDev
40.2% 51/127 passed
377 AI queries 0 tool calls TOKENS IN3 776 355 TOKENS OUT841 546
Category InstructionFollowing
36.4% 4/11 passed
33 AI queries 0 tool calls TOKENS IN60 200 TOKENS OUT10 033
Category Reasoning
65.2% 30/46 passed
67 AI queries 225 tool calls TOKENS IN931 599 TOKENS OUT107 598

Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.

Benchmark items
Script / test Status Tokens
Agentic - Blue-Green Deploy (health-gated cutover) 1 passed
100.0%
 IN16 459 OUT670
✓
AgenticBlueGreenCutoverHealth-gated blue-green cutover: green is deterministically unhealthy, so the correct outcome is to keep blue live (failed-safe) after observing the precheck; cutting over ships a broken env.1 AI query 5 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN16 459 OUT670
Agentic - Bugfix (root-cause vs regression) 1 passed
100.0%
 IN20 320 OUT466
✓
AgenticBugfixRegressionGuardBugfix with regression guard: fix a defect so the reproduction passes without breaking regression. The naive over-narrow fix passes the repro but fails a regression case; the root-cause fix passes all.1 AI query 1 tool callcategory:AgenticToolUsecategory:CodeGenerationlevel:VeryHardsource:Ai4Users
 Passed IN20 320 OUT466
Agentic - Cache Invalidation (exact set) 1 passed
100.0%
 IN16 422 OUT1 138
✓
AgenticCacheInvalidationCompletenessCache-invalidation completeness: after changing specific record fields, invalidate exactly the dependent caches (miss one = stale; evict an extra = unnecessary). Both fail the strict check.1 AI query 8 tool callscategory:AgenticToolUselevel:Hardsource:Ai4Users
 Passed IN16 422 OUT1 138
Agentic - Canary (metric-gated release) 1 passed
100.0%
 IN34 867 OUT1 358
✓
AgenticCanaryMetricGatedReleaseMetric-driven canary: two candidates (one healthy, one regressed); the agent must read canary metrics and promote only the healthy one. Blind promotion ships the regression and fails.1 AI query 9 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN34 867 OUT1 358
Agentic - Canary variants (5 metric cases) 4 passed1 failed
80.0%
 IN154 357 OUT6 284
✓
CanaryVariantBothRegressedTwo candidates, BOTH regressed -> promote none, keep the stable build in production.1 AI query 9 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN24 415 OUT934
✓
CanaryVariantOnlyAHealthyTwo candidates, only A is healthy -> promote A.1 AI query 9 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN35 527 OUT1 607
×
CanaryVariantOnlyBHealthyTwo candidates, only B is healthy -> promote B (do not assume the first listed is best).1 AI query 2 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: prod == expected Context: Production ended on build_1900 but the correct choice was build_2302. Values: prod = "build_1900" expected = "build_2302"
 Failed IN5 671 OUT344
✓
CanaryVariantThreeOneHealthyThree candidates, exactly one healthy -> promote it, reject the other two.1 AI query 13 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN60 273 OUT2 418
✓
CanaryVariantTieBreakLatencyTwo candidates, BOTH healthy -> tie-break on lowest p95 latency (promote the faster one).1 AI query 8 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN28 471 OUT981
Agentic - Code Review (find injected defect) 1 passed
100.0%
 IN13 739 OUT1 247
✓
AgenticFindInjectedDefectCode review: identify the one hunk that breaks the contract (H3: off-by-one in remaining days) among behaviour-preserving red herrings.1 AI query 8 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN13 739 OUT1 247
Agentic - Config Drift (remediate exact) 1 passed
100.0%
 IN9 002 OUT755
✓
AgenticConfigDriftRemediationConfig-drift remediation: fix exactly the non-allowlisted drifted keys to desired without touching the environment-specific key. Over-correcting or missing a drifted key fails.1 AI query 5 tool callscategory:AgenticToolUselevel:Hardsource:Ai4Users
 Passed IN9 002 OUT755
Agentic - Constraints (multi-section satisfaction) 1 failed
0.0%
 IN4 663 OUT280
×
AgenticConstraintSatisfactionConstraint satisfaction across multiple runbook sections with a tempting-but-invalid default; the agent must intersect all constraints and pick a feasible pool_size.1 AI query 2 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
Assertion expression evaluated to false: p >= 40 && p <= 56 && p % 8 == 0 Context: pool_size 0 does not satisfy all constraints (need 40..56, multiple of 8). Values: p = 0
 Failed IN4 663 OUT280
Agentic - Constraints variants (5 cases) 3 passed2 failed
60.0%
 IN40 756 OUT3 148
✓
ConstraintsVariantBatchSizebatch_size in [100,1000], multiple of 100, divides 10000 (feasible {100,200,500,1000}); tempting 300 does not divide 10000.1 AI query 3 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN11 137 OUT954
×
ConstraintsVariantCacheMbcache_mb in [128,512], power of two (feasible {128,256,512}); tempting "start at 100" is not a power of two.1 AI query 1 tool callcategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("cv.value_set", false) Context: Agent did not set a value.
 Failed IN4 490 OUT539
×
ConstraintsVariantPoolSizepool_size in [40,56], multiple of 8 (feasible {40,48,56}); tempting default 24 violates the minimum.1 AI query 1 tool callcategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("cv.value_set", false) Context: Agent did not set a value.
 Failed IN4 514 OUT477
✓
ConstraintsVariantTimeoutMstimeout_ms in [200,1000], multiple of 50, and > 750; tempting "typical 500" violates the >750 rule.1 AI query 3 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN10 495 OUT666
✓
ConstraintsVariantWorkerThreadsworker_threads in [12,64], multiple of 4, and NOT 32; tempting "start at 32" is forbidden.1 AI query 3 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN10 120 OUT512
Agentic - Data Reconciliation (per-field precedence) 1 passed
100.0%
 IN14 185 OUT844
✓
AgenticTwoSourceReconciliationTwo-source reconciliation with per-field precedence (email->crm, plan->billing) and missing-record creation; blind whole-source copy or wrong precedence fails the per-field grade.1 AI query 8 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN14 185 OUT844
Agentic - Diagnosis (root cause under budget) 1 passed
100.0%
 IN10 358 OUT1 074
✓
AgenticRootCauseDiagnosisRoot-cause diagnosis with upward fault propagation: identify the deepest unhealthy component whose dependencies are all healthy (the true root), not a symptomatic component higher up.1 AI query 13 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN10 358 OUT1 074
Agentic - Diagnosis variants (5 cases) 4 passed1 failed
80.0%
 IN65 369 OUT4 614
×
DiagnosisVariantChainDiagnosis variant — linear chain a->b->c->d, fault at c (deepest unhealthy with a healthy dep).1 AI query 5 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("diag.has_report", false) Context: Agent did not report a root cause.
 Failed IN7 867 OUT472
✓
DiagnosisVariantDeepLeafDiagnosis variant — shared leaf dependency (svcA->svcB->svcC, svcD->svcB), fault at the deep leaf svcC.1 AI query 10 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN12 581 OUT1 044
✓
DiagnosisVariantDiamondDiagnosis variant — diamond (top->left,right; left->core; right->core), fault at the convergence node core. Two branches look unhealthy but both converge on the single root.1 AI query 10 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN12 421 OUT913
✓
DiagnosisVariantTreeDiagnosis variant — tree (web->api,cache; api->auth,db), fault at db.1 AI query 10 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN15 815 OUT1 066
✓
DiagnosisVariantWideGraphDiagnosis variant — larger 7-node graph, fault mid-graph at n4 (n4->n6 healthy).1 AI query 12 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN16 685 OUT1 119
Agentic - ETL Repair (root-cause in stage 2) 1 failed
0.0%
 IN9 263 OUT957
×
AgenticEtlPipelineRepairETL repair with propagating error: stage2 misconfiguration only surfaces at the stage4 report; the agent must trace back, fix stage2's scale, and reprocess downstream — not patch the symptom.1 AI query 6 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("etl.reconciled", false) Context: The stage4 report did not reconcile (root cause not fixed / downstream not reprocessed).
 Failed IN9 263 OUT957
Agentic - Feature Flag (staged rollout + kill-switch) 1 failed
0.0%
 IN11 228 OUT426
×
AgenticFeatureFlagKillSwitchStaged feature-flag rollout with a kill-switch: the feature is healthy at 10% but spikes from 50%; the agent must observe the spike and roll back to a safe stage rather than ship to 100%.1 AI query 4 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("ff.saw_spike", false) Context: Agent never observed the error spike before deciding (did not monitor the 50% stage).
 Failed IN11 228 OUT426
Agentic - Game Backend: 1D bouncing ball 1 passed
100.0%
 IN34 776 OUT5 412
✓
GameBackendBounceCodeGenCode-gen: model implements the bouncing-ball position; graded vs exact oracle.3 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN34 776 OUT5 412
Agentic - Game Backend: 1D swept anti-tunneling 1 passed
100.0%
 IN8 811 OUT1 219
✓
GameBackendAntiTunnelCodeGenCode-gen: model implements swept 1D movement; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 811 OUT1 219
Agentic - Game Backend: 2048 left-move (slide + merge) 1 failed
0.0%
 IN47 025 OUT21 457
×
GameBackend2048CodeGenCode-gen: model implements a 2048 left-move; graded vs exact oracle.4 AI queries 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
MoveLeft did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN47 025 OUT21 457
Agentic - Game Backend: 8-direction from vector 1 passed
100.0%
 IN18 951 OUT2 196
✓
GameBackendDir8CodeGenCode-gen: model implements 8-direction classification; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN18 951 OUT2 196
Agentic - Game Backend: AABB collision resolution (MTV) 1 failed
0.0%
 IN35 852 OUT4 368
×
GameBackendAabbResolveCodeGenCode-gen: model implements AABB push-out; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
ResolveAabb did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 852 OUT4 368
Agentic - Game Backend: AABB containment 1 failed
0.0%
 IN35 212 OUT2 990
×
GameBackendAabbContainsCodeGenCode-gen: model implements AABB containment; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Contains did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 212 OUT2 990
Agentic - Game Backend: Acceleration to max speed 1 passed
100.0%
 IN8 739 OUT2 663
✓
GameBackendAccelCodeGenCode-gen: model implements acceleration; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 739 OUT2 663
Agentic - Game Backend: Achievement thresholds 1 failed
0.0%
 IN35 188 OUT3 881
×
GameBackendAchievementsCodeGenCode-gen: model implements achievements; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Unlocked did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN35 188 OUT3 881
Agentic - Game Backend: Active status effects 1 failed
0.0%
 IN35 276 OUT5 677
×
GameBackendStatusDurationCodeGenCode-gen: model implements status duration; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
ActiveAt did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 276 OUT5 677
Agentic - Game Backend: Animation frame at time 1 passed
100.0%
 IN18 164 OUT1 226
✓
GameBackendFrameAtTimeCodeGenCode-gen: model implements frame-at-time; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN18 164 OUT1 226
Agentic - Game Backend: Army upkeep income 1 failed
0.0%
 IN35 140 OUT5 519
×
GameBackendUpkeepCodeGenCode-gen: model implements upkeep; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
NetIncome did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 140 OUT5 519
Agentic - Game Backend: Axis-aligned path follow 1 failed
0.0%
 IN66 434 OUT17 972
×
GameBackendPathFollowCodeGenCode-gen: model implements path following; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
PosAt did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 4:18: Unexpected character '`'. 4:19: Unexpected character '#'. 4:20: Unexpected character '`'. 6:18: Unexpected character '`'. 6:22: Unexpected character '`'. 10:4: Char literals must contain exactly one character and close with '''. 80:624: Char literals must contain exactly one character and close with '''. 84:21: Unexpected character '`'. 84:31: Unexpected character '`'. 84:45: Unexpected character '`'. 84:50: Unexpected character '`'. 84:61: Unexpected character '`'. 84:71: Unexpected character '`'. 84:110: Unexpected character '`'. 84:124: Unexpected character '`'. 86:21: Unexpected character '`'. 86:32: Unexpected character '`'. 88:15: Unexpected character '`'. 88:28: Unexpected character '`'. 90:15: Unexpected character '`'. 90:25: Unexpected character '`'. 92:15: Unexpected character '`'. 92:25: Unexpected character '`'. 94:15: Unexpected character '`'. 94:25: Unexpected character '`'. 96:15: Unexpected character '`'. 96:20: Unexpected character '`'. 98:27: Unexpected character '`'. 98:36: Unexpected character '`'. 100:15: Unexpected character '`'. 100:27: Unexpected character '`'. 102:27: Unexpected character '`'. 102:58: Unexpected character '`'. 102:63: Unexpected character '`'. 102:72: Unexpected character '`'. 104:15: Unexpected character '`'. 104:41: Unexpected character '`'. 106:15: Unexpected character '`'. 106:28: Unexpected character '`'. 110:16: Unexpected character '#'. 110:29: Unexpected character '`'. 110:51: Unexpected character '`'. 110:95: Unexpected character '`'. 110:106: Unexpected character '`'. 110:124: Unexpected character '`'. 110:141: Unexpected character '`'. 110:154: Unexpected character '`'. 110:170: Unexpected character '`'. 112:57: Unexpected character '`'. 112:62: Unexpected character '`'. 114:30: Unexpected character '`'. 114:39: Unexpected character '`'. 116:30: Unexpected character '`'. 116:41: Unexpected character '`'. 118:30: Unexpected character '`'. 118:54: Unexpected character '`'. 120:30: Unexpected character '`'. 120:41: Unexpected character '`'. 122:30: Unexpected character '`'. 122:31: Unexpected character '#'. 122:32: Unexpected character '`'. 122:60: Unexpected character '`'. 122:63: Unexpected character '#'. 122:64: Unexpected character '`'. 124:18: Unexpected character '`'. 124:19: Unexpected character '#'. 124:20: Unexpected character '`'. 126:27: Unexpected character '`'. 126:38: Unexpected character '`'. 126:101: Unexpected character '`'. 126:154: Unexpected character '`'. 128:15: Unexpected character '`'. 128:25: Unexpected character '`'. 130:4: Char literals must contain exactly one character and close with '''. 132:27: Unexpected character '`'. 132:51: Unexpected character '`'. 134:27: Unexpected character '`'. 134:38: Unexpected character '`'. 136:27: Unexpected character '`'. 136:36: Unexpected character '`'. 138:15: Unexpected character '`'. 138:20: Unexpected character '`'. 142:4: Char literals must contain exactly one character and close with '''. 160:53: Unexpected character '`'. 160:54: Unexpected character '#'. 160:55: Unexpected character '`'. 160:99: Unexpected character '#'. 160:157: Char literals must contain exactly one character and close with '''. 168:4: Char literals must contain exactly one character and close with '''.
 Failed IN66 434 OUT17 972
Agentic - Game Backend: Battleship shot resolution 1 passed
100.0%
 IN30 157 OUT3 725
✓
GameBackendBattleshipCodeGenCode-gen: model implements Battleship shot resolution; graded vs exact oracle.3 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN30 157 OUT3 725
Agentic - Game Backend: Bounce restitution 1 passed
100.0%
 IN8 758 OUT329
✓
GameBackendRestitutionCodeGenCode-gen: model implements restitution bounce; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN8 758 OUT329
Agentic - Game Backend: Bresenham line rasterization 1 passed
100.0%
 IN19 307 OUT2 641
✓
GameBackendBresenhamCodeGenCode-gen: model implements Bresenham line; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN19 307 OUT2 641
Agentic - Game Backend: Buff/debuff stacking 1 passed
100.0%
 IN8 784 OUT1 567
✓
GameBackendBuffsCodeGenCode-gen: model implements buff stacking; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 784 OUT1 567
Agentic - Game Backend: Build queue completions 1 passed
100.0%
 IN8 752 OUT601
✓
GameBackendBuildQueueCodeGenCode-gen: model implements the build queue; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN8 752 OUT601
Agentic - Game Backend: Camera-relative movement 1 failed
0.0%
 IN45 921 OUT10 289
×
GameBackendCameraMoveCodeGenCode-gen: model implements camera-relative movement; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
WorldMove did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:154: Unexpected character '#'. 4:102: Unexpected character '#'. 4:166: Unexpected character '#'. 4:178: Char literals must contain exactly one character and close with '''. 8:58: Char literals must contain exactly one character and close with '''. 14:32: Unexpected character '#'. 16:4: Char literals must contain exactly one character and close with '''. 20:7: Unexpected character '`'. 20:19: Unexpected character '`'. 20:77: Unexpected character '`'. 20:89: Unexpected character '`'. 22:7: Unexpected character '`'. 22:14: Unexpected character '`'. 24:7: Unexpected character '`'. 24:21: Unexpected character '`'. 26:36: Unexpected character '`'. 26:38: Unexpected character '`'. 28:7: Unexpected character '`'. 28:11: Unexpected character '`'. 30:9: Unexpected character '#'. 34:16: Unexpected character '`'. 34:65: Unexpected character '`'. 36:11: Unexpected character '`'. 36:32: Unexpected character '`'. 36:245: Unexpected character '`'. 36:246: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."? 36:247: Unexpected character '`'. 38:11: Unexpected character '`'. 38:55: Unexpected character '`'. 38:205: Char literals must contain exactly one character and close with '''. 40:29: Unexpected character '`'. 40:73: Unexpected character '`'. 42:43: Unexpected character '`'. 42:53: Unexpected character '`'. 44:23: Unexpected character '`'. 44:35: Unexpected character '`'. 46:25: Unexpected character '`'. 46:35: Unexpected character '`'. 48:98: Unexpected character '#'. 62:222: Char literals must contain exactly one character and close with '''. 68:44: Char literals must contain exactly one character and close with '''. 80:20: Unexpected character '#'. 88:37: Unexpected character '#'. 90:4: Char literals must contain exactly one character and close with '''. 92:11: Unexpected character '`'. 92:18: Unexpected character '`'. 92:420: Char literals must contain exactly one character and close with '''. 94:11: Unexpected character '`'. 94:23: Unexpected character '`'. 94:38: Unexpected character '`'. 94:47: Unexpected character '`'. 96:28: Unexpected character '`'. 96:35: Unexpected character '`'. 98:16: Unexpected character '`'. 98:21: Unexpected character '`'. 98:31: Unexpected character '`'. 98:46: Unexpected character '`'. 98:49: Unexpected character '`'. 98:74: Unexpected character '`'. 98:80: Unexpected character '`'. 98:89: Unexpected character '`'. 98:108: Unexpected character '#'. 100:11: Unexpected character '`'. 100:16: Unexpected character '`'. 100:23: Unexpected character '`'. 100:26: Unexpected character '`'. 100:49: Unexpected character '`'. 100:52: Unexpected character '`'. 100:101: Unexpected character '#'. 100:137: Unexpected character '`'. 100:168: Unexpected character '`'. 104:27: Unexpected character '`'. 104:31: Unexpected character '`'. 104:54: Unexpected character '`'. 104:72: Unexpected character '`'. 106:11: Unexpected character '`'. 106:20: Unexpected character '`'. 106:48: Unexpected character '`'. 106:55: Unexpected character '`'. 115:139: Char literals must contain exactly one character and close with '''. 118:51: Unexpected character '`'. 118:54: Unexpected character '`'. 118:92: Unexpected character '`'. 118:102: Unexpected character '`'. 118:165: Unexpected character '`'. 118:167: Unexpected character '`'. 120:31: Unexpected character '`'. 120:33: Unexpected character '`'. 120:60: Unexpected character '`'. 120:62: Unexpected character '`'. 120:88: Unexpected character '#'. 120:123: Unexpected character '#'. 120:183: Char literals must contain exactly one character and close with '''. 120:333: Unexpected character '`'. 120:335: Unexpected character '`'. 120:377: Unexpected character '`'. 120:379: Unexpected character '`'. 136:98: Unexpected character '#'. 138:11: Unexpected character '`'. 138:29: Unexpected character '`'. 138:71: Too many lexical errors; further errors are not reported.
 Failed IN45 921 OUT10 289
Agentic - Game Backend: Card hand sort 1 passed
100.0%
 IN8 760 OUT1 439
✓
GameBackendHandSortCodeGenCode-gen: model implements hand sort; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 760 OUT1 439
Agentic - Game Backend: Card-game rules (blackjack/deck/compare) 1 failed
0.0%
 IN64 462 OUT32 153
×
GameBackendCardsCodeGenCode-gen: model implements the card-rule functions to spec; each is graded against an exact oracle over fixed cases + random hands drawn from a real deck.4 AI queries 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
BlackjackValue did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:1: Expected '{' to start a block, found end of file.
 Failed IN64 462 OUT32 153
Agentic - Game Backend: Chart scaling & histogram bins 1 failed
0.0%
 IN35 776 OUT7 350
×
GameBackendChartsCodeGenCode-gen: model implements chart scaling + binning; graded vs oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
MapToPixel did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 776 OUT7 350
Agentic - Game Backend: Checkers captures 1 failed
0.0%
 IN70 773 OUT30 247
×
GameBackendCheckersCodeGenCode-gen: model implements checkers captures; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Captures did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 4:37: Unexpected character '`'. 4:38: Unexpected character '#'. 4:39: Unexpected character '`'. 6:4: Char literals must contain exactly one character and close with '''. 12:43: Unexpected character '`'. 12:46: Unexpected character '`'. 16:82: Char literals must contain exactly one character and close with '''. 18:4: Char literals must contain exactly one character and close with '''. 18:468: Unexpected character '`'. 18:488: Unexpected character '`'. 18:573: Unexpected character '`'. 18:589: Unexpected character '`'. 20:9: Unexpected character '#'. 20:19: Unexpected character '`'. 20:41: Unexpected character '`'. 20:51: Unexpected character '`'. 20:61: Unexpected character '`'. 20:84: Unexpected character '#'. 22:9: Unexpected character '#'. 24:34: Char literals must contain exactly one character and close with '''. 30:53: Unexpected character '#'. 40:4: Char literals must contain exactly one character and close with '''.
 Failed IN70 773 OUT30 247
Agentic - Game Backend: Collision primitives (circle/aabb/segment) 1 failed
0.0%
 IN9 016 OUT1 310
×
GameBackendCollisionCodeGenCode-gen: model implements the collision primitives to spec; each is graded against an exact integer oracle over fixed edge cases + random inputs.1 AI query 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
AabbOverlap did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: Entry point 'AabbOverlap' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, CircleCircle.
 Failed IN9 016 OUT1 310
Agentic - Game Backend: Combat turn order (initiative) 1 failed
0.0%
 IN35 196 OUT6 311
×
GameBackendInitiativeCodeGenCode-gen: model implements combat turn order; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
TurnOrder did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 196 OUT6 311
Agentic - Game Backend: Combo scoring (rising multiplier) 1 failed
0.0%
 IN35 486 OUT3 637
×
GameBackendComboCodeGenCode-gen: model implements combo scoring; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
ComboScore did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 486 OUT3 637
Agentic - Game Backend: Control-point capture 1 passed
100.0%
 IN8 799 OUT1 320
✓
GameBackendCapturePointCodeGenCode-gen: model implements capture point; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 799 OUT1 320
Agentic - Game Backend: Controls & Inventory (spec-driven function battery) 1 failed
0.0%
 IN35 928 OUT6 008
×
GameBackendControlsCodeGenCode-gen: model implements TurnRight/TurnLeft/StepForward/ReloadMag to spec; each is run over an oracle-checked battery (fixed edge cases + random inputs).4 AI queries 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
TurnRight did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 928 OUT6 008
Agentic - Game Backend: Cooldowns & resource regen 1 failed
0.0%
 IN40 559 OUT2 095
×
GameBackendCooldownCodeGenCode-gen: model implements cooldown/regen helpers; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
AbilityReady did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:24: Unexpected character '#'. 5:33: Unexpected character '#'. 9:4: Unexpected character '`'. 9:63: Unexpected character '`'. 9:75: Unexpected character '`'. 9:106: Unexpected character '`'. 10:4: Unexpected character '`'. 10:67: Unexpected character '`'. 10:79: Unexpected character '`'. 10:111: Unexpected character '`'. 11:4: Unexpected character '`'. 11:84: Unexpected character '`'. 13:22: Unexpected character '#'.
 Failed IN40 559 OUT2 095
Agentic - Game Backend: Coyote-time jump 1 passed
100.0%
 IN17 777 OUT509
✓
GameBackendCoyoteCodeGenCode-gen: model implements coyote-time; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN17 777 OUT509
Agentic - Game Backend: Crafting capacity 1 passed
100.0%
 IN8 791 OUT1 292
✓
GameBackendCraftingCodeGenCode-gen: model implements crafting count; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 791 OUT1 292
Agentic - Game Backend: Cribbage fifteens 1 passed
100.0%
 IN8 749 OUT896
✓
GameBackendCribbageCodeGenCode-gen: model implements cribbage fifteens; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 749 OUT896
Agentic - Game Backend: Critical-hit damage 1 passed
100.0%
 IN17 987 OUT684
✓
GameBackendCritCodeGenCode-gen: model implements crit damage; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN17 987 OUT684
Agentic - Game Backend: Damage over time 1 passed
100.0%
 IN8 765 OUT907
✓
GameBackendDoTCodeGenCode-gen: model implements damage-over-time; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 765 OUT907
Agentic - Game Backend: Day/night cycle phase 1 passed
100.0%
 IN8 770 OUT591
✓
GameBackendDayNightCodeGenCode-gen: model implements day/night phase; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 770 OUT591
Agentic - Game Backend: Dice notation max roll 1 failed
0.0%
 IN44 252 OUT10 288
×
GameBackendDiceCodeGenCode-gen: model implements dice-notation max roll; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
MaxRoll did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:171: Unexpected character '#'. 1:298: Char literals must contain exactly one character and close with '''. 10:139: Char literals must contain exactly one character and close with '''. 21:53: Unexpected character '#'. 26:57: Char literals must contain exactly one character and close with '''. 43:124: Unexpected character '`'. 43:131: Unexpected character '`'. 43:213: Char literals must contain exactly one character and close with '''. 43:255: Unexpected character '`'. 43:258: Unexpected character '`'. 45:9: Unexpected character '#'. 45:28: Unexpected character '`'. 45:33: Unexpected character '`'. 45:54: Unexpected character '`'. 45:57: Unexpected character '`'. 47:16: Unexpected character '`'. 47:39: Unexpected character '`'. 49:66: Unexpected character '`'. 49:68: Unexpected character '`'. 49:139: Unexpected character '`'. 49:175: Unexpected character '`'. 49:192: Unexpected character '#'. 49:199: Char literals must contain exactly one character and close with '''. 51:9: Unexpected character '#'. 51:26: Unexpected character '`'. 51:33: Unexpected character '`'. 53:10: Unexpected character '`'. 53:29: Unexpected character '`'. 55:62: Unexpected character '`'. 55:119: Unexpected character '`'. 57:204: Unexpected character '#'. 57:386: Unexpected character '`'. 57:394: Unexpected character '`'. 57:416: Unexpected character '`'. 57:424: Unexpected character '`'. 57:456: Unexpected character '`'. 57:461: Unexpected character '`'. 59:70: Unexpected character '`'. 59:72: Unexpected character '`'. 61:9: Unexpected character '#'. 61:62: Char literals must contain exactly one character and close with '''. 63:22: Unexpected character '`'. 63:45: Unexpected character '`'. 63:56: Unexpected character '`'. 63:74: Unexpected character '`'. 65:4: Char literals must contain exactly one character and close with '''. 77:182: Unexpected character '`'. 77:205: Unexpected character '`'. 80:100: Unexpected character '#'. 80:387: Unexpected character '`'. 80:395: Unexpected character '`'. 80:413: Unexpected character '#'. 80:447: Unexpected character '#'. 80:467: Unexpected character '#'. 80:479: Unexpected character '`'. 80:518: Unexpected character '`'. 80:523: Unexpected character '`'. 80:531: Unexpected character '`'. 80:591: Unexpected character '`'. 80:599: Unexpected character '`'. 80:623: Unexpected character '`'. 80:653: Unexpected character '`'. 120:119: Unexpected character '#'. 120:194: Unexpected character '`'. 120:206: Unexpected character '`'. 120:222: Unexpected character '`'. 120:226: Unexpected character '`'. 120:233: Unexpected character '`'. 120:241: Unexpected character '`'. 120:244: Unexpected character '`'. 120:253: Unexpected character '`'. 120:290: Unexpected character '#'. 122:18: Unexpected character '`'. 122:31: Unexpected character '`'. 122:40: Unexpected character '`'. 122:60: Unexpected character '`'. 122:130: Unexpected character '`'. 122:143: Unexpected character '`'. 122:173: Unexpected character '`'. 122:180: Unexpected character '`'. 122:190: Unexpected character '#'. 122:219: Unexpected character '`'. 122:237: Unexpected character '`'. 122:309: Unexpected character '`'. 122:339: Unexpected character '`'. 124:18: Unexpected character '`'. 124:40: Unexpected character '`'. 124:68: Unexpected character '`'. 124:94: Unexpected character '`'. 136:158: Char literals must contain exactly one character and close with '''. 138:103: Unexpected character '`'. 138:134: Unexpected character '`'. 138:140: Char literals must contain exactly one character and close with '''. 145:135: Unexpected character '#'. 145:146: Unexpected character '`'. 145:152: Unexpected character '`'. 147:27: Unexpected character '`'. 147:42: Unexpected character '`'. 149:20: Unexpected character '#'. 149:22: Unexpected character '`'. 149:35: Too many lexical errors; further errors are not reported.
 Failed IN44 252 OUT10 288
Agentic - Game Backend: Enemy AI state machine 1 passed
100.0%
 IN8 852 OUT1 839
✓
GameBackendEnemyFsmCodeGenCode-gen: model implements the enemy FSM; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
 Passed IN8 852 OUT1 839
Agentic - Game Backend: Equipment slot validation 1 failed
0.0%
 IN35 328 OUT1 738
×
GameBackendInventorySlotsCodeGenCode-gen: model implements slot validation; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
SlotAccepts did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN35 328 OUT1 738
Agentic - Game Backend: Flood-fill region size 1 failed
0.0%
 IN79 767 OUT38 887
×
GameBackendFloodFillCodeGenCode-gen: model implements flood-fill region size; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
RegionSize did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:141: Unexpected character '#'. 1:218: Unexpected character '#'. 1:411: Char literals must contain exactly one character and close with '''. 1:515: Unexpected character '`'. 1:516: Char literals must contain exactly one character and close with '''. 1:776: Unexpected character '\'. 1:778: Char literals must contain exactly one character and close with '''. 1:838: Unexpected character '#'. 1:871: Unexpected character '\'. 1:925: Unexpected character '\'. 1:983: Invalid Unicode escape sequence. Use exactly four hexadecimal digits after '\u'. 1:1042: Unknown escape sequence '\0'. Supported escapes: \" \\ \n \r \t. 1:1174: Unknown escape sequence '\0'. Supported escapes: \" \\ \n \r \t. 1:1219: Unknown escape sequence '\0'. Supported escapes: \" \\ \n \r \t. 1:1305: Unknown escape sequence '\0'. Supported escapes: \" \\ \n \r \t. 1:1332: Unexpected character '`'. 1:1337: Unexpected character '`'. 1:1876: Unknown escape sequence '\0'. Supported escapes: \" \\ \n \r \t. 1:1882: Invalid Unicode escape sequence. Use exactly four hexadecimal digits after '\u'. 1:2042: Unknown escape sequence '\0'. Supported escapes: \" \\ \n \r \t. 1:3156: Unexpected character '#'. 1:3213: Unterminated string literal. Strings must be closed with '"' on the same line. 5:15: Char literals must contain exactly one character and close with '''. 5:358: Unexpected character '#'. 5:399: Unexpected character '`'. 5:402: Unexpected character '#'. 5:403: Unexpected character '`'. 5:601: Unexpected character '#'. 7:4: Char literals must contain exactly one character and close with '''. 8:3: Unexpected character '#'. 8:7: Unexpected character '#'. 10:5: Unexpected character '#'. 13:9: Unexpected character '`'. 13:14: Unexpected character '`'. 13:24: Unexpected character '`'. 13:29: Unexpected character '`'. 16:127: Char literals must contain exactly one character and close with '''. 18:93: Unexpected character '`'. 18:106: Unexpected character '`'. 18:137: Char literals must contain exactly one character and close with '''. 20:4: Char literals must contain exactly one character and close with '''. 20:99: Unexpected character '\'. 20:101: Char literals must contain exactly one character and close with '''. 22:51: Unexpected character '`'. 22:83: Unexpected character '`'. 22:89: Unexpected character '`'. 22:94: Unexpected character '`'. 26:24: Unexpected character '#'. 28:112: Char literals must contain exactly one character and close with '''. 28:258: Char literals must contain exactly one character and close with '''. 30:52: Unexpected character '`'. 30:65: Unexpected character '`'. 30:92: Char literals must contain exactly one character and close with '''. 30:536: Char literals must contain exactly one character and close with '''. 32:388: Char literals must contain exactly one character and close with '''. 36:4: Char literals must contain exactly one character and close with '''. 37:3: Unexpected character '`'. 37:35: Unexpected character '`'. 37:46: Unexpected character '`'. 37:51: Unexpected character '`'. 38:3: Unexpected character '`'. 38:69: Unexpected character '`'. 39:3: Unexpected character '`'. 39:24: Unexpected character '`'. 40:3: Unexpected character '`'. 40:17: Unexpected character '`'. 41:3: Unexpected character '`'. 41:23: Unexpected character '`'. 43:9: Unexpected character '#'. 43:23: Unexpected character '`'. 43:36: Unexpected character '`'. 45:9: Unexpected character '#'. 45:23: Unexpected character '`'. 45:60: Unexpected character '`'. 45:70: Unexpected character '`'. 45:86: Unexpected character '`'. 47:9: Unexpected character '#'. 47:23: Unexpected character '`'. 47:37: Unexpected character '`'. 49:9: Unexpected character '#'. 49:23: Unexpected character '`'. 49:60: Unexpected character '`'. 51:9: Unexpected character '#'. 51:23: Unexpected character '`'. 51:55: Unexpected character '`'. 53:9: Unexpected character '#'. 53:23: Unexpected character '`'. 53:51: Unexpected character '`'. 55:9: Unexpected character '#'. 55:23: Unexpected char ... (message truncated)
 Failed IN79 767 OUT38 887
Agentic - Game Backend: Fog of war visibility 1 passed
100.0%
 IN18 433 OUT1 522
✓
GameBackendFogOfWarCodeGenCode-gen: model implements fog-of-war visibility; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN18 433 OUT1 522
Agentic - Game Backend: Fog reveal (Chebyshev radius) 1 broken
0.0%
 IN33 023 OUT52 601
!
GameBackendFogRadiusCodeGenCode-gen: model implements fog reveal; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
repair response incomplete: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.
 Broken IN33 023 OUT52 601
Agentic - Game Backend: Friction stop frame 1 failed
0.0%
 IN35 252 OUT2 257
×
GameBackendFrictionCodeGenCode-gen: model implements friction stop; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
StopFrame did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 252 OUT2 257
Agentic - Game Backend: Game of Life (code-gen vs oracle) 1 failed
0.0%
 IN8 825 OUT1 399
×
GameOfLifeCodeGenCode-gen: the model implements Tick(); we simulate its candidate on random boards for several steps and compare every generation to the reference oracle.1 AI query 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
candidate did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:24: Char literals must contain exactly one character and close with '''. 5:40: Unexpected character '`'. 5:44: Unexpected character '`'. 10:50: Char literals must contain exactly one character and close with '''. 15:37: Unexpected character '`'. 15:41: Unexpected character '`'. 19:1: Unexpected character '`'. 19:2: Unexpected character '`'. 19:3: Unexpected character '`'. 79:1: Unexpected character '`'. 79:2: Unexpected character '`'. 79:3: Unexpected character '`'. 81:22: Unexpected character '#'. 82:3: Unexpected character '`'. 82:17: Unexpected character '`'. 82:37: Unexpected character '`'. 82:44: Unexpected character '`'. 82:53: Unexpected character '✓'. 83:3: Unexpected character '`'. 83:19: Unexpected character '`'. 84:29: Unexpected character '`'. 84:46: Unexpected character '`'. 85:3: Unexpected character '`'. 85:17: Unexpected character '`'. 85:19: Unexpected character '✓'. 86:3: Unexpected character '`'. 86:7: Unexpected character '`'. 86:20: Unexpected character '`'. 86:24: Unexpected character '`'. 86:26: Unexpected character '✓'. 87:3: Unexpected character '`'. 87:12: Unexpected character '`'. 87:14: Unexpected character '✓'. 88:19: Unexpected character '`'. 88:32: Unexpected character '`'. 88:47: Unexpected character '✓'. 89:32: Unexpected character '`'. 89:36: Unexpected character '`'. 89:38: Unexpected character '✓'. 91:22: Unexpected character '`'. 91:37: Unexpected character '`'. 91:70: Unexpected character '`'. 91:87: Unexpected character '`'. 91:298: Unexpected character '`'. 91:313: Unexpected character '`'. 93:130: Unexpected character '`'. 93:147: Unexpected character '`'. 97:12: Unexpected character '`'. 97:27: Unexpected character '`'. 97:103: Unexpected character '`'. 97:118: Unexpected character '`'. 99:48: Unexpected character '`'. 99:65: Unexpected character '`'. 99:71: Unexpected character '`'. 99:92: Unexpected character '`'. 99:166: Char literals must contain exactly one character and close with '''. 101:33: Unexpected character '`'. 101:37: Unexpected character '`'. 103:62: Unexpected character '`'. 103:79: Unexpected character '`'. 103:89: Unexpected character '`'. 103:91: Unexpected character '`'. 103:97: Unexpected character '`'. 103:100: Unexpected character '`'. 103:112: Unexpected character '`'. 103:115: Unexpected character '`'. 105:31: Char literals must contain exactly one character and close with '''. 107:100: Char literals must contain exactly one character and close with '''. 109:1: Unexpected character '`'. 109:2: Unexpected character '`'. 109:3: Unexpected character '`'. 113:1: Unexpected character '`'. 113:2: Unexpected character '`'. 113:3: Unexpected character '`'.
 Failed IN8 825 OUT1 399
Agentic - Game Backend: Greedy coin change 1 passed
100.0%
 IN18 237 OUT1 322
✓
GameBackendMakeChangeCodeGenCode-gen: model implements greedy change; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN18 237 OUT1 322
Agentic - Game Backend: Grid inventory & stacking (Diablo-style) 1 failed
0.0%
 IN35 812 OUT10 202
×
GameBackendInventoryCodeGenCode-gen: model implements the grid-inventory functions to spec; each is graded against an exact oracle over fixed cases + random inputs.4 AI queries 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
CanPlace did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 812 OUT10 202
Agentic - Game Backend: Grid pathfinding (shortest path, BFS) 1 passed
100.0%
 IN8 836 OUT2 168
✓
GameBackendPathfindCodeGenCode-gen: model implements grid shortest-path length; graded against an exact BFS oracle over fixed cases + random grids.1 AI query 0 tool callscategory:AgenticCodingcategory:Algorithmscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
 Passed IN8 836 OUT2 168
Agentic - Game Backend: Grid raycast 1 failed
0.0%
 IN52 936 OUT17 056
×
GameBackendRayMarchCodeGenCode-gen: model implements raycast; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
RayHit did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:14: Expected '(' to declare a function or '=' to declare a global variable after 'string RayHit', found '.'. Global variables must have an initializer.
 Failed IN52 936 OUT17 056
Agentic - Game Backend: Health bar segments 1 failed
0.0%
 IN56 338 OUT19 662
×
GameBackendHealthBarCodeGenCode-gen: model implements the health bar; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
FilledSegments did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:31: Unexpected character '`'. 2:83: Unexpected character '`'. 2:101: Char literals must contain exactly one character and close with '''. 2:221: Unexpected character '`'. 2:237: Unexpected character '`'. 4:27: Unexpected character '#'. 8:206: Unexpected character '`'. 8:226: Unexpected character '`'. 8:287: Unexpected character '`'. 8:307: Unexpected character '`'. 8:401: Unexpected character '`'. 8:421: Unexpected character '`'. 10:50: Unexpected character '#'. 10:56: Unexpected character '\'. 10:158: Unexpected character '#'. 10:315: Unexpected character '#'. 12:51: Char literals must contain exactly one character and close with '''. 12:157: Char literals must contain exactly one character and close with '''. 12:168: Char literals must contain exactly one character and close with '''. 14:58: Unexpected character '`'. 14:63: Unexpected character '`'. 14:111: Unexpected character '`'. 14:135: Unexpected character '`'. 29:32: Unexpected character '`'. 29:51: Unexpected character '`'. 33:64: Unexpected character '#'. 33:230: Char literals must contain exactly one character and close with '''. 33:330: Unexpected character '`'. 33:354: Unexpected character '`'. 39:50: Unexpected character '`'. 39:102: Unexpected character '`'. 39:120: Char literals must contain exactly one character and close with '''. 43:104: Unexpected character '`'. 43:105: Unexpected character '\'. 43:107: Unexpected character '`'. 45:4: Char literals must contain exactly one character and close with '''. 45:125: Char literals must contain exactly one character and close with '''. 45:244: Char literals must contain exactly one character and close with '''. 45:310: Char literals must contain exactly one character and close with '''. 45:319: Char literals must contain exactly one character and close with '''. 47:15: Char literals must contain exactly one character and close with '''. 47:94: Char literals must contain exactly one character and close with '''. 47:103: Char literals must contain exactly one character and close with '''. 49:53: Unexpected character '`'. 49:78: Unexpected character '`'. 49:104: Char literals must contain exactly one character and close with '''. 51:52: Unexpected character '`'. 51:81: Unexpected character '`'. 51:104: Char literals must contain exactly one character and close with '''. 53:21: Unexpected character '`'. 53:41: Unexpected character '`'. 53:112: Unexpected character '`'. 53:126: Unexpected character '`'. 55:254: Unexpected character '`'. 55:268: Unexpected character '`'. 57:110: Unexpected character '`'. 57:139: Unexpected character '`'. 65:43: Unexpected character '`'. 65:47: Unexpected character '`'. 65:129: Char literals must contain exactly one character and close with '''. 67:29: Unexpected character '#'. 67:43: Char literals must contain exactly one character and close with '''. 69:43: Unexpected character '`'. 69:48: Unexpected character '`'. 71:27: Unexpected character '`'. 71:41: Unexpected character '`'. 73:19: Char literals must contain exactly one character and close with '''. 75:161: Unexpected character '`'. 75:180: Unexpected character '`'. 75:197: Char literals must contain exactly one character and close with '''. 75:318: Unexpected character '`'. 75:323: Unexpected character '`'. 75:342: Unexpected character '#'. 81:43: Unexpected character '`'. 81:62: Unexpected character '`'. 83:15: Char literals must contain exactly one character and close with '''. 84:10: Unexpected character '`'. 84:19: Unexpected character '`'. 84:25: Unexpected character '`'. 84:34: Unexpected character '`'. 85:10: Unexpected character '`'. 85:24: Unexpected character '`'. 85:30: Unexpected character '`'. 85:43: Unexpected character '`'. 86:10: Unexpected character '`'. 86:15: Unexpected character '`'. 88:6: Unexpected character '#'. 90:234: Unexpected character '`'. 90:248: Unexpected character '`'. 90:260: Unexpected character '`'. 90:265: Unexpected character '`'. 90:281: Unexpected character '`'. 90:295: Unexpected c ... (message truncated)
 Failed IN56 338 OUT19 662
Agentic - Game Backend: High-score table insert 1 failed
0.0%
 IN35 300 OUT7 670
×
GameBackendHighScoreCodeGenCode-gen: model implements high-score insertion; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Insert did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 300 OUT7 670
Agentic - Game Backend: Hit chance 1 failed
0.0%
 IN35 236 OUT1 620
×
GameBackendHitChanceCodeGenCode-gen: model implements hit chance; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Hits did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 236 OUT1 620
Agentic - Game Backend: Inventory encumbrance 1 failed
0.0%
 IN35 344 OUT3 006
×
GameBackendEncumbranceCodeGenCode-gen: model implements encumbrance; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Status did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 344 OUT3 006
Agentic - Game Backend: Inventory first-fit placement 1 failed
0.0%
 IN56 952 OUT25 349
×
GameBackendInventoryFitCodeGenCode-gen: model implements inventory first-fit; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
PlaceFirstFit did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:215: Unexpected character '#'. 1:255: Unexpected character '#'. 3:22: Unexpected character '#'. 5:18: Unexpected character '#'. 5:383: Unexpected character '#'. 34:16: Unexpected character '#'. 34:88: Unexpected character '`'. 34:120: Unexpected character '`'. 34:128: Unexpected character '`'. 34:141: Unexpected character '`'. 34:149: Unexpected character '`'. 34:151: Unexpected character '`'. 36:27: Unexpected character '`'. 36:45: Unexpected character '`'. 38:18: Unexpected character '`'. 38:21: Unexpected character '`'. 38:113: Unexpected character '`'. 38:132: Unexpected character '`'. 40:27: Unexpected character '`'. 40:41: Unexpected character '`'. 42:38: Unexpected character '`'. 42:120: Unexpected character '`'. 50:15: Unexpected character '`'. 50:19: Unexpected character '`'.
 Failed IN56 952 OUT25 349
Agentic - Game Backend: Inventory stack split 1 passed
100.0%
 IN17 904 OUT769
✓
GameBackendStackSplitCodeGenCode-gen: model implements stack split; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN17 904 OUT769
Agentic - Game Backend: Isometric projection 1 failed
0.0%
 IN35 356 OUT1 890
×
GameBackendIsometricCodeGenCode-gen: model implements iso projection; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Project did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 356 OUT1 890
Agentic - Game Backend: Jump buffer 1 failed
0.0%
 IN40 473 OUT2 061
×
GameBackendJumpBufferCodeGenCode-gen: model implements jump buffer; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Jumped did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:9: Char literals must contain exactly one character and close with '''. 3:18: Unexpected character '#'. 4:46: Unexpected character '✓'. 5:3: Unexpected character '`'. 5:8: Unexpected character '`'. 5:22: Unexpected character '✓'. 6:3: Unexpected character '`'. 6:10: Unexpected character '`'. 6:17: Unexpected character '✓'. 7:3: Unexpected character '`'. 7:7: Unexpected character '`'. 7:14: Unexpected character '✓'. 8:18: Unexpected character '`'. 8:22: Unexpected character '`'. 8:25: Unexpected character '`'. 8:29: Unexpected character '`'. 8:50: Unexpected character '✓'. 9:3: Unexpected character '`'. 9:11: Unexpected character '`'. 9:23: Unexpected character '`'. 9:28: Unexpected character '`'. 9:30: Unexpected character '✓'. 10:3: Unexpected character '`'. 10:15: Unexpected character '`'. 10:37: Unexpected character '✓'. 11:3: Unexpected character '`'. 11:7: Unexpected character '`'. 11:25: Unexpected character '✓'. 12:3: Unexpected character '`'. 12:6: Unexpected character '`'. 12:23: Unexpected character '✓'. 13:3: Unexpected character '`'. 13:6: Unexpected character '`'. 13:34: Unexpected character '✓'. 23:30: Unexpected character '#'.
 Failed IN40 473 OUT2 061
Agentic - Game Backend: Knockback with arena clamp 1 failed
0.0%
 IN35 464 OUT3 461
×
GameBackendKnockbackCodeGenCode-gen: model implements knockback; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Resolve did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 464 OUT3 461
Agentic - Game Backend: Leaderboard rank 1 broken
0.0%
 IN17 510 OUT26 936
!
GameBackendLeaderboardRankCodeGenCode-gen: model implements leaderboard rank; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
repair response incomplete: AI output was marked broken by AiRunner. The response text is partial and must not be treated as a complete answer.
 Broken IN17 510 OUT26 936
Agentic - Game Backend: Lifecycle & counters (HP/death/respawn/game-over) 1 failed
0.0%
 IN41 723 OUT2 482
×
GameBackendLifecycleCodeGenCode-gen: model implements the lifecycle/counter functions to spec; each is graded against an exact oracle over fixed edge cases + random inputs.4 AI queries 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
ClampHp did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:36: Unexpected character '#'. 1:86: Unexpected character '#'. 3:25: Unexpected character '`'. 3:32: Unexpected character '`'. 3:61: Unexpected character '`'. 3:79: Unexpected character '`'. 4:4: Unexpected character '`'. 4:13: Unexpected character '`'. 4:19: Unexpected character '`'. 4:28: Unexpected character '`'. 10:33: Unexpected character '`'. 10:40: Unexpected character '`'. 10:107: Char literals must contain exactly one character and close with '''. 14:46: Unexpected character '#'. 16:18: Unexpected character '`'. 16:57: Unexpected character '`'. 18:1: Unexpected character '`'. 18:97: Unexpected character '`'. 20:12: Unexpected character '`'. 20:64: Unexpected character '`'. 20:72: Unexpected character '`'. 20:74: Unexpected character '`'. 20:248: Unexpected character '`'. 20:260: Unexpected character '`'. 22:141: Unexpected character '`'. 22:143: Unexpected character '`'. 22:181: Unexpected character '#'. 22:184: Unexpected character '`'. 22:197: Unexpected character '`'. 22:248: Unexpected character '#'. 24:12: Unexpected character '`'. 24:24: Unexpected character '`'. 26:2: Char literals must contain exactly one character and close with '''. 26:96: Char literals must contain exactly one character and close with '''.
 Failed IN41 723 OUT2 482
Agentic - Game Backend: Lifesteal healing 1 failed
0.0%
 IN49 217 OUT18 066
×
GameBackendLifestealCodeGenCode-gen: model implements lifesteal; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
HealTotal did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN49 217 OUT18 066
Agentic - Game Backend: Lights Out toggle 1 failed
0.0%
 IN35 304 OUT8 771
×
GameBackendLightsOutCodeGenCode-gen: model implements Lights Out toggle; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Toggle did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 304 OUT8 771
Agentic - Game Backend: Match-3 cascade 1 failed
0.0%
 IN48 208 OUT8 737
×
GameBackendMatch3CascadeCodeGenCode-gen: model implements match-3 cascade; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
Resolve did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 5:5: Expected an expression, found '.'.
 Failed IN48 208 OUT8 737
Agentic - Game Backend: Match-3 detection 1 failed
0.0%
 IN35 590 OUT7 256
×
GameBackendMatchThreeCodeGenCode-gen: model implements match-3 detection; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
CountMatched did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 590 OUT7 256
Agentic - Game Backend: Match-3 swap validity 1 failed
0.0%
 IN70 866 OUT33 996
×
GameBackendMatch3SwapCodeGenCode-gen: model implements match-3 swap validity; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
ValidSwap did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:25: Unexpected character '#'. 3:270: Unexpected character '#'. 3:329: Unexpected character '#'. 5:87: Unexpected character '#'. 7:15: Unexpected character '#'. 7:28: Char literals must contain exactly one character and close with '''. 9:23: Unexpected character '#'. 11:18: Unexpected character '`'. 11:31: Unexpected character '`'. 13:18: Unexpected character '`'. 13:55: Unexpected character '`'. 15:18: Unexpected character '`'. 15:44: Unexpected character '`'. 17:18: Unexpected character '`'. 17:39: Unexpected character '`'. 19:18: Unexpected character '`'. 19:48: Unexpected character '`'. 21:18: Unexpected character '`'. 21:44: Unexpected character '`'. 23:18: Unexpected character '`'. 23:31: Unexpected character '`'. 25:18: Unexpected character '`'. 25:29: Unexpected character '`'. 27:18: Unexpected character '`'. 27:22: Unexpected character '`'. 29:18: Unexpected character '`'. 29:25: Unexpected character '`'. 31:18: Unexpected character '`'. 31:31: Unexpected character '`'. 33:18: Unexpected character '`'. 33:32: Unexpected character '`'. 39:89: Unexpected character '`'. 39:102: Unexpected character '`'. 39:108: Unexpected character '`'. 39:112: Unexpected character '`'. 39:118: Unexpected character '`'. 39:129: Unexpected character '`'. 39:162: Unexpected character '`'. 39:173: Unexpected character '`'. 41:31: Unexpected character '`'. 41:48: Unexpected character '`'. 41:58: Unexpected character '`'. 41:71: Unexpected character '`'. 41:197: Unexpected character '`'. 41:199: Unexpected character '`'. 43:15: Char literals must contain exactly one character and close with '''. 49:65: Unexpected character '`'. 49:83: Unexpected character '`'. 51:18: Unexpected character '`'. 51:31: Unexpected character '`'. 51:143: Unexpected character '`'. 51:145: Unexpected character '`'. 51:173: Char literals must contain exactly one character and close with '''. 53:18: Unexpected character '`'. 53:29: Unexpected character '`'. 55:9: Unexpected character '#'. 55:30: Unexpected character '`'. 55:41: Unexpected character '`'. 55:63: Unexpected character '`'. 55:68: Unexpected character '`'. 57:9: Unexpected character '#'. 57:30: Unexpected character '`'. 57:34: Unexpected character '`'. 57:86: Unexpected character '`'. 57:90: Unexpected character '`'. 57:106: Unexpected character '`'. 57:133: Unexpected character '`'. 59:9: Unexpected character '#'. 59:30: Unexpected character '`'. 59:43: Unexpected character '`'. 59:303: Unexpected character '`'. 59:316: Unexpected character '`'. 91:17: Unexpected character '#'. 91:19: Unexpected character '`'. 91:56: Unexpected character '`'. 91:88: Unexpected character '`'. 91:109: Unexpected character '`'. 93:9: Unexpected character '#'. 93:21: Unexpected character '`'. 93:40: Unexpected character '`'. 95:9: Unexpected character '#'. 95:21: Unexpected character '`'. 95:42: Unexpected character '`'. 97:9: Unexpected character '#'. 97:21: Unexpected character '`'. 97:46: Unexpected character '`'. 99:9: Unexpected character '#'. 99:21: Unexpected character '`'. 99:32: Unexpected character '`'. 101:9: Unexpected character '#'. 101:21: Unexpected character '`'. 101:29: Unexpected character '`'. 103:9: Unexpected character '#'. 105:9: Unexpected character '#'. 105:21: Unexpected character '`'. 105:27: Unexpected character '`'. 107:9: Unexpected character '#'. 107:21: Unexpected character '`'. 107:34: Unexpected character '`'. 109:9: Unexpected character '#'. 109:21: Too many lexical errors; further errors are not reported.
 Failed IN70 866 OUT33 996
Agentic - Game Backend: Memory match pairs 1 failed
0.0%
 IN35 244 OUT3 097
×
GameBackendMemoryMatchCodeGenCode-gen: model implements memory match; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Pairs did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:5: Expected a name, found end of file.
 Failed IN35 244 OUT3 097
Agentic - Game Backend: Mercy invincibility (i-frames) 1 passed
100.0%
 IN8 800 OUT1 028
✓
GameBackendInvincibilityCodeGenCode-gen: model implements i-frames damage; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 800 OUT1 028
Agentic - Game Backend: Minesweeper (adjacency + flood-fill reveal) 1 failed
0.0%
 IN47 075 OUT5 933
×
GameBackendMinesweeperCodeGenCode-gen: model implements minesweeper adjacency + flood-fill reveal; graded against an exact oracle over fixed + random boards.4 AI queries 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
AdjacentMines did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:27: Unexpected character '#'. 4:30: Unexpected character '✓'. 5:24: Unexpected character '✓'. 6:4: Unexpected character '`'. 6:17: Unexpected character '`'. 6:24: Unexpected character '`'. 6:31: Unexpected character '`'. 6:34: Unexpected character '`'. 6:46: Unexpected character '`'. 6:50: Unexpected character '✓'. 7:4: Unexpected character '`'. 7:14: Unexpected character '`'. 7:21: Unexpected character '`'. 7:28: Unexpected character '`'. 7:31: Unexpected character '`'. 7:39: Unexpected character '`'. 7:42: Unexpected character '`'. 7:49: Unexpected character '`'. 7:53: Unexpected character '✓'. 8:4: Unexpected character '`'. 8:8: Unexpected character '`'. 8:21: Unexpected character '`'. 8:25: Unexpected character '`'. 8:42: Unexpected character '✓'. 9:4: Unexpected character '`'. 9:13: Unexpected character '`'. 9:25: Unexpected character '✓'. 10:4: Unexpected character '`'. 10:7: Unexpected character '`'. 10:29: Unexpected character '✓'. 11:4: Unexpected character '`'. 11:6: Unexpected character '`'. 11:19: Unexpected character '✓'. 11:40: Unexpected character '`'. 11:72: Unexpected character '`'. 12:4: Unexpected character '`'. 12:7: Unexpected character '`'. 12:13: Unexpected character '`'. 12:16: Unexpected character '`'. 12:34: Unexpected character '✓'. 13:5: Unexpected character '`'. 13:7: Unexpected character '`'. 13:18: Unexpected character '✓'. 13:39: Unexpected character '#'. 14:21: Unexpected character '`'. 14:29: Unexpected character '`'. 14:39: Unexpected character '`'. 14:44: Unexpected character '`'. 14:48: Unexpected character '✓'. 15:34: Unexpected character '`'. 15:38: Unexpected character '`'. 15:42: Unexpected character '✓'. 17:42: Unexpected character '#'. 19:57: Unexpected character '`'. 19:61: Unexpected character '`'. 19:91: Unexpected character '`'. 19:121: Unexpected character '`'. 19:128: Unexpected character '`'. 19:144: Unexpected character '`'. 19:158: Unexpected character '`'. 19:162: Unexpected character '`'. 19:260: Unexpected character '`'. 19:275: Unexpected character '`'. 21:16: Unexpected character '`'. 21:29: Unexpected character '`'. 21:86: Char literals must contain exactly one character and close with '''.
 Failed IN47 075 OUT5 933
Agentic - Game Backend: Minimap downsample 1 passed
100.0%
 IN8 776 OUT1 034
✓
GameBackendMinimapCodeGenCode-gen: model implements minimap downsampling; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 776 OUT1 034
Agentic - Game Backend: Minimap marker position 1 failed
0.0%
 IN35 380 OUT2 509
×
GameBackendMinimapMarkerCodeGenCode-gen: model implements minimap marker; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Marker did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 380 OUT2 509
Agentic - Game Backend: N-in-a-row win detection (Connect-4) 1 failed
0.0%
 IN35 560 OUT13 279
×
GameBackendConnect4CodeGenCode-gen: model implements N-in-a-row winner detection; graded vs exact oracle.4 AI queries 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Winner did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 560 OUT13 279
Agentic - Game Backend: Nonogram row validation 1 passed
100.0%
 IN18 352 OUT1 474
✓
GameBackendNonogramCodeGenCode-gen: model implements nonogram validation; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN18 352 OUT1 474
Agentic - Game Backend: OBB collision (rotating rectangles, SAT) 1 failed
0.0%
 IN138 536 OUT44 418
×
GameBackendObbCodeGenCode-gen: model implements ObbOverlap for two oriented rectangles; graded against an exact integer SAT oracle over fixed + random boxes.4 AI queries 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
ObbOverlap did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:1: Unexpected character '#'. 1:5: Unexpected character '#'. 3:3: Unexpected character '#'. 3:29: Unexpected character '`'. 3:48: Unexpected character '#'. 3:391: Char literals must contain exactly one character and close with '''. 5:25: Unexpected character '#'. 5:40: Unexpected character '`'. 5:44: Unexpected character '`'. 5:85: Unexpected character '`'. 5:152: Unexpected character '`'. 9:18: Char literals must contain exactly one character and close with '''. 11:91: Char literals must contain exactly one character and close with '''. 13:41: Char literals must contain exactly one character and close with '''. 30:78: Unexpected character '#'. 32:13: Unexpected character '`'. 32:18: Unexpected character '`'. 36:18: Unexpected character '`'. 36:56: Unexpected character '`'. 36:114: Unexpected character '#'. 36:198: Unexpected character '`'. 36:242: Unexpected character '`'. 38:18: Unexpected character '`'. 38:63: Unexpected character '`'. 40:18: Unexpected character '`'. 40:57: Unexpected character '`'. 44:35: Unexpected character '`'. 44:43: Unexpected character '`'. 44:55: Unexpected character '#'. 44:121: Unexpected character '`'. 44:129: Unexpected character '`'. 46:4: Char literals must contain exactly one character and close with '''. 48:63: Unexpected character '`'. 48:130: Unexpected character '`'. 48:148: Unexpected character '`'. 48:152: Unexpected character '`'.
 Failed IN138 536 OUT44 418
Agentic - Game Backend: Particle lifetime & expiry 1 passed
100.0%
 IN8 838 OUT780
✓
GameBackendParticlesCodeGenCode-gen: model implements particle lifetime functions; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN8 838 OUT780
Agentic - Game Backend: Party XP share 1 passed
100.0%
 IN8 756 OUT633
✓
GameBackendXpShareCodeGenCode-gen: model implements XP share; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN8 756 OUT633
Agentic - Game Backend: Patrol waypoints (ping-pong) 1 failed
0.0%
 IN35 380 OUT6 786
×
GameBackendPatrolCodeGenCode-gen: model implements patrol positioning; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
PatrolPos did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 380 OUT6 786
Agentic - Game Backend: Platformer jump arc 1 failed
0.0%
 IN35 509 OUT32 397
×
GameBackendJumpCodeGenCode-gen: model implements the jump arc; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
JumpHeight did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 509 OUT32 397
Agentic - Game Backend: Point on circle (trig) 1 passed
100.0%
 IN8 833 OUT939
✓
GameBackendCircleMotionCodeGenCode-gen: model implements point-on-circle; graded vs oracle with +/-1 tolerance.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 833 OUT939
Agentic - Game Backend: Poker hand comparison 1 failed
0.0%
 IN56 398 OUT11 945
×
GameBackendPokerCompareCodeGenCode-gen: model compares poker hands; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:VeryHardsource:Ai4Users
Compare did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN56 398 OUT11 945
Agentic - Game Backend: Poker hand ranking 1 passed
100.0%
 IN8 822 OUT2 325
✓
GameBackendPokerCodeGenCode-gen: model implements poker hand-category ranking; graded vs exact oracle.1 AI query 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
 Passed IN8 822 OUT2 325
Agentic - Game Backend: Pop-cap training 1 failed
0.0%
 IN35 292 OUT2 188
×
GameBackendPopCapCodeGenCode-gen: model implements pop-cap training; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Trainable did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 292 OUT2 188
Agentic - Game Backend: Progress bar fill 1 passed
100.0%
 IN18 145 OUT2 166
✓
GameBackendProgressBarCodeGenCode-gen: model implements progress-bar fill; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN18 145 OUT2 166
Agentic - Game Backend: Projectile range (physics/trig) 1 failed
0.0%
 IN35 506 OUT3 820
×
GameBackendProjectileCodeGenCode-gen: model implements projectile range; graded vs oracle with +/-1 tolerance.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
ProjectileRange did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 506 OUT3 820
Agentic - Game Backend: Projectile with ground bounce 1 passed
100.0%
 IN18 014 OUT1 411
✓
GameBackendProjectileCodeGenCode-gen: model implements the projectile sim; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN18 014 OUT1 411
Agentic - Game Backend: Resource gathering 1 passed
100.0%
 IN17 771 OUT694
✓
GameBackendGatherCodeGenCode-gen: model implements resource gathering; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN17 771 OUT694
Agentic - Game Backend: Reversi flip count 1 failed
0.0%
 IN35 572 OUT10 561
×
GameBackendReversiCodeGenCode-gen: model implements Reversi flip count; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
FlipCount did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 572 OUT10 561
Agentic - Game Backend: Rotate 2D vector (trig) 1 failed
0.0%
 IN35 352 OUT6 046
×
GameBackendRotateVectorCodeGenCode-gen: model implements vector rotation; graded vs oracle with +/-1 tolerance.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
RotateVector did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 352 OUT6 046
Agentic - Game Backend: Round-robin deal 1 passed
100.0%
 IN8 761 OUT859
✓
GameBackendDeckDealCodeGenCode-gen: model implements dealing; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN8 761 OUT859
Agentic - Game Backend: RTS resource economy 1 passed
100.0%
 IN27 935 OUT1 286
✓
GameBackendRtsEconomyCodeGenCode-gen: model implements RTS economy; graded vs exact oracle.3 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN27 935 OUT1 286
Agentic - Game Backend: RTS unit counters & damage 1 passed
100.0%
 IN18 253 OUT1 296
✓
GameBackendRtsCodeGenCode-gen: model implements RTS counters + damage; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN18 253 OUT1 296
Agentic - Game Backend: Safest neighbour (danger map) 1 failed
0.0%
 IN58 218 OUT16 732
×
GameBackendDangerMapCodeGenCode-gen: model implements danger-map safest neighbour; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Safest did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:14: Expected '(' to declare a function or '=' to declare a global variable after 'string Safest', found '.'. Global variables must have an initializer.
 Failed IN58 218 OUT16 732
Agentic - Game Backend: Scroll viewport 1 failed
0.0%
 IN35 360 OUT2 066
×
GameBackendScrollViewCodeGenCode-gen: model implements scroll viewport; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Visible did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 360 OUT2 066
Agentic - Game Backend: Set card validity 1 failed
0.0%
 IN54 385 OUT15 952
×
GameBackendSetGameCodeGenCode-gen: model implements Set validity; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
IsSet did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:20: Char literals must contain exactly one character and close with '''. 1:185: Unexpected character '#'. 1:186: Char literals must contain exactly one character and close with '''. 1:193: Unexpected character '`'. 1:194: Char literals must contain exactly one character and close with '''. 1:639: Unexpected character '#'. 1:746: Char literals must contain exactly one character and close with '''. 2:3: Unexpected character '#'. 2:7: Unexpected character '#'. 4:5: Unexpected character '#'. 5:20: Unexpected character '#'. 7:275: Unexpected character '#'. 9:326: Unexpected character '#'. 9:531: Unexpected character '#'. 11:256: Char literals must contain exactly one character and close with '''. 15:20: Unexpected character '#'. 15:55: Unexpected character '`'. 15:73: Unexpected character '`'. 15:151: Unexpected character '#'. 15:409: Char literals must contain exactly one character and close with '''. 17:48: Unexpected character '#'. 17:79: Char literals must contain exactly one character and close with '''. 46:29: Unexpected character '`'. 46:47: Unexpected character '`'. 46:52: Char literals must contain exactly one character and close with '''. 48:7: Unexpected character '`'. 48:17: Unexpected character '`'. 50:7: Unexpected character '`'. 50:17: Unexpected character '`'. 50:40: Unexpected character '`'. 50:44: Unexpected character '`'. 52:7: Unexpected character '`'. 52:31: Unexpected character '`'. 54:7: Unexpected character '`'. 54:10: Unexpected character '`'. 56:7: Unexpected character '`'. 56:35: Unexpected character '`'. 58:21: Unexpected character '`'. 58:23: Unexpected character '`'. 60:7: Unexpected character '`'. 60:52: Unexpected character '`'. 62:7: Unexpected character '`'. 62:33: Unexpected character '`'. 64:155: Char literals must contain exactly one character and close with '''. 68:89: Unexpected character '#'. 70:83: Unexpected character '#'. 72:27: Unexpected character '#'. 72:46: Unexpected character '`'. 72:64: Unexpected character '`'. 72:329: Unexpected character '`'. 72:341: Unexpected character '`'. 72:347: Unexpected character '`'. 72:388: Unexpected character '`'. 72:394: Unexpected character '`'. 72:405: Unexpected character '`'. 72:431: Unexpected character '`'. 72:445: Unexpected character '`'. 72:526: Char literals must contain exactly one character and close with '''. 108:32: Unexpected character '`'. 108:38: Unexpected character '`'. 108:62: Unexpected character '`'. 110:7: Unexpected character '`'. 110:51: Unexpected character '`'. 112:7: Unexpected character '`'. 112:18: Unexpected character '`'. 118:11: Unexpected character '`'. 118:21: Unexpected character '`'. 118:189: Unexpected character '`'. 118:205: Unexpected character '`'. 118:249: Char literals must contain exactly one character and close with '''. 133:141: Char literals must contain exactly one character and close with '''. 211:141: Char literals must contain exactly one character and close with '''. 258:312: Char literals must contain exactly one character and close with '''. 297:50: Unexpected character '#'. 297:158: Unexpected character '`'. 297:161: Unexpected character '`'. 340:134: Unexpected character '`'. 340:151: Unexpected character '`'. 342:18: Unexpected character '`'. 342:50: Unexpected character '`'. 344:131: Unexpected character '`'. 344:142: Unexpected character '`'. 344:169: Char literals must contain exactly one character and close with '''. 354:8: Char literals must contain exactly one character and close with '''. 356:9: Unexpected character '#'. 358:9: Unexpected character '#'. 358:40: Unexpected character '`'. 358:43: Unexpected character '`'. 360:9: Unexpected character '#'. 360:26: Unexpected character '`'. 360:31: Unexpected character '`'. 362:9: Unexpected character '#'. 362:26: Unexpected character '`'. 362:46: Unexpected character '`'. 364:9: Unexpected character '#'. 364:26: Unexpected character '`'. 364:33: Unexpected character '`'. 366:9: Unexpected character '#'. 366:26: Unexpected character '`'. 366: ... (message truncated)
 Failed IN54 385 OUT15 952
Agentic - Game Backend: Shop affordability 1 passed
100.0%
 IN27 486 OUT1 255
✓
GameBackendShopAffordCodeGenCode-gen: model implements shop affordability; graded vs exact oracle.3 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN27 486 OUT1 255
Agentic - Game Backend: Sliding puzzle move 1 passed
100.0%
 IN8 764 OUT1 089
✓
GameBackendSlidePuzzleCodeGenCode-gen: model implements sliding puzzle move; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 764 OUT1 089
Agentic - Game Backend: Snake head movement 1 failed
0.0%
 IN43 052 OUT6 808
×
GameBackendSnakeCodeGenCode-gen: model implements snake head movement; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
NextHead did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:20: Char literals must contain exactly one character and close with '''. 3:39: Char literals must contain exactly one character and close with '''. 5:17: Unexpected character '#'. 7:183: Char literals must contain exactly one character and close with '''. 9:14: Unexpected character '#'. 13:18: Unexpected character '`'. 13:41: Unexpected character '`'. 15:16: Unexpected character '`'. 15:38: Unexpected character '`'. 17:18: Unexpected character '`'. 17:54: Unexpected character '`'. 19:18: Unexpected character '`'. 19:29: Unexpected character '`'. 21:18: Unexpected character '`'. 21:28: Unexpected character '`'. 31:87: Unexpected character '`'. 31:97: Unexpected character '`'. 35:535: Char literals must contain exactly one character and close with '''. 41:14: Unexpected character '#'. 41:21: Char literals must contain exactly one character and close with '''. 43:14: Unexpected character '#'. 43:58: Char literals must contain exactly one character and close with '''. 45:14: Unexpected character '#'. 47:14: Unexpected character '#'. 47:21: Char literals must contain exactly one character and close with '''. 49:14: Unexpected character '#'. 49:21: Char literals must contain exactly one character and close with '''. 51:14: Unexpected character '#'. 51:21: Char literals must contain exactly one character and close with '''. 53:14: Unexpected character '#'. 53:21: Char literals must contain exactly one character and close with '''. 55:14: Unexpected character '#'. 55:21: Char literals must contain exactly one character and close with '''. 57:14: Unexpected character '#'. 57:21: Char literals must contain exactly one character and close with '''. 59:14: Unexpected character '#'. 59:21: Char literals must contain exactly one character and close with '''. 90:153: Char literals must contain exactly one character and close with '''. 94:14: Unexpected character '#'. 94:29: Char literals must contain exactly one character and close with '''. 96:14: Unexpected character '#'. 96:21: Char literals must contain exactly one character and close with '''. 98:14: Unexpected character '#'. 98:21: Char literals must contain exactly one character and close with '''. 100:14: Unexpected character '#'. 100:21: Char literals must contain exactly one character and close with '''. 102:14: Unexpected character '#'. 102:21: Char literals must contain exactly one character and close with '''. 104:14: Unexpected character '#'. 104:21: Char literals must contain exactly one character and close with '''. 106:14: Unexpected character '#'. 119:119: Unexpected character '`'. 119:127: Unexpected character '`'. 121:11: Unexpected character '`'. 121:18: Unexpected character '`'. 175:21: Unexpected character '`'. 175:27: Unexpected character '`'. 183:121: Char literals must contain exactly one character and close with '''. 198:43: Unexpected character '#'. 198:143: Char literals must contain exactly one character and close with '''. 202:14: Unexpected character '#'. 202:21: Char literals must contain exactly one character and close with '''. 204:14: Unexpected character '#'. 204:21: Char literals must contain exactly one character and close with '''. 206:14: Unexpected character '#'. 206:21: Char literals must contain exactly one character and close with '''. 208:12: Unexpected character '`'. 208:26: Unexpected character '`'. 210:34: Unexpected character '`'. 210:47: Unexpected character '`'. 212:15: Unexpected character '`'. 212:19: Unexpected character '`'. 212:69: Char literals must contain exactly one character and close with '''. 214:15: Unexpected character '`'. 214:19: Unexpected character '`'. 214:103: Unexpected character '`'. 214:117: Unexpected character '`'. 214:177: Unexpected character '`'. 214:198: Unexpected character '`'. 224:50: Unexpected character '`'. 224:124: Unexpected character '`'. 226:4: Char literals must contain exactly one character and close with '''.
 Failed IN43 052 OUT6 808
Agentic - Game Backend: Snake tick 1 passed
100.0%
 IN8 921 OUT1 492
✓
GameBackendSnakeCodeGenCode-gen: model implements the Snake tick; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
 Passed IN8 921 OUT1 492
Agentic - Game Backend: Snap to grid 1 passed
100.0%
 IN8 749 OUT482
✓
GameBackendSnapGridCodeGenCode-gen: model implements snap-to-grid; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN8 749 OUT482
Agentic - Game Backend: Sokoban move/push legality 1 passed
100.0%
 IN8 852 OUT889
✓
GameBackendSokobanCodeGenCode-gen: model implements Sokoban move legality; graded vs exact oracle.1 AI query 0 tool callscategory:AgenticCodingcategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 852 OUT889
Agentic - Game Backend: Solitaire tableau move 1 failed
0.0%
 IN35 428 OUT3 401
×
GameBackendSolitaireCodeGenCode-gen: model implements solitaire move legality; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
CanStack did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 428 OUT3 401
Agentic - Game Backend: Spawn scheduler count 1 failed
0.0%
 IN44 451 OUT9 626
×
GameBackendSpawnSchedulerCodeGenCode-gen: model implements spawn scheduler; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
SpawnsBy did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:24: Unexpected character '#'.
 Failed IN44 451 OUT9 626
Agentic - Game Backend: Spawn-wave alive count 1 passed
100.0%
 IN8 779 OUT755
✓
GameBackendSpawnWaveCodeGenCode-gen: model implements spawn-wave alive count; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 779 OUT755
Agentic - Game Backend: Stat level scaling 1 failed
0.0%
 IN40 818 OUT4 091
×
GameBackendLevelScalingCodeGenCode-gen: model implements level scaling; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
ScaledStat did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:1: Expected '{' to start a block, found end of file.
 Failed IN40 818 OUT4 091
Agentic - Game Backend: Steering seek (greedy pursuit) 1 failed
0.0%
 IN54 632 OUT12 329
×
GameBackendSeekCodeGenCode-gen: model implements steering seek; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Seek did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 8:14: Unexpected character '#'. 8:21: Char literals must contain exactly one character and close with '''. 10:27: Unexpected character '`'. 10:64: Unexpected character '`'. 10:93: Unexpected character '`'. 10:95: Unexpected character '`'. 10:114: Unexpected character '`'. 10:125: Unexpected character '`'. 12:27: Unexpected character '`'. 12:49: Unexpected character '`'. 14:27: Unexpected character '`'. 14:37: Unexpected character '`'. 16:27: Unexpected character '`'. 16:39: Unexpected character '`'. 18:27: Unexpected character '`'. 18:43: Unexpected character '`'. 22:9: Unexpected character '#'. 22:47: Char literals must contain exactly one character and close with '''. 26:27: Unexpected character '`'. 26:32: Unexpected character '`'. 28:27: Unexpected character '`'. 28:35: Unexpected character '`'. 32:27: Unexpected character '`'. 32:33: Unexpected character '`'. 34:27: Unexpected character '`'. 34:33: Unexpected character '`'. 40:107: Unexpected character '#'. 42:27: Unexpected character '`'. 42:28: Unexpected character '#'. 42:29: Unexpected character '`'. 42:308: Unexpected character '#'. 44:26: Unexpected character '#'. 44:44: Unexpected character '`'. 44:48: Unexpected character '`'. 46:42: Unexpected character '`'. 46:95: Unexpected character '`'. 46:223: Unexpected character '`'. 46:224: Unexpected character '#'. 46:228: Unexpected character '#'. 46:236: Unexpected character '`'. 46:264: Unexpected character '#'. 48:69: Unexpected character '`'. 48:70: Unexpected character '#'. 48:74: Unexpected character '#'. 48:75: Unexpected character '`'. 48:88: Char literals must contain exactly one character and close with '''. 50:99: Unexpected character '#'. 56:4: Char literals must contain exactly one character and close with '''. 58:11: Unexpected character '`'. 58:29: Unexpected character '`'. 58:35: Unexpected character '`'. 58:44: Unexpected character '`'. 60:11: Unexpected character '`'. 60:60: Unexpected character '`'. 62:25: Unexpected character '`'. 62:38: Unexpected character '`'. 64:59: Char literals must contain exactly one character and close with '''. 66:27: Unexpected character '`'. 66:32: Unexpected character '`'. 68:15: Unexpected character '`'. 68:21: Unexpected character '`'. 70:4: Char literals must contain exactly one character and close with '''. 96:18: Unexpected character '`'. 96:34: Unexpected character '`'. 96:45: Unexpected character '`'. 96:58: Unexpected character '`'. 96:256: Char literals must contain exactly one character and close with '''. 106:71: Unexpected character '`'. 106:79: Unexpected character '`'. 115:11: Unexpected character '`'. 115:90: Unexpected character '`'. 117:15: Unexpected character '`'. 117:16: Unexpected character '#'. 117:17: Unexpected character '`'. 123:4: Char literals must contain exactly one character and close with '''. 125:27: Unexpected character '#'. 125:29: Unexpected character '`'. 125:34: Unexpected character '`'. 125:67: Unexpected character '`'. 125:81: Unexpected character '`'. 125:89: Unexpected character '`'. 125:113: Unexpected character '`'. 125:151: Unexpected character '`'. 125:161: Unexpected character '`'. 125:199: Unexpected character '`'. 125:208: Unexpected character '`'. 127:25: Unexpected character '`'. 127:35: Unexpected character '`'. 129:25: Unexpected character '`'. 129:62: Unexpected character '`'. 129:69: Unexpected character '#'. 129:72: Unexpected character '`'. 129:74: Unexpected character '`'. 131:11: Unexpected character '`'. 131:32: Unexpected character '`'. 135:4: Char literals must contain exactly one character and close with '''. 147:13: Unexpected character '`'. 147:21: Unexpected character '`'. 147:39: Unexpected character '`'. 147:48: Unexpected character '`'. 151:8: Too many lexical errors; further errors are not reported.
 Failed IN54 632 OUT12 329
Agentic - Game Backend: Tactical movement range 1 failed
0.0%
 IN35 544 OUT7 763
×
GameBackendMoveRangeCodeGenCode-gen: model implements movement range; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
Reachable did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 544 OUT7 763
Agentic - Game Backend: Tech tree eligibility 1 failed
0.0%
 IN35 472 OUT5 668
×
GameBackendTechTreeCodeGenCode-gen: model implements tech-tree eligibility; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
CanResearch did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:5: Expected a name, found end of file.
 Failed IN35 472 OUT5 668
Agentic - Game Backend: Tetris line clearing 1 passed
100.0%
 IN8 763 OUT925
✓
GameBackendTetrisCodeGenCode-gen: model implements Tetris line clearing; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 763 OUT925
Agentic - Game Backend: Threat table top target 1 passed
100.0%
 IN8 757 OUT1 233
✓
GameBackendThreatCodeGenCode-gen: model implements threat table; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 757 OUT1 233
Agentic - Game Backend: Tic-tac-toe result 1 failed
0.0%
 IN38 036 OUT5 035
×
GameBackendTicTacToeCodeGenCode-gen: model implements tic-tac-toe result; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Winner did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: Entry point 'Winner' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, SameThree.
 Failed IN38 036 OUT5 035
Agentic - Game Backend: Toroidal wrap (Pac-Man) 1 failed
0.0%
 IN60 766 OUT17 757
×
GameBackendTorusWrapCodeGenCode-gen: model implements toroidal wrap; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
Wrap did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:86: Unexpected character '#'. 4:73: Unexpected character '#'. 4:106: Unexpected character '`'. 4:119: Unexpected character '`'. 4:149: Unexpected character '`'. 4:150: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."? 4:151: Unexpected character '`'. 4:231: Unexpected character '#'. 4:285: Unexpected character '#'. 4:292: Char literals must contain exactly one character and close with '''. 4:374: Unexpected character '`'. 4:376: Unexpected character '`'. 4:418: Unexpected character '#'. 4:425: Char literals must contain exactly one character and close with '''. 4:492: Unexpected character '`'. 4:495: Unexpected character '`'. 4:505: Unexpected character '`'. 4:510: Unexpected character '`'. 4:541: Unexpected character '#'. 4:548: Char literals must contain exactly one character and close with '''. 4:616: Unexpected character '`'. 4:629: Unexpected character '`'. 6:259: Unexpected character '#'.
 Failed IN60 766 OUT17 757
Agentic - Game Backend: Tower targeting (nearest in range) 1 failed
0.0%
 IN35 550 OUT5 436
×
GameBackendTowerTargetCodeGenCode-gen: model implements tower targeting; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
PickTarget did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 550 OUT5 436
Agentic - Game Backend: Transaction ledger 1 failed
0.0%
 IN35 364 OUT3 029
×
GameBackendBankCodeGenCode-gen: model implements the ledger; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
ProcessTransactions did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 364 OUT3 029
Agentic - Game Backend: Trick-taking winner 1 failed
0.0%
 IN65 198 OUT15 803
×
GameBackendTrickWinnerCodeGenCode-gen: model implements trick winner; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
Winner did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:54: Unexpected character '#'. 1:482: Unexpected character '#'. 1:652: Unexpected character '`'. 1:656: Unexpected character '`'. 1:746: Unexpected character '#'. 1:911: Unexpected character '`'. 1:963: Unexpected character '`'. 3:20: Unexpected character '#'. 3:187: Unexpected character '`'. 3:198: Unexpected character '`'. 3:237: Unexpected character '`'. 3:244: Unexpected character '`'. 3:259: Unexpected character '`'. 3:269: Unexpected character '`'. 3:284: Unexpected character '`'. 3:292: Unexpected character '`'. 5:72: Unexpected character '`'. 5:124: Unexpected character '`'. 5:130: Char literals must contain exactly one character and close with '''. 5:239: Char literals must contain exactly one character and close with '''. 5:391: Unexpected character '#'. 7:18: Unexpected character '`'. 7:31: Unexpected character '`'. 9:14: Unexpected character '#'. 9:33: Unexpected character '`'. 9:49: Unexpected character '`'. 9:59: Unexpected character '`'. 9:66: Unexpected character '`'. 11:18: Unexpected character '`'. 11:21: Unexpected character '`'. 13:18: Unexpected character '`'. 13:21: Unexpected character '`'. 15:18: Unexpected character '`'. 15:55: Unexpected character '`'. 19:18: Unexpected character '`'. 19:35: Unexpected character '`'. 21:13: Unexpected character '`'. 21:26: Unexpected character '`'. 21:43: Unexpected character '`'. 21:82: Unexpected character '`'. 47:21: Unexpected character '`'. 47:23: Unexpected character '`'. 49:17: Unexpected character '`'. 49:20: Unexpected character '`'. 49:30: Unexpected character '`'. 49:43: Unexpected character '`'. 51:15: Unexpected character '`'. 51:21: Unexpected character '`'. 51:26: Unexpected character '`'. 51:30: Unexpected character '`'. 51:36: Unexpected character '`'. 51:41: Unexpected character '`'. 51:46: Unexpected character '`'. 51:50: Unexpected character '`'. 53:15: Unexpected character '`'. 53:21: Unexpected character '`'. 55:15: Unexpected character '`'. 55:28: Unexpected character '`'. 57:21: Unexpected character '`'. 57:27: Unexpected character '`'. 57:58: Unexpected character '`'. 57:69: Unexpected character '`'. 57:74: Char literals must contain exactly one character and close with '''. 61:15: Unexpected character '`'. 61:21: Unexpected character '`'. 63:15: Unexpected character '`'. 63:21: Unexpected character '`'. 65:15: Unexpected character '`'. 65:21: Unexpected character '`'. 67:15: Unexpected character '`'. 67:21: Unexpected character '`'. 69:15: Unexpected character '`'. 69:21: Unexpected character '`'. 71:15: Unexpected character '`'. 71:20: Unexpected character '`'. 73:15: Unexpected character '`'. 73:21: Unexpected character '`'. 75:15: Unexpected character '`'. 75:21: Unexpected character '`'. 75:35: Unexpected character '`'. 75:40: Unexpected character '`'. 77:15: Unexpected character '`'. 77:21: Unexpected character '`'. 87:18: Unexpected character '`'. 89:18: Unexpected character '`'. 89:24: Unexpected character '`'. 93:18: Unexpected character '`'. 93:41: Unexpected character '`'. 95:13: Unexpected character '`'. 95:23: Unexpected character '`'. 97:13: Unexpected character '`'. 97:20: Unexpected character '`'. 97:26: Unexpected character '`'. 97:33: Unexpected character '`'. 99:13: Unexpected character '`'. 99:26: Unexpected character '`'. 99:37: Unexpected character '`'. 99:67: Unexpected character '`'. 99:138: Unexpected character '`'. 99:142: Unexpected character '`'. 101:15: Too many lexical errors; further errors are not reported.
 Failed IN65 198 OUT15 803
Agentic - Game Backend: Turn-based duel resolution 1 passed
100.0%
 IN8 835 OUT1 309
✓
GameBackendCombatDuelCodeGenCode-gen: model implements the duel resolver; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN8 835 OUT1 309
Agentic - Game Backend: Typed armor mitigation 1 failed
0.0%
 IN35 300 OUT7 491
×
GameBackendArmorCodeGenCode-gen: model implements armor mitigation; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
TotalDamage did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN35 300 OUT7 491
Agentic - Game Backend: UI menu (wrap/clamp/paginate) 1 failed
0.0%
 IN41 388 OUT3 489
×
GameBackendUiMenuCodeGenCode-gen: model implements the UI/menu helpers; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
MenuMove did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: Entry point 'MenuMove' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, SumValues.
 Failed IN41 388 OUT3 489
Agentic - Game Backend: Uno playable cards 1 passed
100.0%
 IN8 787 OUT904
✓
GameBackendUnoCodeGenCode-gen: model implements Uno playability; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN8 787 OUT904
Agentic - Game Backend: Velocity reflection 1 failed
0.0%
 IN35 308 OUT1 491
×
GameBackendReflectCodeGenCode-gen: model implements reflection; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
Reflect did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 308 OUT1 491
Agentic - Game Backend: Wall-slide fall 1 passed
100.0%
 IN27 300 OUT1 180
✓
GameBackendWallSlideCodeGenCode-gen: model implements wall slide; graded vs exact oracle.3 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN27 300 OUT1 180
Agentic - Game Backend: War card compare 1 passed
100.0%
 IN40 623 OUT3 726
✓
GameBackendWarCodeGenCode-gen: model implements War comparison; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN40 623 OUT3 726
Agentic - Game Backend: Weighted loot table 1 passed
100.0%
 IN8 792 OUT954
✓
GameBackendLootTableCodeGenCode-gen: model implements weighted loot pick; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Easysource:Ai4Users
 Passed IN8 792 OUT954
Agentic - Game Backend: Weighted terrain pathfinding 1 passed
100.0%
 IN8 812 OUT3 174
✓
GameBackendWeightedPathCodeGenCode-gen: model implements weighted grid pathfinding; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
 Passed IN8 812 OUT3 174
Agentic - Game Backend: Word-wrap line count 1 failed
0.0%
 IN35 240 OUT4 521
×
GameBackendTooltipWrapCodeGenCode-gen: model implements word-wrap line count; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
LineCount did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 240 OUT4 521
Agentic - Game Backend: XP to character level 1 passed
100.0%
 IN18 897 OUT2 915
✓
GameBackendXpLevelCodeGenCode-gen: model implements XP-to-level; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:GameDevlevel:Mediumsource:Ai4Users
 Passed IN18 897 OUT2 915
Agentic - Idempotency (poison-retry / no double charge) 1 passed
100.0%
 IN19 184 OUT870
✓
AgenticIdempotentReconciliationPoison-retry hazard: a charge returns a transient failure AFTER the effect applied; blind retry double-charges. Correct agent reconciles via status before retrying.1 AI query 9 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN19 184 OUT870
Agentic - Irreversible Commit (measure twice) 1 passed
100.0%
 IN21 396 OUT871
✓
AgenticIrreversibleCommitSafetyIrreversible commit under prerequisites: the agent must consult all sources, resolve the dispute and apply the pending credit, THEN finalize. Finalizing early commits a permanent bad state.1 AI query 9 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN21 396 OUT871
Agentic - Least Privilege (exact grants) 1 passed
100.0%
 IN12 143 OUT469
✓
AgenticLeastPrivilegeGrantLeast-privilege grant: provision exactly the documented permission set. Missing one fails the job; granting any extra fails the least-privilege audit. Both are caught by the strict check.1 AI query 6 tool callscategory:AgenticToolUselevel:Hardsource:Ai4Users
 Passed IN12 143 OUT469
Agentic - Ledger (simulated DB read-modify-write) 1 failed
0.0%
 IN5 411 OUT1 209
×
AgenticLedgerReadModifyWriteSimulated-DB read-modify-write over an account store: apply chained transfers in order, skipping overdrafts, preserving money conservation. Stale reads or reordering fail the grade.1 AI query 2 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: actual == expected Context: Account alice ended at 1000 but expected 0 (in-order apply with overdraft-skip). Values: actual = 1000 expected = 0
 Failed IN5 411 OUT1 209
Agentic - Ledger variants (5 RMW cases) 5 passed
100.0%
 IN195 023 OUT9 728
✓
LedgerVariantEarlyChainLedger RMW variant v2 (4 accounts, early chain then a skip).1 AI query 15 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN66 934 OUT2 734
✓
LedgerVariantFiveAccountsLedger RMW variant v4 (5 accounts, two chains + a skip).1 AI query 8 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN13 757 OUT1 030
✓
LedgerVariantLongChainLedger RMW variant v5 (6 accounts, long dependent chain).1 AI query 9 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN15 254 OUT1 641
✓
LedgerVariantSmallChainLedger RMW variant v1 (4 accounts, chained batch, one overdraft-skip).1 AI query 16 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN57 038 OUT2 463
✓
LedgerVariantTightMarginsLedger RMW variant v3 (3 accounts, tight overdraft margins).1 AI query 13 tool callscategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
 Passed IN42 040 OUT1 860
Agentic - Lock Ordering (deadlock avoidance) 1 failed
0.0%
 IN5 762 OUT982
×
AgenticLockOrderingDeadlockAvoidanceDeadlock avoidance: acquire each operation's locks in canonical order; the listed order is a trap. Any out-of-order acquire, missed operation, or leftover held lock fails.1 AI query 2 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("op.done." + op, false) Context: Operation not performed: op_post
 Failed IN5 762 OUT982
Agentic - Lock Ordering variants (5 cases) 1 passed4 failed
20.0%
 IN158 723 OUT6 114
×
LockVariantFiveResFourOps1 AI query 7 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("op.done." + op, false) Context: Operation not performed: op_b
 Failed IN20 733 OUT1 028
×
LockVariantFourResThreeOps1 AI query 2 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("op.done." + op, false) Context: Operation not performed: op_post
 Failed IN5 736 OUT1 077
✓
LockVariantHeavyReuse1 AI query 23 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN120 801 OUT2 624
×
LockVariantReverseListed1 AI query 2 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("op.done." + op, false) Context: Operation not performed: op_x
 Failed IN5 736 OUT649
×
LockVariantThreeLockOp1 AI query 2 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("op.done." + op, false) Context: Operation not performed: op_post
 Failed IN5 717 OUT736
Agentic - Migration (cascading rollback + reapply) 1 failed
0.0%
 IN8 979 OUT723
×
AgenticCascadingMigrationRollbackMulti-stage agentic migration with a happy-path trap: naive in-order apply passes each step but leaves an inconsistent backfill; recovery needs dependency-aware rollback + reapply.1 AI query 5 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: passed Context: Agent did not reach a passing migration health check (consistent, all migrations applied). Values: passed = false
 Failed IN8 979 OUT723
Agentic - Requirements (detect contradiction) 1 passed
100.0%
 IN7 734 OUT694
✓
AgenticDetectConflictingRequirementsRequirements-analysis: find the one contradictory requirement pair (deterministic; pinned by stated latency numbers) among consistent red herrings.1 AI query 2 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN7 734 OUT694
Agentic - Retry/Circuit-breaker (transient vs hard-down) 1 failed
0.0%
 IN5 298 OUT403
×
AgenticRetryCircuitBreakerRetry/circuit-breaker discipline: persist through a transient failure (svc_a succeeds on attempt 3) but stop and open the circuit on a hard-down service (svc_b), without exceeding the retry budget.1 AI query 2 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: Globals.Get<bool>("res.success.svc_a", false) Context: svc_a never succeeded (agent gave up before the transient failure cleared on attempt 3).
 Failed IN5 298 OUT403
Agentic - Saga (compensation / distributed rollback) 1 passed
100.0%
 IN21 994 OUT1 173
✓
AgenticSagaCompensationDistributed-transaction saga where the final step fails permanently; the only consistent outcome is full compensation of earlier steps in reverse order.1 AI query 6 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN21 994 OUT1 173
Agentic - Saga variants (5 compensation cases) 4 passed1 failed
80.0%
 IN91 763 OUT2 682
✓
SagaVariantCommitAllSucceed3-step saga where every step succeeds -> COMMITTED; the agent must NOT over-compensate a success.1 AI query 4 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN14 095 OUT424
×
SagaVariantFailStep2Of33-step saga where step 2 fails permanently -> compensate only step 1 (aborted); never reaches step 3.1 AI query 2 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: !eff Context: Step 1 left in effect; a failed saga must be fully compensated (aborted). Values: eff = true
 Failed IN7 719 OUT178
✓
SagaVariantFailStep3Of33-step saga where the final step (3) fails permanently -> compensate steps 1-2 (aborted).1 AI query 6 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN17 551 OUT550
✓
SagaVariantFailStep3Of44-step saga where step 3 fails permanently -> compensate steps 1-2 (aborted); step 4 never runs.1 AI query 7 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN24 088 OUT712
✓
SagaVariantFailStep4Of44-step saga where the final step (4) fails permanently -> compensate steps 1-3 (aborted).1 AI query 8 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN28 310 OUT818
Agentic - Schema Compat (expand-contract) 1 passed
100.0%
 IN20 303 OUT805
✓
AgenticBackwardCompatibleMigrationBackward-compatible (expand-contract) schema migration: add nullable -> backfill -> mark NOT NULL. Any other order permanently breaks compatibility; graded on the whole path, not just the end state.1 AI query 6 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN20 303 OUT805
Agentic - Scratchpad (self-repair, Dyck, tool-decision) 5 passed2 failed
71.4%
 IN161 292 OUT15 439
×
AgenticCongruentWindowsRepairAgentic coding: write, execute, inspect feedback, repair, retry.1 AI query 0 tool callscategory:AgenticToolUsecategory:CodeGenerationlevel:Hardsource:Ai4Users
Assertion expression evaluated to false: passed Context: Agent failed to produce a passing implementation within 10 attempts. Values: passed = false
 Failed IN10 553 OUT2 939
✓
AgenticDyckExecutableScratchpadDyck completion using model-written executable SC# scratchpad.1 AI query 3 tool callscategory:AgenticReasoningcategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN47 281 OUT4 258
✓
ToolDecisionEasyArithmeticTool-decision calibration: trivial arithmetic.1 AI query 0 tool callscategory:AgenticReasoningcategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN10 556 OUT67
✓
ToolDecisionExclusiveMultiplesTool-decision calibration: moderate exact counting.1 AI query 0 tool callscategory:AgenticReasoningcategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN10 578 OUT400
✓
ToolDecisionFibonacciModuloTool-decision calibration: exact larger arithmetic.1 AI query 1 tool callcategory:AgenticReasoningcategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN21 456 OUT514
✓
ToolDecisionLongDyckTool-decision calibration: long Dyck symbolic tracking.1 AI query 1 tool callcategory:AgenticReasoningcategory:CodeGenerationlevel:VeryHardsource:Ai4Users
 Passed IN26 831 OUT5 788
×
ToolDecisionSubsetCountTool-decision calibration: exact subset counting.1 AI query 2 tool callscategory:AgenticReasoningcategory:CodeGenerationlevel:Hardsource:Ai4Users
Assertion expression evaluated to false: answer != "" Context: ToolDecisionSubsetCount: missing ANSWER marker. Values: answer = ""
 Failed IN34 037 OUT1 473
Agentic - Secret Rotation (zero-downtime ordering) 1 passed
100.0%
 IN14 791 OUT755
✓
AgenticZeroDowntimeSecretRotationZero-downtime secret rotation: safe order is add-new -> switch-consumers -> revoke-old. Any other order opens a downtime window (permanently flagged). Graded on the whole path, not just end state.1 AI query 4 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN14 791 OUT755
Agentic - StatefulOps (incident recovery, concurrency replan) 1 passed1 failed
50.0%
 IN61 399 OUT2 942
×
AgenticCheckoutIncidentRecoveryStateful agentic incident recovery: inspect, plan, modify, restart, verify and repair.1 AI query 5 tool callscategory:AgenticToolUselevel:Hardsource:Ai4Users
Assertion expression evaluated to false: passed Context: Agent did not restore the checkout environment and obtain a passing verification. Values: passed = false
 Failed IN12 966 OUT978
✓
AgenticConcurrentDeploymentRecoveryStateful agentic recovery with optimistic concurrency, hidden concurrent mutation, deterministic verification and up to 10 conversational recovery attempts.3 AI queries 9 tool callscategory:AgenticToolUselevel:VeryHardsource:Ai4Users
 Passed IN48 433 OUT1 964
Agentic - Test Authoring (write a test that catches the bug) 1 passed
100.0%
 IN11 537 OUT803
✓
AgenticWriteBugCatchingTestTest authoring: submit a single test case that is correct per the contract AND catches the bug in the current (upper-bound-missing) Clamp implementation.1 AI query 5 tool callscategory:AgenticToolUsecategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN11 537 OUT803
Agentic - Topological Planning (deps + prereq) 1 failed
0.0%
 IN5 108 OUT484
×
AgenticTopologicalPipelineTopological execution of a task DAG with one unmet external prerequisite; naive listed-order runs fail and the missing credential must be provisioned before its dependent task.1 AI query 1 tool callcategory:AgenticToolUsecategory:Reasoninglevel:VeryHardsource:Ai4Users
Assertion expression evaluated to false: TaskDone(id) Context: Task not completed: t5_integration_tests (respect dependencies and provision the external prerequisite).
 Failed IN5 108 OUT484
Algorithms - 0/1 Knapsack (dynamic programming) 1 passed
100.0%
 IN8 796 OUT1 090
✓
KnapsackCodeGenCode-gen: model implements 0/1 knapsack; graded against an exact DP oracle over fixed + random item sets.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 796 OUT1 090
Algorithms - 1D elastic collision 1 failed
0.0%
 IN40 791 OUT3 622
×
ElasticCollision1DCodeGenCode-gen: model implements 1D elastic collision; graded vs oracle within tolerance.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
Collide did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:156: Char literals must contain exactly one character and close with '''. 4:28: Unexpected character '#'. 6:22: Unexpected character '`'. 6:77: Unexpected character '`'. 6:91: Unexpected character '`'. 6:93: Unexpected character '`'. 8:169: Char literals must contain exactly one character and close with '''. 10:32: Unexpected character '`'. 10:40: Unexpected character '`'. 15:4: Unexpected character '`'. 15:16: Unexpected character '`'. 15:35: Unexpected character '`'. 15:40: Unexpected character '`'. 19:79: Unexpected character '`'. 19:87: Unexpected character '`'. 19:129: Char literals must contain exactly one character and close with '''. 21:126: Unexpected character '`'. 21:153: Unexpected character '`'. 23:139: Unexpected character '`'. 23:141: Unexpected character '`'. 23:250: Unexpected character '`'. 23:252: Unexpected character '`'. 25:64: Char literals must contain exactly one character and close with '''. 27:107: Unexpected character '`'. 27:121: Unexpected character '`'. 29:48: Unexpected character '`'. 29:52: Unexpected character '`'. 29:230: Unexpected character '`'. 29:285: Unexpected character '\'. 29:288: Unexpected character '`'.
 Failed IN40 791 OUT3 622
Algorithms - All-pairs shortest path pair 1 passed
100.0%
 IN48 044 OUT7 677
✓
FloydWarshallPairCodeGenCode-gen: model implements all-pairs shortest path query; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN48 044 OUT7 677
Algorithms - Balanced brackets (stack) 1 passed
100.0%
 IN8 767 OUT799
✓
ValidBracketsCodeGenCode-gen: model implements bracket validation; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN8 767 OUT799
Algorithms - Bellman-Ford shortest path (neg weights/cycles) 1 failed
0.0%
 IN50 442 OUT8 554
×
BellmanCodeGenCode-gen: model implements Bellman-Ford; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
ShortestPath did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:141: Unexpected character '→'. 1:144: Unexpected character '#'. 5:34: Unexpected character '#'. 7:26: Unexpected character '#'. 7:71: Char literals must contain exactly one character and close with '''. 9:7: Unexpected character '`'. 9:13: Unexpected character '`'. 10:4: Unexpected character '`'. 10:12: Unexpected character '`'. 11:4: Unexpected character '`'. 11:28: Unexpected character '`'. 11:38: Unexpected character '`'. 11:51: Unexpected character '`'. 12:4: Unexpected character '`'. 12:20: Unexpected character '`'. 12:26: Unexpected character '`'. 12:43: Unexpected character '`'. 13:4: Unexpected character '`'. 13:12: Unexpected character '`'. 14:4: Unexpected character '`'. 14:15: Unexpected character '`'. 15:4: Unexpected character '`'. 15:16: Unexpected character '`'. 16:4: Unexpected character '`'. 16:21: Unexpected character '`'. 17:4: Unexpected character '`'. 17:20: Unexpected character '`'. 19:5: Unexpected character '`'. 19:8: Unexpected character '`'. 19:14: Unexpected character '`'. 19:17: Unexpected character '`'. 20:5: Unexpected character '`'. 20:11: Unexpected character '`'. 20:17: Unexpected character '`'. 20:26: Unexpected character '`'. 21:5: Unexpected character '`'. 21:13: Unexpected character '`'. 22:5: Unexpected character '`'. 22:9: Unexpected character '`'. 23:23: Unexpected character '`'. 23:28: Unexpected character '`'. 23:31: Unexpected character '`'. 23:37: Unexpected character '`'. 24:28: Unexpected character '`'. 24:31: Unexpected character '`'. 25:5: Unexpected character '`'. 25:7: Unexpected character '`'. 28:44: Unexpected character '`'. 28:58: Unexpected character '`'. 29:15: Unexpected character '`'. 29:17: Unexpected character '`'. 29:26: Unexpected character '`'. 29:35: Unexpected character '`'. 30:6: Unexpected character '`'. 30:10: Unexpected character '`'. 33:38: Unexpected character '#'. 35:54: Unexpected character '`'. 35:55: Unexpected character '→'. 35:56: Unexpected character '`'. 37:34: Unexpected character '#'. 37:44: Unexpected character '`'. 37:49: Unexpected character '`'. 37:64: Char literals must contain exactly one character and close with '''. 39:18: Unexpected character '`'. 39:20: Unexpected character '`'. 39:62: Unexpected character '`'. 39:71: Unexpected character '`'. 39:86: Unexpected character '`'. 39:107: Unexpected character '`'. 39:172: Unexpected character '`'. 39:199: Unexpected character '`'. 41:60: Unexpected character '`'. 41:78: Unexpected character '`'. 41:135: Char literals must contain exactly one character and close with '''. 43:136: Unexpected character '`'. 43:156: Unexpected character '`'.
 Failed IN50 442 OUT8 554
Algorithms - Binary Gray code 1 passed
100.0%
 IN17 738 OUT414
✓
GrayCodeCodeGenCode-gen: model implements Gray code; graded vs exact oracle.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN17 738 OUT414
Algorithms - Bipartite graph check 1 failed
0.0%
 IN73 133 OUT26 325
×
BipartiteCodeGenCode-gen: model implements bipartite check; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
IsBipartite did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:20: Char literals must contain exactly one character and close with '''. 1:231: Unexpected character '#'. 1:232: Char literals must contain exactly one character and close with '''. 1:239: Unexpected character '`'. 1:240: Char literals must contain exactly one character and close with '''. 3:15: Unexpected character '#'. 4:3: Unexpected character '`'. 4:21: Unexpected character '`'. 5:3: Unexpected character '`'. 5:16: Unexpected character '`'. 6:3: Unexpected character '`'. 6:13: Unexpected character '`'. 7:3: Unexpected character '`'. 7:31: Unexpected character '`'. 8:3: Unexpected character '`'. 8:24: Unexpected character '`'. 9:3: Unexpected character '`'. 9:19: Unexpected character '`'. 9:158: Char literals must contain exactly one character and close with '''. 9:528: Unexpected character '#'. 9:630: Unexpected character '`'. 9:640: Unexpected character '`'. 9:700: Unexpected character '`'. 9:716: Unexpected character '`'. 9:747: Unexpected character '`'. 9:774: Unexpected character '`'. 9:803: Unexpected character '`'. 9:813: Unexpected character '`'. 9:950: Unexpected character '`'. 9:965: Unexpected character '`'. 9:968: Unexpected character '`'. 9:981: Unexpected character '`'. 9:1055: Unexpected character '`'. 9:1070: Unexpected character '`'. 9:1073: Unexpected character '`'. 9:1088: Unexpected character '`'. 9:1091: Unexpected character '`'. 9:1106: Unexpected character '`'. 9:1410: Char literals must contain exactly one character and close with '''. 11:60: Unexpected character '`'. 11:65: Unexpected character '`'. 11:71: Unexpected character '`'. 11:74: Unexpected character '`'. 11:87: Unexpected character '`'. 11:107: Unexpected character '`'. 11:120: Unexpected character '`'. 11:136: Unexpected character '`'. 11:155: Unexpected character '`'. 11:172: Unexpected character '`'. 11:196: Unexpected character '`'. 11:230: Unexpected character '`'. 11:243: Unexpected character '`'. 11:261: Unexpected character '`'. 11:274: Unexpected character '`'. 11:290: Unexpected character '`'. 15:79: Unexpected character '`'. 17:18: Unexpected character '`'. 17:23: Unexpected character '`'. 17:80: Unexpected character '`'. 17:85: Unexpected character '`'. 17:123: Unexpected character '`'. 17:160: Unexpected character '`'. 19:18: Unexpected character '`'. 19:36: Unexpected character '`'. 19:128: Char literals must contain exactly one character and close with '''. 21:18: Unexpected character '`'. 21:31: Unexpected character '`'. 21:94: Unexpected character '`'. 21:112: Unexpected character '`'. 21:127: Unexpected character '`'. 21:135: Unexpected character '`'. 21:179: Unexpected character '#'. 21:212: Unexpected character '#'. 21:245: Unexpected character '`'. 21:300: Unexpected character '`'. 21:338: Unexpected character '`'. 21:360: Unexpected character '`'. 21:442: Unexpected character '`'. 21:446: Unexpected character '`'. 23:18: Unexpected character '`'. 23:42: Unexpected character '`'. 25:18: Unexpected character '`'. 25:37: Unexpected character '`'. 27:21: Unexpected character '`'. 27:27: Unexpected character '`'. 29:21: Unexpected character '`'. 29:23: Unexpected character '`'. 31:21: Unexpected character '`'. 31:23: Unexpected character '`'. 43:21: Unexpected character '`'. 43:23: Unexpected character '`'. 47:27: Unexpected character '`'. 47:41: Unexpected character '`'. 47:47: Unexpected character '`'. 47:52: Unexpected character '`'. 49:27: Unexpected character '`'. 49:46: Unexpected character '`'. 51:14: Unexpected character '#'. 51:33: Unexpected character '`'. 51:53: Unexpected character '`'. 53:11: Too many lexical errors; further errors are not reported.
 Failed IN73 133 OUT26 325
Algorithms - Bit tricks (popcount/pow2/lowbit) 1 failed
0.0%
 IN8 784 OUT810
×
BitTricksCodeGenCode-gen: model implements bit tricks; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
IsPowerOfTwo did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: Entry point 'IsPowerOfTwo' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, PopCount.
 Failed IN8 784 OUT810
Algorithms - Caesar cipher 1 failed
0.0%
 IN35 316 OUT3 991
×
CaesarCodeGenCode-gen: model implements the Caesar cipher; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Caesar did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 316 OUT3 991
Algorithms - Catalan number 1 passed
100.0%
 IN8 755 OUT956
✓
CatalanCodeGenCode-gen: model implements Catalan number; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 755 OUT956
Algorithms - Chinese remainder theorem 1 passed
100.0%
 IN42 440 OUT11 222
✓
CrtCodeGenCode-gen: model implements CRT; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN42 440 OUT11 222
Algorithms - Circle-circle MTV 1 failed
0.0%
 IN63 687 OUT20 647
×
CircleMtvCodeGenCode-gen: model implements circle-circle MTV; graded vs oracle within tolerance.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Mtv did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:137: Unexpected character '#'. 1:235: Unexpected character '#'. 1:457: Unexpected character '#'. 3:225: Char literals must contain exactly one character and close with '''. 3:429: Unexpected character '`'. 3:447: Unexpected character '`'. 5:21: Unexpected character '#'. 9:278: Char literals must contain exactly one character and close with '''. 11:34: Unexpected character '#'. 11:53: Unexpected character '`'. 11:71: Unexpected character '`'. 13:32: Unexpected character '`'. 13:33: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."? 13:34: Unexpected character '`'. 13:251: Unexpected character '`'. 13:258: Unexpected character '`'. 17:15: Char literals must contain exactly one character and close with '''. 19:28: Unexpected character '#'. 19:42: Char literals must contain exactly one character and close with '''. 28:3: Unexpected character '#'. 30:18: Unexpected character '`'. 30:67: Unexpected character '`'. 32:18: Unexpected character '`'. 32:35: Unexpected character '`'. 34:18: Unexpected character '`'. 34:50: Unexpected character '`'. 34:57: Unexpected character '`'. 34:75: Unexpected character '`'. 36:18: Unexpected character '`'. 36:42: Unexpected character '`'. 38:18: Unexpected character '`'. 38:40: Unexpected character '`'. 42:25: Unexpected character '`'. 42:32: Unexpected character '`'. 42:42: Unexpected character '`'. 42:46: Unexpected character '`'. 44:18: Unexpected character '`'. 44:48: Unexpected character '`'. 46:18: Unexpected character '`'. 46:30: Unexpected character '`'. 48:18: Unexpected character '`'. 48:30: Unexpected character '`'. 50:18: Unexpected character '`'. 50:30: Unexpected character '`'. 52:18: Unexpected character '`'. 52:55: Unexpected character '`'. 54:9: Unexpected character '#'. 56:9: Unexpected character '#'. 56:39: Unexpected character '`'. 56:55: Unexpected character '`'. 58:9: Unexpected character '#'. 58:21: Unexpected character '`'. 58:28: Unexpected character '`'. 60:9: Unexpected character '#'. 60:19: Unexpected character '`'. 60:23: Unexpected character '`'. 62:9: Unexpected character '#'. 62:18: Unexpected character '`'. 62:25: Unexpected character '`'. 64:9: Unexpected character '#'. 64:18: Unexpected character '`'. 64:28: Unexpected character '`'. 66:9: Unexpected character '#'. 66:18: Unexpected character '`'. 66:29: Unexpected character '`'. 68:9: Unexpected character '#'. 68:18: Unexpected character '`'. 68:26: Unexpected character '`'. 72:32: Unexpected character '`'. 72:91: Unexpected character '`'. 74:24: Unexpected character '`'. 74:36: Unexpected character '`'. 76:7: Unexpected character '`'. 76:19: Unexpected character '`'. 78:7: Unexpected character '`'. 78:14: Unexpected character '`'. 80:18: Unexpected character '`'. 80:26: Unexpected character '`'. 82:7: Unexpected character '`'. 82:18: Unexpected character '`'. 84:7: Unexpected character '`'. 84:25: Unexpected character '`'. 84:43: Unexpected character '`'. 84:54: Unexpected character '`'. 86:7: Unexpected character '`'. 86:18: Unexpected character '`'. 92:300: Unexpected character '#'. 94:100: Unexpected character '#'. 94:191: Unexpected character '`'. 94:194: Unterminated string literal. Strings must be closed with '"' on the same line. 100:20: Unexpected character '#'. 100:37: Unexpected character '`'. 100:44: Unexpected character '`'. 102:9: Unexpected character '#'. 102:21: Unexpected character '`'. 102:24: Unexpected character '`'. 102:63: Unexpected character '`'. 102:66: Unexpected character '`'. 104:9: Unexpected character '#'. 106:9: Unexpected character '#'. 106:21: Too many lexical errors; further errors are not reported.
 Failed IN63 687 OUT20 647
Algorithms - Coin change combinations 1 passed
100.0%
 IN29 460 OUT3 034
✓
CoinCombosCodeGenCode-gen: model implements coin combinations; graded vs exact oracle.3 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN29 460 OUT3 034
Algorithms - Collatz step count 1 passed
100.0%
 IN8 766 OUT675
✓
CollatzCodeGenCode-gen: model implements Collatz step count; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN8 766 OUT675
Algorithms - Connected components (union-find) 1 failed
0.0%
 IN42 627 OUT5 087
×
UnionFindCodeGenCode-gen: model implements connected-component counting; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
CountComponents did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:26: Unexpected character '#'. 2:58: Char literals must contain exactly one character and close with '''. 4:3: Unexpected character '`'. 4:13: Unexpected character '`'. 5:3: Unexpected character '`'. 5:25: Unexpected character '`'. 6:3: Unexpected character '`'. 6:15: Unexpected character '`'. 7:3: Unexpected character '`'. 7:19: Unexpected character '`'. 8:3: Unexpected character '`'. 8:15: Unexpected character '`'. 9:3: Unexpected character '`'. 9:11: Unexpected character '`'. 10:3: Unexpected character '`'. 10:9: Unexpected character '`'. 11:3: Unexpected character '`'. 11:7: Unexpected character '`'. 12:3: Unexpected character '`'. 12:11: Unexpected character '`'. 15:3: Unexpected character '`'. 15:15: Unexpected character '`'. 16:3: Unexpected character '`'. 16:19: Unexpected character '`'. 17:3: Unexpected character '`'. 17:14: Unexpected character '`'. 19:83: Unexpected character '#'. 21:233: Unexpected character '`'. 21:238: Unexpected character '`'. 21:244: Unexpected character '`'. 21:250: Unexpected character '`'. 21:301: Unexpected character '`'. 21:317: Unexpected character '`'. 21:323: Char literals must contain exactly one character and close with '''.
 Failed IN42 627 OUT5 087
Algorithms - Container with most water 1 passed
100.0%
 IN8 744 OUT921
✓
ContainerWaterCodeGenCode-gen: model implements container with most water; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 744 OUT921
Algorithms - Count anagram groups 1 failed
0.0%
 IN35 370 OUT6 114
×
AnagramGroupsCodeGenCode-gen: model implements anagram grouping; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
GroupCount did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 370 OUT6 114
Algorithms - Count grid color regions 1 passed
100.0%
 IN8 747 OUT1 552
✓
CountComponentsGridCodeGenCode-gen: model implements region counting; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 747 OUT1 552
Algorithms - Count islands 1 passed
100.0%
 IN8 780 OUT1 525
✓
CountIslandsCodeGenCode-gen: model implements island counting; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 780 OUT1 525
Algorithms - Count overlapping occurrences 1 failed
0.0%
 IN35 196 OUT2 957
×
RabinKarpCountCodeGenCode-gen: model implements overlapping occurrence count; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Count did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:5: Expected a name, found end of file.
 Failed IN35 196 OUT2 957
Algorithms - Count primes (Sieve of Eratosthenes) 1 passed
100.0%
 IN8 712 OUT1 251
✓
CountPrimesCodeGenCode-gen: model implements prime counting; graded vs exact sieve oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN8 712 OUT1 251
Algorithms - Cumulative distribution 1 passed
100.0%
 IN8 764 OUT1 299
✓
CdfCodeGenCode-gen: model implements CDF; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 764 OUT1 299
Algorithms - Dijkstra shortest path (weighted graph) 1 passed
100.0%
 IN8 816 OUT1 563
✓
DijkstraCodeGenCode-gen: model implements Dijkstra shortest distance; graded against an exact oracle over fixed + random weighted graphs.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 816 OUT1 563
Algorithms - Directed cycle detection 1 failed
0.0%
 IN35 356 OUT6 884
×
DirectedCycleCodeGenCode-gen: model implements cycle detection; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
HasCycle did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 356 OUT6 884
Algorithms - Distinct subsequences 1 failed
0.0%
 IN35 180 OUT5 036
×
DistinctSubseqCodeGenCode-gen: model implements distinct subsequences; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:VeryHardsource:Ai4Users
Count did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:5: Expected a name, found end of file.
 Failed IN35 180 OUT5 036
Algorithms - Divisor count 1 passed
100.0%
 IN8 717 OUT838
✓
DivisorCountCodeGenCode-gen: model implements divisor count; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN8 717 OUT838
Algorithms - Downsample by block average 1 failed
0.0%
 IN71 061 OUT21 905
×
ResampleCodeGenCode-gen: model implements block-average downsampling; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Resample did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 4:27: Unexpected character '`'. 4:70: Unexpected character '`'. 6:9: Unexpected character '#'. 8:9: Unexpected character '#'. 8:16: Char literals must contain exactly one character and close with '''. 10:9: Unexpected character '#'. 10:16: Char literals must contain exactly one character and close with '''. 12:9: Unexpected character '#'. 12:16: Char literals must contain exactly one character and close with '''. 14:9: Unexpected character '#'. 14:16: Char literals must contain exactly one character and close with '''. 16:9: Unexpected character '#'. 16:16: Char literals must contain exactly one character and close with '''. 18:9: Unexpected character '#'. 18:16: Char literals must contain exactly one character and close with '''. 26:4: Char literals must contain exactly one character and close with '''. 30:27: Unexpected character '`'. 30:40: Unexpected character '`'. 36:129: Char literals must contain exactly one character and close with '''. 46:4: Char literals must contain exactly one character and close with '''. 50:135: Unexpected character '#'. 54:4: Char literals must contain exactly one character and close with '''. 60:27: Unexpected character '`'. 60:52: Unexpected character '`'. 60:57: Unexpected character '`'. 60:63: Unexpected character '`'. 60:68: Unexpected character '`'. 60:81: Unexpected character '`'. 60:87: Unexpected character '`'. 60:99: Unexpected character '`'. 60:109: Unexpected character '`'. 60:122: Unexpected character '`'. 62:27: Unexpected character '`'. 62:45: Unexpected character '`'. 62:76: Unexpected character '#'. 62:78: Unexpected character '`'. 62:84: Unexpected character '`'. 62:94: Unexpected character '`'. 62:107: Unexpected character '`'. 64:27: Unexpected character '`'. 64:45: Unexpected character '`'. 64:56: Unexpected character '`'. 64:79: Unexpected character '`'. 66:9: Unexpected character '#'. 68:79: Unexpected character '#'. 72:30: Unexpected character '`'. 72:38: Unexpected character '`'. 72:56: Unexpected character '`'. 72:80: Unexpected character '`'. 72:87: Unexpected character '`'. 72:133: Unexpected character '`'. 72:272: Char literals must contain exactly one character and close with '''. 74:11: Unexpected character '`'. 74:19: Unexpected character '`'. 74:47: Unexpected character '#'. 76:11: Unexpected character '`'. 76:16: Unexpected character '`'. 80:20: Unexpected character '`'. 80:53: Unexpected character '`'. 82:4: Char literals must contain exactly one character and close with '''.
 Failed IN71 061 OUT21 905
Algorithms - Dutch national flag sort 1 passed
100.0%
 IN8 725 OUT948
✓
DutchFlagSortCodeGenCode-gen: model implements Dutch flag sort; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 725 OUT948
Algorithms - Egg drop trials 1 failed
0.0%
 IN41 874 OUT3 793
×
EggDropCodeGenCode-gen: model implements egg drop; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
MinTrials did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 12:63: Char literals must contain exactly one character and close with '''. 12:267: Char literals must contain exactly one character and close with '''. 14:93: Unexpected character '#'. 16:25: Unexpected character '#'. 17:3: Unexpected character '`'. 17:11: Unexpected character '`'. 17:13: Unexpected character '✓'. 18:3: Unexpected character '`'. 18:8: Unexpected character '`'. 18:15: Unexpected character '✓'. 19:19: Unexpected character '✓'. 20:3: Unexpected character '`'. 20:9: Unexpected character '`'. 20:17: Unexpected character '✓'. 21:3: Unexpected character '`'. 21:6: Unexpected character '`'. 21:19: Unexpected character '✓'. 22:23: Unexpected character '✓'. 23:23: Unexpected character '✓'. 25:21: Unexpected character '#'. 25:56: Unexpected character '`'. 25:60: Unexpected character '`'. 25:65: Unexpected character '`'. 25:70: Unexpected character '`'. 25:183: Unexpected character '`'. 25:200: Unexpected character '`'. 25:206: Unexpected character '`'. 25:222: Unexpected character '`'. 27:43: Unexpected character '`'. 27:65: Unexpected character '`'. 29:5: Unexpected character '`'. 29:16: Unexpected character '`'. 29:57: Unexpected character '#'. 29:115: Unexpected character '`'. 29:132: Unexpected character '`'. 31:133: Unexpected character '`'. 31:142: Unexpected character '`'. 35:102: Char literals must contain exactly one character and close with '''.
 Failed IN41 874 OUT3 793
Algorithms - Equal subset-sum partition 1 failed
0.0%
 IN35 296 OUT26 869
×
PartitionEqualCodeGenCode-gen: model implements equal partition; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
CanPartition did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 296 OUT26 869
Algorithms - Euler totient 1 passed
100.0%
 IN18 156 OUT1 315
✓
EulerPhiCodeGenCode-gen: model implements Euler's totient; graded vs exact oracle.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN18 156 OUT1 315
Algorithms - Eulerian path existence 1 passed
100.0%
 IN8 806 OUT1 596
✓
EulerianPathExistsCodeGenCode-gen: model implements Eulerian-path existence; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 806 OUT1 596
Algorithms - Extended Euclidean 1 failed
0.0%
 IN52 894 OUT10 586
×
ExtendedGcdCodeGenCode-gen: model implements extended gcd; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
Solve did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:169: Unexpected character '#'. 1:318: Unexpected character '#'. 1:403: Unexpected character '#'. 1:516: Unexpected character '#'. 1:636: Unexpected character '#'. 3:204: Unexpected character '#'. 3:431: Unexpected character '`'. 3:442: Unexpected character '`'. 3:449: Unexpected character '`'. 3:457: Unexpected character '`'. 5:16: Unexpected character '`'. 5:17: Unexpected character '#'. 5:18: Unexpected character '`'. 7:20: Unexpected character '#'. 34:192: Unexpected character '#'. 34:201: Unexpected character '#'. 34:237: Unexpected character '`'. 34:245: Unexpected character '`'. 34:248: Unexpected character '`'. 34:256: Unexpected character '`'. 34:327: Unexpected character '`'. 34:348: Unexpected character '`'. 42:27: Unexpected character '`'. 42:43: Unexpected character '`'. 50:128: Char literals must contain exactly one character and close with '''. 52:50: Unexpected character '#'. 60:96: Char literals must contain exactly one character and close with '''. 64:73: Char literals must contain exactly one character and close with '''. 66:17: Unexpected character '#'. 70:20: Unexpected character '#'. 70:22: Unexpected character '`'. 70:27: Unexpected character '`'. 76:4: Char literals must contain exactly one character and close with '''. 78:17: Unexpected character '`'. 78:18: Unexpected character '#'. 78:19: Unexpected character '`'. 80:20: Unexpected character '#'. 80:37: Unexpected character '`'. 80:58: Unexpected character '`'. 82:34: Unexpected character '`'. 82:49: Unexpected character '`'. 82:116: Char literals must contain exactly one character and close with '''. 84:27: Unexpected character '`'. 84:35: Unexpected character '`'. 86:16: Unexpected character '`'. 86:20: Unexpected character '`'. 88:4: Char literals must contain exactly one character and close with '''.
 Failed IN52 894 OUT10 586
Algorithms - Format duration H:MM:SS 1 failed
0.0%
 IN35 360 OUT3 658
×
FormatTimeCodeGenCode-gen: model implements duration formatting; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
FormatTime did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 360 OUT3 658
Algorithms - GCD and LCM 1 passed
100.0%
 IN18 092 OUT886
✓
GcdLcmCodeGenCode-gen: model implements GCD + LCM; graded vs exact oracle.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN18 092 OUT886
Algorithms - Grid transpose 1 failed
0.0%
 IN44 542 OUT7 850
×
TransposeCodeGenCode-gen: model implements grid transpose; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Transpose did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 6:79: Char literals must contain exactly one character and close with '''. 8:11: Unexpected character '`'. 8:49: Unexpected character '`'. 10:20: Unexpected character '`'. 10:60: Unexpected character '`'. 12:27: Unexpected character '`'. 12:31: Unexpected character '`'. 28:4: Unexpected character '#'. 30:66: Unexpected character '#'. 30:115: Unexpected character '#'. 30:332: Char literals must contain exactly one character and close with '''. 38:31: Unexpected character '`'. 38:32: Unexpected character '#'. 38:33: Unexpected character '`'. 40:143: Unexpected character '#'. 40:226: Unexpected character '`'. 40:232: Unexpected character '`'. 40:272: Char literals must contain exactly one character and close with '''. 45:17: Unexpected character '#'. 45:134: Char literals must contain exactly one character and close with '''. 49:27: Unexpected character '`'. 49:41: Unexpected character '`'. 51:27: Unexpected character '`'. 51:47: Unexpected character '`'. 51:52: Unexpected character '`'. 51:57: Unexpected character '`'. 51:63: Unexpected character '`'. 51:69: Unexpected character '`'. 53:27: Unexpected character '`'. 53:37: Unexpected character '`'. 55:27: Unexpected character '`'. 55:37: Unexpected character '`'. 55:152: Unexpected character '#'. 55:174: Unexpected character '`'. 55:223: Unexpected character '`'. 57:11: Unexpected character '`'. 57:60: Unexpected character '`'. 57:69: Unexpected character '`'. 57:76: Unexpected character '`'. 59:11: Unexpected character '`'. 59:51: Unexpected character '`'. 63:11: Unexpected character '`'. 63:55: Unexpected character '`'. 65:11: Unexpected character '`'. 65:25: Unexpected character '`'. 67:11: Unexpected character '`'. 67:33: Unexpected character '`'. 69:18: Unexpected character '`'. 69:48: Unexpected character '`'. 69:53: Unexpected character '`'. 69:67: Unexpected character '`'. 69:101: Unexpected character '`'. 69:105: Unexpected character '`'. 71:28: Unexpected character '`'. 71:42: Unexpected character '`'.
 Failed IN44 542 OUT7 850
Algorithms - House robber (max non-adjacent sum) 1 failed
0.0%
 IN66 943 OUT25 279
×
HouseRobberCodeGenCode-gen: model implements house robber; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
MaxLoot did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 2:179: Unexpected character '#'. 2:734: Char literals must contain exactly one character and close with '''. 5:17: Unexpected character '#'. 5:26: Unexpected character '#'. 5:35: Unexpected character '#'. 5:44: Unexpected character '#'. 5:92: Unexpected character '#'. 5:101: Unexpected character '#'. 5:150: Unexpected character '#'. 5:159: Unexpected character '#'. 5:189: Unexpected character '#'. 6:67: Char literals must contain exactly one character and close with '''.
 Failed IN66 943 OUT25 279
Algorithms - Integer linear interpolation 1 passed
100.0%
 IN17 737 OUT616
✓
LerpIntCodeGenCode-gen: model implements integer lerp; graded vs exact oracle.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN17 737 OUT616
Algorithms - Integer square root 1 failed
0.0%
 IN35 192 OUT3 824
×
IntSqrtCodeGenCode-gen: model implements integer sqrt; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Isqrt did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 192 OUT3 824
Algorithms - Integer to base-N string 1 failed
0.0%
 IN35 336 OUT3 761
×
BaseConversionCodeGenCode-gen: model implements int->base string; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
ToBase did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 336 OUT3 761
Algorithms - Integer to Roman numeral 1 failed
0.0%
 IN35 452 OUT3 013
×
RomanCodeGenCode-gen: model implements int->Roman; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
ToRoman did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 452 OUT3 013
Algorithms - Josephus survivor 1 failed
0.0%
 IN39 195 OUT1 432
×
JosephusCodeGenCode-gen: model implements Josephus survivor; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Survivor did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:25: Unexpected character '#'. 1:212: Unexpected character '#'. 8:16: Unexpected character '#'. 10:3: Unexpected character '`'. 10:7: Unexpected character '`'. 10:15: Unexpected character '`'. 10:19: Unexpected character '`'. 10:27: Unexpected character '`'. 10:29: Unexpected character '`'. 10:48: Unexpected character '`'. 10:50: Unexpected character '`'. 13:44: Unexpected character '#'.
 Failed IN39 195 OUT1 432
Algorithms - Jump game (min jumps) 1 passed
100.0%
 IN8 751 OUT1 334
✓
JumpGameCodeGenCode-gen: model implements jump game; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 751 OUT1 334
Algorithms - Kth largest element 1 failed
0.0%
 IN35 426 OUT2 322
×
KthLargestCodeGenCode-gen: model implements kth largest; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
KthLargest did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 426 OUT2 322
Algorithms - Least-squares line fit 1 passed
100.0%
 IN8 813 OUT1 205
✓
LinearRegressionCodeGenCode-gen: model implements OLS line fit; graded vs oracle within tolerance.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 813 OUT1 205
Algorithms - Levenshtein edit distance (DP) 1 failed
0.0%
 IN35 108 OUT4 322
×
EditDistanceCodeGenCode-gen: model implements Levenshtein edit distance; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
EditDistance did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:5: Expected a name, found end of file.
 Failed IN35 108 OUT4 322
Algorithms - Log-scale bucket index 1 passed
100.0%
 IN8 806 OUT725
✓
LogScaleBinsCodeGenCode-gen: model implements log-scale binning; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 806 OUT725
Algorithms - Logarithmic bin index 1 passed
100.0%
 IN8 765 OUT1 095
✓
LogBinsCodeGenCode-gen: model implements log bins; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 765 OUT1 095
Algorithms - Longest common prefix 1 failed
0.0%
 IN53 235 OUT15 605
×
CommonPrefixCodeGenCode-gen: model implements longest common prefix; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
LongestCommonPrefix did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:292: Char literals must contain exactly one character and close with '''. 3:354: Unexpected character '`'. 3:355: Char literals must contain exactly one character and close with '''. 11:29: Unexpected character '`'. 11:70: Unexpected character '`'. 13:33: Unexpected character '`'. 13:46: Unexpected character '`'. 15:27: Unexpected character '`'. 15:46: Unexpected character '`'. 17:27: Unexpected character '`'. 17:57: Unexpected character '`'. 17:91: Unexpected character '`'. 17:104: Unexpected character '`'. 19:27: Unexpected character '`'. 19:45: Unexpected character '`'. 21:27: Unexpected character '`'. 21:51: Unexpected character '`'. 23:27: Unexpected character '`'. 23:40: Unexpected character '`'. 25:293: Char literals must contain exactly one character and close with '''. 27:484: Unexpected character '`'. 27:530: Unexpected character '`'. 58:27: Unexpected character '`'. 58:47: Unexpected character '`'. 60:11: Unexpected character '`'. 60:25: Unexpected character '`'. 60:44: Unexpected character '`'. 60:46: Unexpected character '`'. 60:59: Unexpected character '`'. 60:76: Unexpected character '`'. 62:13: Unexpected character '`'. 62:20: Unexpected character '`'. 62:46: Unexpected character '`'. 62:61: Unexpected character '`'. 64:51: Unexpected character '`'. 64:70: Unexpected character '`'. 82:45: Unexpected character '#'. 84:27: Unexpected character '`'. 84:38: Unexpected character '`'. 88:84: Char literals must contain exactly one character and close with '''. 90:8: Char literals must contain exactly one character and close with '''. 92:27: Unexpected character '`'. 92:50: Unexpected character '`'. 92:157: Unexpected character '`'. 92:183: Unexpected character '`'. 92:190: Unexpected character '`'. 92:214: Unexpected character '`'. 92:258: Unexpected character '`'. 92:287: Unexpected character '`'. 165:16: Unexpected character '`'. 165:21: Unexpected character '`'. 165:41: Unexpected character '`'. 165:103: Unexpected character '`'. 165:117: Unexpected character '`'. 165:167: Unexpected character '`'. 165:277: Char literals must contain exactly one character and close with '''. 167:18: Unexpected character '`'. 167:29: Unexpected character '`'. 169:27: Unexpected character '`'. 169:58: Unexpected character '`'. 171:27: Unexpected character '`'. 171:46: Unexpected character '`'. 173:27: Unexpected character '`'. 173:46: Unexpected character '`'. 175:69: Char literals must contain exactly one character and close with '''. 179:4: Char literals must contain exactly one character and close with '''. 181:26: Unexpected character '`'. 181:38: Unexpected character '`'. 213:97: Unexpected character '`'. 213:101: Unexpected character '`'. 246:37: Char literals must contain exactly one character and close with '''. 304:27: Unexpected character '`'. 304:73: Unexpected character '`'. 306:27: Unexpected character '`'. 306:47: Unexpected character '`'. 308:27: Unexpected character '`'. 308:52: Unexpected character '`'. 310:27: Unexpected character '`'. 310:40: Unexpected character '`'. 312:16: Unexpected character '`'. 312:23: Unexpected character '`'. 320:15: Unexpected character '#'. 320:25: Unexpected character '`'. 320:29: Unexpected character '`'. 320:41: Unexpected character '`'. 320:85: Unexpected character '`'. 322:15: Unexpected character '#'. 322:25: Unexpected character '`'. 322:50: Unexpected character '`'. 324:13: Unexpected character '`'. 324:54: Unexpected character '`'. 326:13: Unexpected character '`'. 326:32: Unexpected character '`'. 326:42: Unexpected character '`'. 326:55: Unexpected character '`'. 328:11: Unexpected character '`'. 328:23: Unexpected character '`'. 330:40: Unexpected character '`'. 330:81: Unexpected character '`'. 332:4: Char literals must contain exactly one character and close with '''. 335:146: Too many lexical errors; further errors are not reported.
 Failed IN53 235 OUT15 605
Algorithms - Longest common subsequence 1 failed
0.0%
 IN35 298 OUT4 737
×
LcsCodeGenCode-gen: model implements LCS length; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Lcs did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 298 OUT4 737
Algorithms - Longest common substring 1 failed
0.0%
 IN70 362 OUT32 953
×
CommonSubstringCodeGenCode-gen: model implements longest common substring; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Length did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:169: Unexpected character '#'. 1:392: Unexpected character '#'. 1:462: Unexpected character '#'. 1:573: Unexpected character '#'. 1:776: Unexpected character '`'. 1:808: Unexpected character '`'. 1:814: Unexpected character '`'. 1:834: Unexpected character '`'. 1:874: Unexpected character '#'. 1:891: Unexpected character '`'. 1:901: Unexpected character '`'. 1:931: Unexpected character '`'. 1:933: Unexpected character '`'. 1:974: Unexpected character '`'. 1:981: Unexpected character '`'. 1:997: Unexpected character '`'. 1:1011: Unexpected character '`'. 1:1057: Unexpected character '`'. 1:1088: Unexpected character '`'. 1:1113: Unexpected character '`'. 1:1120: Unexpected character '`'. 1:1141: Unexpected character '#'. 1:1153: Unexpected character '#'. 1:1175: Unexpected character '`'. 1:1182: Unexpected character '`'. 1:1251: Unexpected character '`'. 1:1258: Unexpected character '`'. 3:27: Unexpected character '`'. 3:47: Unexpected character '`'. 3:55: Unexpected character '`'. 3:59: Unexpected character '`'. 5:27: Unexpected character '`'. 5:38: Unexpected character '`'. 7:27: Unexpected character '`'. 7:50: Unexpected character '`'. 9:27: Unexpected character '`'. 9:56: Unexpected character '`'. 11:27: Unexpected character '`'. 11:57: Unexpected character '`'. 13:27: Unexpected character '`'. 13:42: Unexpected character '`'. 13:48: Unexpected character '`'. 13:59: Unexpected character '`'. 15:27: Unexpected character '`'. 15:45: Unexpected character '`'. 15:108: Unexpected character '`'. 15:115: Unexpected character '`'. 15:122: Unexpected character '#'. 15:125: Unexpected character '`'. 15:134: Unexpected character '`'. 15:173: Unexpected character '#'. 15:207: Unexpected character '`'. 15:214: Unexpected character '`'. 17:27: Unexpected character '`'. 17:64: Unexpected character '`'. 19:27: Unexpected character '`'. 19:59: Unexpected character '`'. 21:27: Unexpected character '`'. 21:38: Unexpected character '`'. 23:27: Unexpected character '`'. 23:40: Unexpected character '`'. 25:83: Unexpected character '`'. 25:114: Unexpected character '`'. 25:162: Char literals must contain exactly one character and close with '''. 27:477: Char literals must contain exactly one character and close with '''. 27:677: Unexpected character '#'. 27:785: Unexpected character '#'. 27:791: Unexpected character '`'. 27:792: Unexpected character '`'. 27:821: Unexpected character '#'. 27:873: Unexpected character '#'. 27:923: Char literals must contain exactly one character and close with '''. 27:1199: Char literals must contain exactly one character and close with '''. 27:1264: Unexpected character '`'. 27:1265: Char literals must contain exactly one character and close with '''. 27:1377: Char literals must contain exactly one character and close with '''. 29:298: Char literals must contain exactly one character and close with '''. 33:925: Unexpected character '#'. 33:995: Unexpected character '`'. 33:996: Unexpected character '#'. 33:1000: Unexpected character '#'. 33:1029: Unexpected character '`'. 33:1110: Unexpected character '#'. 33:1116: Char literals must contain exactly one character and close with '''. 35:312: Unexpected character '`'. 35:313: Unexpected character '#'. 35:314: Unexpected character '`'. 35:384: Unexpected character '`'. 35:388: Unexpected character '`'. 35:508: Unexpected character '#'. 35:537: Unexpected character '#'. 35:589: Unexpected character '`'. 35:593: Unexpected character '`'. 37:374: Unexpected character '#'. 37:413: Char literals must contain exactly one character and close with '''. 38:3: Unexpected character '#'. 38:7: Unexpected character '#'. 40:5: Unexpected character '#'. 40:41: Unexpected character '#'. 43:9: Unexpected character '`'. 43:14: Too many lexical errors; further errors are not reported.
 Failed IN70 362 OUT32 953
Algorithms - Longest increasing subsequence (DP) 1 failed
0.0%
 IN35 554 OUT6 228
×
LISCodeGenCode-gen: model implements LIS length; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
LIS did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 554 OUT6 228
Algorithms - Longest palindromic substring 1 passed
100.0%
 IN8 751 OUT1 018
✓
LongestPalindromeCodeGenCode-gen: model implements longest palindromic substring length; graded vs oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 751 OUT1 018
Algorithms - Longest repeated substring 1 failed
0.0%
 IN105 795 OUT47 086
×
LongestRepeatedCodeGenCode-gen: model implements longest repeated substring; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
Length did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:50: Unexpected character '`'. 3:71: Unexpected character '`'. 3:108: Char literals must contain exactly one character and close with '''. 3:226: Char literals must contain exactly one character and close with '''. 3:306: Unexpected character '#'. 3:440: Char literals must contain exactly one character and close with '''. 3:488: Unexpected character '`'. 3:495: Unexpected character '`'. 3:557: Char literals must contain exactly one character and close with '''. 3:643: Char literals must contain exactly one character and close with '''. 5:64: Unexpected character '`'. 5:85: Unexpected character '`'. 5:249: Unexpected character '`'. 5:256: Unexpected character '`'. 5:281: Char literals must contain exactly one character and close with '''. 5:765: Unexpected character '`'. 5:772: Unexpected character '`'. 5:833: Char literals must contain exactly one character and close with '''. 7:155: Unexpected character '`'. 7:176: Unexpected character '`'. 9:55: Unexpected character '`'. 9:76: Unexpected character '`'. 9:132: Unexpected character '`'. 9:139: Unexpected character '`'. 9:354: Unexpected character '`'. 9:361: Unexpected character '`'. 9:395: Unexpected character '`'. 9:416: Unexpected character '`'. 11:107: Unexpected character '`'. 13:59: Unexpected character '`'. 13:80: Unexpected character '`'. 13:88: Unexpected character '#'. 18:51: Unexpected character '`'. 18:72: Unexpected character '`'. 21:8: Unexpected character '`'. 21:29: Unexpected character '`'. 21:44: Char literals must contain exactly one character and close with '''. 22:12: Unexpected character '#'. 22:92: Unexpected character '`'. 22:99: Unexpected character '`'. 22:124: Unexpected character '`'. 22:145: Unexpected character '`'. 22:180: Unexpected character '`'. 22:201: Unexpected character '`'. 24:42: Unexpected character '`'. 24:70: Unexpected character '`'. 24:152: Unexpected character '`'. 24:159: Unexpected character '`'. 24:190: Char literals must contain exactly one character and close with '''. 25:20: Unexpected character '`'. 25:41: Unexpected character '`'. 25:121: Unexpected character '`'. 25:128: Unexpected character '`'. 26:17: Unexpected character '`'. 26:38: Unexpected character '`'. 26:49: Unexpected character '`'. 26:71: Unexpected character '`'. 27:18: Unexpected character '`'. 27:39: Unexpected character '`'. 29:119: Unexpected character '`'. 29:121: Unexpected character '`'. 29:146: Unexpected character '`'. 29:153: Unexpected character '`'. 29:176: Unexpected character '`'. 29:190: Unexpected character '`'. 29:223: Unexpected character '`'. 29:244: Unexpected character '`'. 29:275: Unexpected character '#'. 29:282: Char literals must contain exactly one character and close with '''. 31:105: Unexpected character '`'. 31:112: Unexpected character '`'. 31:331: Unexpected character '`'. 31:338: Unexpected character '`'. 31:452: Unexpected character '`'. 31:459: Unexpected character '`'. 31:471: Char literals must contain exactly one character and close with '''. 35:53: Unexpected character '`'. 35:60: Unexpected character '`'. 37:97: Unexpected character '#'. 41:43: Unexpected character '#'. 45:284: Unexpected character '`'. 45:305: Unexpected character '`'. 51:82: Unexpected character '`'. 51:96: Unexpected character '`'. 53:25: Unexpected character '`'. 53:32: Unexpected character '`'. 53:62: Unexpected character '`'. 53:66: Unexpected character '`'. 53:80: Unexpected character '`'. 53:87: Unexpected character '`'. 55:9: Unexpected character '#'. 55:68: Unexpected character '`'. 55:75: Unexpected character '`'. 57:85: Unexpected character '`'. 57:86: Unexpected character '@'. Did you mean a string literal like "...", @"..." or $"..."? 57:96: Unexpected character '`'. 57:129: Char literals must contain exactly one character and close with '''. 59:77: Unexpected character '#'. 61:181: Unexpected character '`'. 61:202: Unexpected character '`'. 61:209: Too many lexical errors; further errors are not reported.
 Failed IN105 795 OUT47 086
Algorithms - Lower bound (binary search) 1 failed
0.0%
 IN35 598 OUT3 812
×
LowerBoundCodeGenCode-gen: model implements lower bound; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
LowerBound did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 598 OUT3 812
Algorithms - Luhn checksum validation 1 failed
0.0%
 IN35 404 OUT7 141
×
LuhnCodeGenCode-gen: model implements Luhn validation; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
LuhnValid did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 404 OUT7 141
Algorithms - Majority element 1 passed
100.0%
 IN8 749 OUT1 199
✓
MajorityElementCodeGenCode-gen: model implements majority element; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 749 OUT1 199
Algorithms - Matrix chain multiplication 1 passed
100.0%
 IN8 784 OUT1 847
✓
MatrixChainCodeGenCode-gen: model implements matrix-chain order; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 784 OUT1 847
Algorithms - Maximum subarray sum (Kadane) 1 passed
100.0%
 IN8 774 OUT801
✓
MaxSubarrayCodeGenCode-gen: model implements maximum subarray sum; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 774 OUT801
Algorithms - Merge K sorted lists 1 passed
100.0%
 IN8 767 OUT1 008
✓
MergeKSortedCodeGenCode-gen: model implements merge K sorted; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 767 OUT1 008
Algorithms - Merge overlapping intervals 1 failed
0.0%
 IN35 460 OUT8 543
×
MergeIntervalsCodeGenCode-gen: model implements interval merging; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
MergeIntervals did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN35 460 OUT8 543
Algorithms - Min-max normalization 1 passed
100.0%
 IN18 035 OUT1 332
✓
MinMaxScaleCodeGenCode-gen: model implements min-max scaling; graded vs exact oracle.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN18 035 OUT1 332
Algorithms - Minimum coin change (DP) 1 failed
0.0%
 IN35 530 OUT5 793
×
CoinChangeCodeGenCode-gen: model implements min-coin change; graded vs exact DP oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
CoinChange did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 530 OUT5 793
Algorithms - Minimum knight moves (BFS) 1 failed
0.0%
 IN42 466 OUT3 894
×
KnightMovesCodeGenCode-gen: model implements min knight moves; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
MinMoves did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:127: Char literals must contain exactly one character and close with '''. 3:48: Unexpected character '#'. 3:80: Unexpected character '#'. 6:4: Unexpected character '`'. 6:14: Unexpected character '`'. 7:23: Unexpected character '`'. 7:52: Unexpected character '`'. 8:4: Unexpected character '`'. 8:8: Unexpected character '`'. 10:7: Unexpected character '`'. 10:13: Unexpected character '`'. 10:19: Unexpected character '`'. 10:25: Unexpected character '`'. 10:31: Unexpected character '`'. 10:41: Unexpected character '`'. 12:71: Unexpected character '#'. 14:78: Char literals must contain exactly one character and close with '''. 14:136: Char literals must contain exactly one character and close with '''. 17:15: Unexpected character '→'. 26:63: Char literals must contain exactly one character and close with '''.
 Failed IN42 466 OUT3 894
Algorithms - Minimum path sum (grid DP) 1 passed
100.0%
 IN8 780 OUT1 320
✓
MinPathSumCodeGenCode-gen: model implements min path sum; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 780 OUT1 320
Algorithms - Minimum spanning tree (Kruskal) 1 passed
100.0%
 IN8 780 OUT1 787
✓
KruskalCodeGenCode-gen: model implements Kruskal MST; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 780 OUT1 787
Algorithms - Minimum window substring 1 failed
0.0%
 IN35 416 OUT6 646
×
MinWindowContainsCodeGenCode-gen: model implements minimum window substring; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:VeryHardsource:Ai4Users
Window did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 416 OUT6 646
Algorithms - Modular exponentiation (fast pow) 1 passed
100.0%
 IN8 799 OUT1 183
✓
ModPowCodeGenCode-gen: model implements modular exponentiation; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 799 OUT1 183
Algorithms - Modular inverse (extended Euclid) 1 failed
0.0%
 IN35 535 OUT8 363
×
ModInverseCodeGenCode-gen: model implements modular inverse; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
ModInverse did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'in'.
 Failed IN35 535 OUT8 363
Algorithms - Moving average 1 failed
0.0%
 IN35 376 OUT8 414
×
MovingAverageCodeGenCode-gen: model implements moving average; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Smooth did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 376 OUT8 414
Algorithms - Next greater element 1 passed
100.0%
 IN8 761 OUT1 338
✓
NextGreaterCodeGenCode-gen: model implements next greater element; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 761 OUT1 338
Algorithms - Next lexicographic permutation 1 passed
100.0%
 IN8 769 OUT1 483
✓
NextPermutationCodeGenCode-gen: model implements next permutation; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 769 OUT1 483
Algorithms - Nth Fibonacci mod m 1 passed
100.0%
 IN8 768 OUT956
✓
FibModCodeGenCode-gen: model implements nth Fibonacci mod; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN8 768 OUT956
Algorithms - Overlapping substring count 1 failed
0.0%
 IN35 276 OUT1 957
×
SubstrCountCodeGenCode-gen: model implements substring count; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Count did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:5: Expected a name, found end of file.
 Failed IN35 276 OUT1 957
Algorithms - Palindrome & anagram checks 1 failed
0.0%
 IN35 414 OUT9 883
×
PalindromeCodeGenCode-gen: model implements palindrome + anagram; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
IsPalindrome did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:6: Expected a name, found end of file.
 Failed IN35 414 OUT9 883
Algorithms - Pascal triangle row 1 passed
100.0%
 IN8 761 OUT709
✓
PascalRowCodeGenCode-gen: model implements Pascal's row; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 761 OUT709
Algorithms - Point-to-segment distance 1 failed
0.0%
 IN79 621 OUT50 832
×
PointSegmentDistanceCodeGenCode-gen: model implements point-to-segment distance; graded vs oracle within tolerance.3 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Assertion expression evaluated to false: AboutEqual(got.Value, exp) Context: Dist got 0 expected 1
 Failed IN79 621 OUT50 832
Algorithms - Prime factorization 1 failed
0.0%
 IN35 352 OUT5 311
×
PrimeFactorizeCodeGenCode-gen: model implements prime factorization; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Factorize did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 352 OUT5 311
Algorithms - Product of array except self 1 failed
0.0%
 IN35 376 OUT30 845
×
ProductExceptSelfCodeGenCode-gen: model implements product except self; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Products did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN35 376 OUT30 845
Algorithms - Quadratic Bezier point 1 broken
0.0%
 IN8 838 OUT15 594
!
BezierPointCodeGenCode-gen: model evaluates a quadratic Bézier; graded vs oracle within tolerance.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
AI output was interrupted/broken
 Broken IN8 838 OUT15 594
Algorithms - Range sum via prefix sums 1 passed
100.0%
 IN28 031 OUT1 122
✓
PrefixSumRangeCodeGenCode-gen: model implements range sum; graded vs exact oracle.3 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN28 031 OUT1 122
Algorithms - Reverse words & Hamming distance 1 failed
0.0%
 IN35 338 OUT5 745
×
ReverseWordsCodeGenCode-gen: model implements reverse-words + Hamming; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
ReverseWords did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 338 OUT5 745
Algorithms - Rod cutting 1 passed
100.0%
 IN8 794 OUT1 631
✓
RodCuttingCodeGenCode-gen: model implements rod cutting; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 794 OUT1 631
Algorithms - Roman numeral to integer 1 passed
100.0%
 IN17 610 OUT2 050
✓
RomanToIntCodeGenCode-gen: model implements Roman-to-int; graded vs exact oracle.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN17 610 OUT2 050
Algorithms - Rotate array right by k 1 failed
0.0%
 IN35 412 OUT3 470
×
RotateArrayCodeGenCode-gen: model implements right rotation; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
RotateRight did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 412 OUT3 470
Algorithms - Rotate grid 90 CW (discriminator) 1 failed
0.0%
 IN35 424 OUT5 858
×
RotateMatrixCodeGenCode-gen: model implements 90-degree CW rotation; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationcategory:GameDevlevel:Hardsource:Ai4Users
Rotate90 did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 424 OUT5 858
Algorithms - RPN evaluator 1 failed
0.0%
 IN43 977 OUT6 014
×
RpnCodeGenCode-gen: model implements an RPN evaluator; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
EvalRpn did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:141: Unexpected character '`'. 1:166: Unexpected character '`'. 1:228: Char literals must contain exactly one character and close with '''. 3:56: Unexpected character '`'. 3:81: Unexpected character '`'. 3:118: Unexpected character '`'. 3:122: Unexpected character '`'. 5:173: Unexpected character '`'. 5:175: Unexpected character '`'. 7:179: Unexpected character '`'. 7:204: Unexpected character '`'. 7:237: Unexpected character '`'. 7:241: Unexpected character '`'. 7:247: Unexpected character '`'. 7:255: Unexpected character '`'. 11:124: Unexpected character '#'. 13:23: Unexpected character '#'. 14:3: Unexpected character '`'. 14:25: Unexpected character '`'. 14:49: Unexpected character '`'. 14:56: Unexpected character '`'. 14:61: Unexpected character '`'. 14:66: Unexpected character '`'. 15:3: Unexpected character '`'. 15:14: Unexpected character '`'. 16:3: Unexpected character '`'. 16:19: Unexpected character '`'. 17:3: Unexpected character '`'. 17:24: Unexpected character '`'. 18:3: Unexpected character '`'. 18:15: Unexpected character '`'. 19:3: Unexpected character '`'. 19:19: Unexpected character '`'. 22:32: Unexpected character '`'. 22:54: Unexpected character '`'. 22:153: Unexpected character '`'. 22:166: Unexpected character '`'. 24:46: Unexpected character '#'. 24:107: Unexpected character '`'. 24:111: Unexpected character '`'. 28:5: Unexpected character '`'. 28:44: Unexpected character '`'. 30:33: Unexpected character '`'. 30:55: Unexpected character '`'. 30:123: Unexpected character '`'. 30:139: Unexpected character '`'. 30:172: Unexpected character '`'. 30:188: Unexpected character '`'. 32:46: Char literals must contain exactly one character and close with '''. 37:11: Unexpected character '→'. 37:67: Unexpected character '✓'. 38:11: Unexpected character '→'. 38:67: Unexpected character '✓'. 40:39: Unexpected character '#'. 40:85: Unexpected character '#'. 42:73: Unexpected character '#'.
 Failed IN43 977 OUT6 014
Algorithms - Run-length compression (if shorter) 1 passed
100.0%
 IN8 764 OUT1 013
✓
StringCompressCodeGenCode-gen: model implements string compression; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
 Passed IN8 764 OUT1 013
Algorithms - Run-length encoding (encode/decode) 1 failed
0.0%
 IN35 522 OUT7 920
×
RunLengthCodeGenCode-gen: model implements RLE encode+decode; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
Encode did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 522 OUT7 920
Algorithms - Semantic versioning (compare + bump) 1 passed
100.0%
 IN18 747 OUT2 372
✓
SemVerCodeGenCode-gen: model implements semver compare + bump; graded vs exact oracle.2 AI queries 0 tool callscategory:CodeGenerationcategory:InstructionFollowinglevel:Easysource:Ai4Users
 Passed IN18 747 OUT2 372
Algorithms - Single-pile Nim (optimal play) 1 passed
100.0%
 IN8 772 OUT1 235
✓
NimCodeGenCode-gen: model implements optimal Nim; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
 Passed IN8 772 OUT1 235
Algorithms - Sliding window maximum 1 failed
0.0%
 IN35 320 OUT9 265
×
SlidingWindowMaxCodeGenCode-gen: model implements sliding window maximum; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
WindowMax did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 320 OUT9 265
Algorithms - Spiral matrix traversal 1 failed
0.0%
 IN35 356 OUT4 267
×
SpiralMatrixCodeGenCode-gen: model implements spiral traversal; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
Spiral did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 356 OUT4 267
Algorithms - Square matrix multiply 1 failed
0.0%
 IN81 529 OUT36 698
×
MatMulCodeGenCode-gen: model implements square matrix multiply; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
MatMul did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:1: Unexpected character '#'. 1:5: Unexpected character '#'.
 Failed IN81 529 OUT36 698
Algorithms - Strongly connected components 1 failed
0.0%
 IN38 494 OUT21 025
×
SccCountCodeGenCode-gen: model implements SCC count; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:VeryHardsource:Ai4Users
Count did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN38 494 OUT21 025
Algorithms - Subset sum reachability 1 failed
0.0%
 IN35 260 OUT54 373
×
SubsetSumCodeGenCode-gen: model implements subset-sum reachability; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Mediumsource:Ai4Users
CanReach did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 260 OUT54 373
Algorithms - Substring search (KMP) 1 passed
100.0%
 IN8 771 OUT2 272
✓
KmpCodeGenCode-gen: model implements KMP search; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 771 OUT2 272
Algorithms - Sum of digits & digital root 1 failed
0.0%
 IN35 282 OUT1 966
×
DigitalRootCodeGenCode-gen: model implements sum-of-digits + digital root; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
SumDigits did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 282 OUT1 966
Algorithms - Temperature conversion 1 failed
0.0%
 IN37 731 OUT1 162
×
TemperatureCodeGenCode-gen: model implements temperature conversion; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
CToF did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:147: Unexpected character '#'. 1:175: Unexpected character '→'. 3:18: Unexpected character '#'. 3:69: Unexpected character '#'. 5:4: Unexpected character '`'. 5:20: Unexpected character '`'. 5:30: Unexpected character '`'. 5:45: Unexpected character '`'. 6:4: Unexpected character '`'. 6:20: Unexpected character '`'. 6:30: Unexpected character '`'. 6:47: Unexpected character '`'. 8:248: Unexpected character '#'. 10:22: Unexpected character '#'. 12:20: Unexpected character '`'. 13:3: Unexpected character '`'. 13:4: Unexpected character '@'. Did you mean a string literal like "...", @"..." or $"..."? 13:10: Unexpected character '`'. 13:16: Unexpected character '`'. 13:17: Unexpected character '@'. Did you mean a string literal like "...", @"..." or $"..."? 13:25: Unexpected character '`'. 14:53: Unexpected character '#'. 17:72: Char literals must contain exactly one character and close with '''.
 Failed IN37 731 OUT1 162
Algorithms - Topological sort (lexicographically smallest) 1 passed
100.0%
 IN17 648 OUT10 035
✓
TopoCodeGenCode-gen: model implements lexicographically-smallest topological sort.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN17 648 OUT10 035
Algorithms - Trailing zeros of factorial 1 failed
0.0%
 IN41 512 OUT11 888
×
TrailingZerosCodeGenCode-gen: model implements trailing zeros; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
TrailingZeros did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 4:20: Unexpected character '`'. 6:92: Char literals must contain exactly one character and close with '''. 10:55: Unexpected character '`'. 10:80: Unexpected character '`'. 14:4: Char literals must contain exactly one character and close with '''. 27:54: Char literals must contain exactly one character and close with '''. 31:304: Unexpected character '#'. 33:9: Unexpected character '#'. 33:47: Unexpected character '`'. 35:9: Unexpected character '#'. 35:41: Unexpected character '`'. 35:46: Unexpected character '`'. 37:9: Unexpected character '#'. 37:19: Unexpected character '`'. 37:30: Unexpected character '`'. 39:9: Unexpected character '#'. 39:37: Unexpected character '`'. 39:44: Unexpected character '`'. 41:9: Unexpected character '#'. 41:19: Unexpected character '`'. 41:44: Unexpected character '`'. 41:59: Unexpected character '`'. 41:64: Unexpected character '`'. 43:9: Unexpected character '#'. 45:9: Unexpected character '#'. 45:21: Unexpected character '`'. 45:25: Unexpected character '`'. 45:50: Unexpected character '`'. 45:54: Unexpected character '`'. 47:9: Unexpected character '#'. 47:21: Unexpected character '`'. 47:27: Unexpected character '`'. 49:9: Unexpected character '#'. 49:21: Unexpected character '`'. 49:40: Unexpected character '`'. 49:50: Unexpected character '`'. 49:52: Unexpected character '`'. 51:9: Unexpected character '#'. 51:40: Unexpected character '`'. 51:46: Unexpected character '`'. 53:9: Unexpected character '#'. 53:21: Unexpected character '`'. 53:32: Unexpected character '`'. 53:42: Unexpected character '`'. 53:44: Unexpected character '`'. 53:58: Unexpected character '`'. 53:62: Unexpected character '`'. 55:9: Unexpected character '#'. 55:19: Unexpected character '`'. 55:29: Unexpected character '`'. 55:35: Unexpected character '`'. 55:42: Unexpected character '`'. 57:9: Unexpected character '#'. 57:19: Unexpected character '`'. 57:33: Unexpected character '`'. 59:9: Unexpected character '#'. 59:19: Unexpected character '`'. 59:34: Unexpected character '`'. 61:9: Unexpected character '#'. 61:19: Unexpected character '`'. 61:40: Unexpected character '`'. 63:38: Unexpected character '`'. 63:52: Unexpected character '`'. 65:37: Unexpected character '`'. 65:62: Unexpected character '`'. 65:70: Unexpected character '#'. 67:9: Unexpected character '#'. 67:47: Unexpected character '`'. 67:49: Unexpected character '`'. 69:9: Unexpected character '#'. 71:9: Unexpected character '#'. 71:19: Unexpected character '`'. 71:26: Unexpected character '`'. 73:9: Unexpected character '#'. 75:9: Unexpected character '#'. 77:9: Unexpected character '#'. 79:27: Unexpected character '`'. 79:37: Unexpected character '`'. 88:257: Unexpected character '`'. 88:278: Unexpected character '`'. 90:76: Unexpected character '#'. 92:9: Unexpected character '#'. 94:9: Unexpected character '#'. 96:9: Unexpected character '#'. 98:9: Unexpected character '#'. 100:9: Unexpected character '#'. 100:21: Unexpected character '`'. 100:27: Unexpected character '`'. 102:9: Unexpected character '#'. 102:19: Unexpected character '`'. 102:40: Unexpected character '`'. 104:11: Unexpected character '`'. 104:15: Unexpected character '`'. 120:9: Unexpected character '#'. 122:24: Char literals must contain exactly one character and close with '''. 124:98: Char literals must contain exactly one character and close with '''. 128:9: Unexpected character '#'. 128:70: Char literals must contain exactly one character and close with '''. 130:9: Unexpected character '#'. 130:22: Unexpected character '`'. 130:37: Too many lexical errors; further errors are not reported.
 Failed IN41 512 OUT11 888
Algorithms - Trapping rain water 1 failed
0.0%
 IN57 913 OUT17 990
×
TrappingRainCodeGenCode-gen: model implements trapping rain water; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
Trapped did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 4:16: Unexpected character '#'. 4:50: Unexpected character '`'. 4:78: Unexpected character '`'. 4:109: Unexpected character '`'. 4:134: Unexpected character '`'. 4:137: Unexpected character '`'. 4:158: Unexpected character '`'. 6:20: Unexpected character '#'. 8:20: Unexpected character '#'. 10:20: Unexpected character '#'. 10:37: Unexpected character '`'. 10:41: Unexpected character '`'. 10:117: Unexpected character '`'. 10:147: Unexpected character '`'. 12:18: Unexpected character '`'. 12:35: Unexpected character '`'. 14:18: Unexpected character '`'. 14:34: Unexpected character '`'. 16:18: Unexpected character '`'. 16:35: Unexpected character '`'. 18:18: Unexpected character '`'. 18:28: Unexpected character '`'. 20:18: Unexpected character '`'. 20:52: Unexpected character '`'. 22:18: Unexpected character '`'. 22:27: Unexpected character '`'. 24:18: Unexpected character '`'. 24:29: Unexpected character '`'. 26:18: Unexpected character '`'. 26:30: Unexpected character '`'. 28:18: Unexpected character '`'. 28:49: Unexpected character '`'. 30:18: Unexpected character '`'. 30:26: Unexpected character '`'. 32:18: Unexpected character '`'. 32:27: Unexpected character '`'. 34:18: Unexpected character '`'. 34:32: Unexpected character '`'. 36:4: Char literals must contain exactly one character and close with '''. 38:27: Unexpected character '`'. 38:42: Unexpected character '`'. 61:130: Unexpected character '#'. 61:164: Unterminated string literal. Strings must be closed with '"' on the same line.
 Failed IN57 913 OUT17 990
Algorithms - Two-sum indices 1 passed
100.0%
 IN18 590 OUT1 635
✓
TwoSumCodeGenCode-gen: model implements two-sum; graded vs exact oracle.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN18 590 OUT1 635
Algorithms - Unique grid paths 1 passed
100.0%
 IN18 162 OUT1 411
✓
UniquePathsCodeGenCode-gen: model implements unique paths; graded vs exact oracle.2 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN18 162 OUT1 411
Algorithms - Valid Sudoku unit 1 passed
100.0%
 IN8 748 OUT471
✓
SudokuUnitCodeGenCode-gen: model implements Sudoku unit validation; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Easysource:Ai4Users
 Passed IN8 748 OUT471
Algorithms - Wildcard string matching 1 passed
100.0%
 IN8 766 OUT1 137
✓
WildcardMatchCodeGenCode-gen: model implements wildcard match; graded vs exact oracle.1 AI query 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
 Passed IN8 766 OUT1 137
Algorithms - Z-function 1 failed
0.0%
 IN35 332 OUT6 598
×
ZFunctionCodeGenCode-gen: model implements the Z-function; graded vs exact oracle.4 AI queries 0 tool callscategory:Algorithmscategory:CodeGenerationlevel:Hardsource:Ai4Users
ZArray did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 332 OUT6 598
Database - Conditional aggregation pivot 1 failed
0.0%
 IN6 577 OUT1 001
×
PivotCountsCodeGenCode-gen: model writes the pivot query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "We": syntax error'.
 Failed IN6 577 OUT1 001
Database - Count distinct opponents 1 failed
0.0%
 IN5 819 OUT727
×
DistinctCountCodeGenCode-gen: model writes the COUNT(DISTINCT) query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Easysource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "The": syntax error'.
 Failed IN5 819 OUT727
Database - Date-range filter (BETWEEN) 1 failed
0.0%
 IN6 867 OUT936
×
DateRangeCodeGenCode-gen: model writes the date-range query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "We": syntax error'.
 Failed IN6 867 OUT936
Database - EXISTS subquery (teams with matches) 1 failed
0.0%
 IN5 638 OUT710
×
ExistsSubqueryCodeGenCode-gen: model writes the EXISTS query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "The": syntax error'.
 Failed IN5 638 OUT710
Database - JOIN player totals (two tables) 1 failed
0.0%
 IN5 270 OUT494
×
JoinScoresCodeGenCode-gen: model writes a JOIN + aggregate query; graded via QueryJson vs oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databasecategory:GameDevlevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "S": syntax error'.
 Failed IN5 270 OUT494
Database - LEFT JOIN anti-join (no items) 1 failed
0.0%
 IN5 130 OUT389
×
LeftJoinNullsCodeGenCode-gen: model writes the anti-join query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "S": syntax error'.
 Failed IN5 130 OUT389
Database - Player stats SQL (multi-aggregate, QueryJson) 1 failed
0.0%
 IN5 340 OUT448
×
PlayerStatsCodeGenCode-gen: model writes a multi-aggregate stats query; graded by comparing its QueryJson output to the oracle query's (arbitrary shape, no struct).5 AI queries 0 tool callscategory:CodeGenerationcategory:Databasecategory:GameDevlevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "S": syntax error'.
 Failed IN5 340 OUT448
Database - Players above average score (subquery) 1 failed
0.0%
 IN6 089 OUT643
×
AboveAverageCodeGenCode-gen: model writes an above-average query; graded via QueryJson vs oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "We": syntax error'.
 Failed IN6 089 OUT643
Database - Players who never played (anti-join) 1 failed
0.0%
 IN5 060 OUT613
×
NeverPlayedCodeGenCode-gen: model writes an anti-join query; graded via QueryJson vs oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "S": syntax error'.
 Failed IN5 060 OUT613
Database - RANK window function 1 failed
0.0%
 IN5 260 OUT532
×
WindowRankCodeGenCode-gen: model writes the RANK() query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Hardsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "S": syntax error'.
 Failed IN5 260 OUT532
Database - Running total (window function) 1 failed
0.0%
 IN6 607 OUT927
×
RunningTotalCodeGenCode-gen: model writes the window running-total query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Hardsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "The": syntax error'.
 Failed IN6 607 OUT927
Database - Scoreboard SQL (players with >1 win, HAVING) 1 failed
0.0%
 IN5 270 OUT618
×
WinFilterCodeGenCode-gen: model writes a HAVING query; runs under try/catch with self-repair; result compared to the oracle query.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databasecategory:GameDevlevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "S": syntax error'.
 Failed IN5 270 OUT618
Database - Scoreboard SQL (top scorers query) 1 failed
0.0%
 IN5 275 OUT520
×
ScoreboardTopScorersCodeGenCode-gen: model writes a scoreboard SQL query; runs under try/catch with a self-repair loop on SQL errors; result compared to the oracle query.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databasecategory:GameDevlevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "S": syntax error'.
 Failed IN5 275 OUT520
Database - Self-join same-team pairs 1 failed
0.0%
 IN5 190 OUT637
×
SelfJoinCodeGenCode-gen: model writes the self-join query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Mediumsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "SE": syntax error'.
 Failed IN5 190 OUT637
Database - Top scorer per region 1 failed
0.0%
 IN5 220 OUT1 049
×
TopPerGroupCodeGenCode-gen: model writes the per-group top query; compared to the oracle.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Hardsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "S": syntax error'.
 Failed IN5 220 OUT1 049
Database - UPDATE restock (SQL write) 1 failed
0.0%
 IN5 110 OUT442
×
UpdateInventoryCodeGenCode-gen: model writes an UPDATE; applied under try/catch, resulting state compared (via QueryJson) to the oracle update's state.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databasecategory:GameDevlevel:Mediumsource:Ai4Users
candidate UPDATE never executed; last error: Database error: SQLite Error 1: 'near "UPDA": syntax error'.
 Failed IN5 110 OUT442
Database - Upsert with max merge 1 failed
0.0%
 IN7 448 OUT1 095
×
UpsertCodeGenCode-gen: model writes the upsert; executed on a fresh DB and the table compared.5 AI queries 0 tool callscategory:CodeGenerationcategory:Databaselevel:Hardsource:Ai4Users
candidate SQL never executed; last error: Database error: SQLite Error 1: 'near "We": syntax error'.
 Failed IN7 448 OUT1 095
SDLC - Acceptance criteria check 1 failed
0.0%
 IN35 428 OUT13 701
×
SdlcAcceptanceCodeGenCode-gen: model implements acceptance-criteria check; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
Failing did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:3: Expected a name, found end of file.
 Failed IN35 428 OUT13 701
SDLC - Breaking API change detection 1 failed
0.0%
 IN41 472 OUT12 188
×
SdlcApiBreakingCodeGenCode-gen: model implements API-breaking detection; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
Breaking did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN41 472 OUT12 188
SDLC - Condense without losing facts 1 failed
0.0%
 IN8 400 OUT2 898
×
NoInfoLossCodeGenRedaction test: model condenses keeping facts; checker + self-review loop.4 AI queries 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
Assertion expression evaluated to false: complaint == "" Context: output still fails the checker: too long: 2319 > 240 chars Values: complaint = "too long: 2319 > 240 chars"
 Failed IN8 400 OUT2 898
SDLC - Config merge with override 1 passed
100.0%
 IN8 782 OUT1 400
✓
ConfigMergeCodeGenCode-gen: model implements config merge; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
 Passed IN8 782 OUT1 400
SDLC - Extract fields into fixed template 1 passed
100.0%
 IN1 002 OUT140
✓
FieldExtractCodeGenRedaction test: model extracts fields into the template; checker + self-review loop.1 AI query 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
 Passed IN1 002 OUT140
SDLC - Locate conflicting requirement pair 1 failed
0.0%
 IN51 390 OUT8 390
×
SdlcConstraintConflictCodeGenCode-gen: model locates the conflicting requirement pair; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Hardsource:Ai4Users
FindConflict did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 1:13: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found 'if'.
 Failed IN51 390 OUT8 390
SDLC - Locate redundant requirement 1 passed
100.0%
 IN8 829 OUT3 638
✓
SdlcRedundantRequirementCodeGenCode-gen: model locates a redundant requirement; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN8 829 OUT3 638
SDLC - Mask PII to placeholder 1 failed
0.0%
 IN4 366 OUT695
×
MaskPiiCodeGenRedaction test: model masks PII to a placeholder; checker + self-review loop.4 AI queries 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
Assertion expression evaluated to false: complaint == "" Context: output still fails the checker: expected the placeholder '[REDACTED]' at least twice; lost fact 'Contact alice'; lost fact 'access'; lost fact 'REF-7'; lost fact 'high priority'; lost fact '2026-05-01' Values: complaint = "expected the placeholder '[REDACTED]' at least twice; lost fact 'Contact alice'; lost fact 'access'; lost fact 'REF-7'; lost fact 'high priority'; lost fact '2026-05-01'"
 Failed IN4 366 OUT695
SDLC - Modules on a dependency cycle 1 passed
100.0%
 IN8 794 OUT2 829
✓
DependencyCycleModulesCodeGenCode-gen: model implements dependency-cycle detection; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Hardsource:Ai4Users
 Passed IN8 794 OUT2 829
SDLC - Remove duplicate requirements 1 failed
0.0%
 IN5 029 OUT269
×
DeduplicateCodeGenRedaction test: model removes duplicate requirements; checker + self-review loop.4 AI queries 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
Assertion expression evaluated to false: complaint == "" Context: output still fails the checker: expected 4 unique lines, got 9; line 1 should be 'login' but was 'The user keeps getting my thinking/reasoning included in the output. I need to output EXACTLY 4 lines and absolutely nothing else. No reasoning, no explanation, no preamble. Just:'; line 2 should be 'logout' but was 'login'; line 3 should be 'password reset' but was 'logout'; line 4 should be 'audit log' but was 'password reset' Values: complaint = "expected 4 unique lines, got 9; line 1 should be 'login' but was 'The user keeps getting my thinking/reasoning included in the output. I need to output EXACTLY 4 lines and absolutely nothing else. No reasoning, no explanation, no preamble. Just:'; line 2 should be 'logout' but was 'login'; line 3 should be 'password reset' but was 'logout'; line 4 should be 'audit log' but was 'password reset'"
 Failed IN5 029 OUT269
SDLC - Renumber requirements sequentially 1 failed
0.0%
 IN5 807 OUT675
×
RenumberCodeGenRedaction test: model renumbers sequentially; checker + self-review loop.4 AI queries 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
Assertion expression evaluated to false: complaint == "" Context: output still fails the checker: expected 4 lines, got 7; line 1 must start with 'REQ-1:'; line 1 must keep keyword 'login'; line 2 must start with 'REQ-2:'; line 3 must start with 'REQ-3:'; line 3 must keep keyword 'password reset'; line 4 must start with 'REQ-4:'; line 4 must keep keyword 'audit log' Values: complaint = "expected 4 lines, got 7; line 1 must start with 'REQ-1:'; line 1 must keep keyword 'login'; line 2 must start with 'REQ-2:'; line 3 must start with 'REQ-3:'; line 3 must keep keyword 'password reset'; line 4 must start with 'REQ-4:'; line 4 must keep keyword 'audit log'"
 Failed IN5 807 OUT675
SDLC - Reorganize notes into section template 1 failed
0.0%
 IN4 784 OUT1 121
×
RewriteSectionsCodeGenRedaction test: model reorganizes the notes; deterministic checker + self-review loop.4 AI queries 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
Assertion expression evaluated to false: complaint == "" Context: document still fails the checker: missing header '## Functional'; missing header '## Non-Functional'; missing header '## Constraints' Values: complaint = "missing header '## Functional'; missing header '## Non-Functional'; missing header '## Constraints'"
 Failed IN4 784 OUT1 121
SDLC - Required semver bump from changelog 1 failed
0.0%
 IN35 412 OUT3 964
×
SemverBumpCodeGenCode-gen: model implements the semver-bump rule; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Easysource:Ai4Users
RequiredBump did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN35 412 OUT3 964
SDLC - Requirement-to-test traceability 1 passed
100.0%
 IN8 809 OUT840
✓
SdlcTraceabilityCodeGenCode-gen: model implements traceability gap; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
 Passed IN8 809 OUT840
SDLC - Requirements completeness (spec coverage gap) 1 passed
100.0%
 IN8 804 OUT1 692
✓
SdlcSpecCoverageCodeGenCode-gen: model implements spec-gap detection; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
 Passed IN8 804 OUT1 692
SDLC - Requirements consistency (contradiction detection) 1 failed
0.0%
 IN35 420 OUT10 428
×
ConsistencyCodeGenCode-gen: model implements requirements-consistency checking; graded vs oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
Consistent did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:4: Expected a name, found end of file.
 Failed IN35 420 OUT10 428
SDLC - Reshape prose into JSON template 1 passed
100.0%
 IN1 012 OUT100
✓
JsonShapeCodeGenRedaction test: model reshapes prose into JSON; checker + self-review loop.1 AI query 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
 Passed IN1 012 OUT100
SDLC - Reshape rows into Markdown table 1 failed
0.0%
 IN4 302 OUT403
×
TableFormatCodeGenRedaction test: model builds a Markdown table; checker + self-review loop.4 AI queries 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
Assertion expression evaluated to false: complaint == "" Context: output still fails the checker: missing header '| Name | Role |'; missing separator row '| --- | --- |'; missing data row '| alice | admin |'; missing data row '| bob | editor |'; missing data row '| carol | viewer |' Values: complaint = "missing header '| Name | Role |'; missing separator row '| --- | --- |'; missing data row '| alice | admin |'; missing data row '| bob | editor |'; missing data row '| carol | viewer |'"
 Failed IN4 302 OUT403
SDLC - Rewrite in imperative requirement style 1 passed
100.0%
 IN989 OUT287
✓
StyleRewriteCodeGenRedaction test: model rewrites notes in imperative style; checker + self-review loop.1 AI query 0 tool callscategory:InstructionFollowinglevel:Mediumsource:Ai4Users
 Passed IN989 OUT287
SDLC - Roll up estimates over task tree 1 failed
0.0%
 IN35 464 OUT4 534
×
EstimateRollupCodeGenCode-gen: model implements estimate roll-up over a task tree; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
Total did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:5: Expected a name, found end of file.
 Failed IN35 464 OUT4 534
SDLC - State-machine reachability 1 passed
100.0%
 IN8 786 OUT1 405
✓
SdlcStateReachableCodeGenCode-gen: model implements state reachability; graded vs exact oracle.1 AI query 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
 Passed IN8 786 OUT1 405
SDLC - Ticket priority sort 1 failed
0.0%
 IN35 380 OUT7 534
×
SdlcPrioritySortCodeGenCode-gen: model implements priority sort; graded vs exact oracle.4 AI queries 0 tool callscategory:CodeGenerationcategory:Reasoninglevel:Mediumsource:Ai4Users
Order did not compile: SC# compilation failed inside the script runner. Compiler diagnostics: 3:2: Expected a name, found end of file.
 Failed IN35 380 OUT7 534
SDLC - Unify glossary term consistency 1 failed
0.0%
 IN5 762 OUT1 073
×
GlossaryCodeGenRedaction test: model unifies synonyms to the canonical term; checker + self-review loop.4 AI queries 0 tool callscategory:ContextProcessingcategory:InstructionFollowinglevel:Mediumsource:Ai4Users
Assertion expression evaluated to false: complaint == "" Context: output still fails the checker: banned term still present: 'customer'; banned term still present: 'client' Values: complaint = "banned term still present: 'customer'; banned term still present: 'client'"
 Failed IN5 762 OUT1 073