Code Generation - LiveCodeBench v1 AtCoder (set 1)
4 passed4 failed |
50.0% |
IN98 480
OUT8 555 |
| × |
LcbAntiDdosCountLiveCodeBench / atcoder abc301_f / hard: Anti (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AntiDdosCount case0: expected [676] got [998264069] |
Failed |
IN9 282
OUT453 |
| × |
LcbAtCoderCardsLiveCodeBench / atcoder abc301_c / medium: AtCoder Cards (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AtCoderCards case0: expected [Yes] got [No] |
Failed |
IN9 214
OUT498 |
| ✓ |
LcbAttackCountLiveCodeBench / atcoder abc302_a / easy: Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 168
OUT68 |
| × |
LcbBitmaskLeqNLiveCodeBench / atcoder abc301_d / medium: Bitmask (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BitmaskLeqN case0: expected [1] got [2] |
Failed |
IN9 242
OUT264 |
| ✓ |
LcbFillTheGapsLiveCodeBench / atcoder abc301_b / easy: Fill the Gaps (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 277
OUT376 |
| ✓ |
LcbFindSnukeLiveCodeBench / atcoder abc302_b / medium: Find snuke (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN33 742
OUT2 827 |
| ✓ |
LcbOverallWinnerLiveCodeBench / atcoder abc301_a / easy: Overall Winner (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 228
OUT293 |
| × |
LcbPacCandyPathLiveCodeBench / atcoder abc301_e / hard: Pac (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PacCandyPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:16: Expected ';' after the variable declaration, found ','.
Values:
run.Compiled = false |
Failed |
IN9 327
OUT3 776 |
Code Generation - LiveCodeBench v1 AtCoder (set 10)
2 passed4 failed2 skipped |
33.3% |
IN66 238
OUT2 555 |
| × |
LcbBlueSpringPassesLiveCodeBench / atcoder / medium: abc318_c Blue Spring (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BlueSpringPasses case0: expected [20] got [23] |
Failed |
IN9 217
OUT221 |
| – |
LcbBookPrerequisitesLiveCodeBench / atcoder / hard: abc315_e Prerequisites (Pattern B, stdin) FLAG: non-unique output (any valid reading order accepted); exact-string driver can reject a correct answer -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: non-unique output: multiple valid reading orders accepted; exact-string driver would reject correct answers |
Skipped |
IN0
OUT0 |
| – |
LcbCheckpointShortcutsLiveCodeBench / atcoder / hard: abc315_f Shortcuts (Pattern B, stdin) FLAG: float output (Euclidean distance sum); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: SC# stdin driver: floating-point distance output needs tolerance comparison; exact string compare unsafe |
Skipped |
IN0
OUT0 |
| ✓ |
LcbFullMoonDaysLiveCodeBench / atcoder / easy: abc318_a Full Moon (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 187
OUT101 |
| × |
LcbIceCreamFlavorsLiveCodeBench / atcoder / medium: abc315_c Flavors (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: IceCreamFlavors case0: expected [16] got [20] |
Failed |
IN9 211
OUT429 |
| ✓ |
LcbMagicalCookiesLiveCodeBench / atcoder / medium: abc315_d Magical Cookies (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN20 206
OUT855 |
| × |
LcbMiddleDayLiveCodeBench / atcoder / easy: abc315_b The Middle Day (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MiddleDay case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:14: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 229
OUT233 |
| × |
LcbOverlappingSheetsLiveCodeBench / atcoder / easy: abc318_b Overlapping sheets (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: OverlappingSheets case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
33:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 188
OUT716 |
Code Generation - LiveCodeBench v1 AtCoder (set 11)
4 passed3 failed1 skipped |
57.1% |
IN99 602
OUT4 394 |
| ✓ |
LcbBusStopsQueriesLiveCodeBench / atcoder / hard: abc319_e Bus Stops (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 306
OUT1 184 |
| – |
LcbFalseHopeProbabilityLiveCodeBench / atcoder / medium: abc319_c False Hope (Pattern B, stdin) FLAG: float output (probability); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: SC# stdin driver: probability (float) output needs tolerance comparison; exact string compare unsafe |
Skipped |
IN0
OUT0 |
| ✓ |
LcbLeylandNumberLiveCodeBench / atcoder / easy: abc320_a Leyland Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 152
OUT83 |
| ✓ |
LcbLongestPalindromeSubLiveCodeBench / atcoder / easy: abc320_b Longest Palindrome (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 161
OUT228 |
| ✓ |
LcbMeasureDivisorsLiveCodeBench / atcoder / easy: abc319_b Measure (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 214
OUT167 |
| × |
LcbMinimumWindowWidthLiveCodeBench / atcoder / medium: abc319_d Minimum Width (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimumWindowWidth case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:349: Char literals must contain exactly one character and close with '''.
7:36: Char literals must contain exactly one character and close with '''.
13:67: Unexpected character '—'.
19:185: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 233
OUT969 |
| × |
LcbSandwichTriplesLiveCodeBench / atcoder / hard: abc318_e Sandwiches (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SandwichTriples case0: expected [3] got [4] |
Failed |
IN9 180
OUT281 |
| × |
LcbWeightedMaxMatchingLiveCodeBench / atcoder / medium: abc318_d General Weighted Max Matching (Pattern B, stdin)1 AI query
3 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: WeightedMaxMatching case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN44 356
OUT1 482 |
Code Generation - LiveCodeBench v1 AtCoder (set 12)
4 passed5 failed |
44.4% |
IN196 981
OUT19 558 |
| ✓ |
Lcb321LikeLiveCodeBench / atcoder / easy: abc321_a 321-like Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 245
OUT88 |
| × |
LcbCompleteBinaryTreeLiveCodeBench / atcoder / hard: abc321_e Complete Binary Tree (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CompleteBinaryTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
27:35: The shift count of '<<' must be 'int', but got 'long'.
28:35: The shift count of '<<' must be 'int', but got 'long'.
37:40: The shift count of '<<' must be 'int', but got 'long'.
38:38: The shift count of '<<' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed |
IN9 235
OUT2 944 |
| × |
LcbCutoffLiveCodeBench / atcoder / easy: abc321_b Cutoff (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Cutoff case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:55: Unexpected character '∈'.
3:193: Unexpected character '≥'.
7:72: Unexpected character '−'.
7:78: Unexpected character '−'.
13:19: Unexpected character '−'.
13:31: Unexpected character '−'.
16:8: Unexpected character '≤'.
16:19: Unexpected character '≥'.
17:7: Unexpected character '≤'.
17:18: Unexpected character '≤'.
17:32: Unexpected character '≤'.
17:56: Unexpected character '≤'.
17:81: Unexpected character '→'.
17:98: Unexpected character '−'.
17:102: Unexpected character '−'.
17:112: Unexpected character '−'.
17:148: Unexpected character '≥'.
17:239: Char literals must contain exactly one character and close with '''.
19:42: Unexpected character '—'.
19:48: Char literals must contain exactly one character and close with '''.
21:7: Unexpected character '≥'.
21:19: Unexpected character '≥'.
21:54: Unexpected character '→'.
21:71: Unexpected character '−'.
21:76: Unexpected character '−'.
21:85: Unexpected character '−'.
23:33: Unexpected character '→'.
23:50: Unexpected character '−'.
23:55: Unexpected character '−'.
26:16: Unexpected character '≤'.
26:29: Unexpected character '−'.
27:17: Unexpected character '≤'.
27:21: Unexpected character '≤'.
27:34: Unexpected character '−'.
27:39: Unexpected character '−'.
28:17: Unexpected character '≥'.
28:30: Unexpected character '−'.
30:39: Unexpected character '−'.
30:56: Unexpected character '−'.
30:59: Unexpected character '−'.
30:70: Unexpected character '−'.
30:74: Unexpected character '✓'.
30:98: Unexpected character '−'.
30:101: Unexpected character '−'.
30:110: Unexpected character '−'.
30:114: Unexpected character '✓'.
32:18: Unexpected character '∈'.
32:37: Unexpected character '≥'.
38:19: Unexpected character '−'.
38:37: Unexpected character '≤'.
39:16: Unexpected character '−'.
39:34: Unexpected character '≥'.
41:13: Unexpected character '≥'.
41:84: Unexpected character '∩'.
41:103: Unexpected character '≥'.
41:126: Unexpected character '≤'.
41:145: Unexpected character '≤'.
41:159: Unexpected character '−'.
41:163: Unexpected character '≥'.
43:19: Unexpected character '≥'.
43:66: Unexpected character '−'.
43:71: Unexpected character '−'.
43:80: Unexpected character '≥'.
43:84: Unexpected character '→'.
43:88: Unexpected character '≥'.
43:92: Unexpected character '−'.
43:98: Unexpected character '−'.
43:103: Unexpected character '−'.
43:170: Unexpected character '—'.
43:265: Unexpected character '−'.
43:271: Unexpected character '−'.
43:276: Unexpected character '−'.
43:296: Unexpected character '≤'.
43:325: Char literals must contain exactly one character and close with '''.
43:637: Char literals must contain exactly one character and close with '''.
43:1094: Unexpected character '→'.
43:1113: Unexpected character '≥'.
43:1141: Unexpected character '≥'.
43:1178: Unexpected character '→'.
47:25: Unexpected character '≥'.
47:104: Unexpected character '≥'.
51:82: Unexpected character '≥'.
51:90: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 263
OUT1 443 |
| ✓ |
LcbFirstAbc2LiveCodeBench / atcoder / easy: abc322_a First ABC 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 186
OUT117 |
| ✓ |
LcbKth321LikeLiveCodeBench / atcoder / medium: abc321_c K-th 321-like Number (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN36 575
OUT4 039 |
| × |
LcbRelativePositionLiveCodeBench / atcoder / medium: abc320_d Relative Position (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RelativePosition case1: expected [0 0
2 1
-1 -2] got [0 0
undecidable
undecidable] |
Failed |
IN31 876
OUT1 035 |
| × |
LcbSetMenuLiveCodeBench / atcoder / medium: abc321_d Set Menu (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SetMenu case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 221
OUT408 |
| × |
LcbSlotStrategy2LiveCodeBench / atcoder / medium: abc320_c Slot Strategy 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SlotStrategy2 case0: expected [6] got [0] |
Failed |
IN9 260
OUT266 |
| ✓ |
LcbSomenNagashiLiveCodeBench / atcoder / hard: abc320_e Somen Nagashi (Pattern B, stdin)1 AI query
4 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN73 120
OUT9 218 |
Code Generation - LiveCodeBench v1 AtCoder (set 13)
3 passed6 failed |
33.3% |
IN83 159
OUT10 329 |
| ✓ |
LcbFestivalLiveCodeBench / atcoder / medium: abc322_c Festival (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 220
OUT238 |
| ✓ |
LcbMergeSlimesLiveCodeBench / atcoder / hard: abc323_d Merge Slimes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 202
OUT1 791 |
| × |
LcbPlaylistProbLiveCodeBench / atcoder / hard: abc323_e Playlist (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PlaylistProb case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 297
OUT2 876 |
| × |
LcbPolyominoLiveCodeBench / atcoder / medium: abc322_d Polyomino (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Polyomino case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
98:6: Not all code paths in function 'PlaceVariant' return a value.
Values:
run.Compiled = false |
Failed |
IN9 274
OUT1 726 |
| ✓ |
LcbPrefixSuffixLiveCodeBench / atcoder / easy: abc322_b Prefix and Suffix (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 232
OUT1 008 |
| × |
LcbProductDevelopmentLiveCodeBench / atcoder / hard: abc322_e Product Development (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ProductDevelopment case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:47: Unexpected character '≥'.
5:17: Char literals must contain exactly one character and close with '''.
7:6: Unexpected character '—'.
7:82: Char literals must contain exactly one character and close with '''.
9:66: Unexpected character '≥'.
9:232: Unexpected character '≤'.
9:236: Unexpected character '→'.
11:183: Unexpected character '·'.
13:85: Char literals must contain exactly one character and close with '''.
15:125: Char literals must contain exactly one character and close with '''.
17:124: Char literals must contain exactly one character and close with '''.
19:54: Unexpected character '—'.
19:157: Unexpected character '≥'.
21:2: Char literals must contain exactly one character and close with '''.
23:21: Char literals must contain exactly one character and close with '''.
25:108: Char literals must contain exactly one character and close with '''.
27:134: Unexpected character '≥'.
37:17: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 235
OUT1 702 |
| × |
LcbRoundRobinRankLiveCodeBench / atcoder / easy: abc323_b Round (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RoundRobinRank case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:32: Expected '{' to start a block, found '('.
Values:
run.Compiled = false |
Failed |
IN9 243
OUT400 |
| × |
LcbWeakBeatsLiveCodeBench / atcoder / easy: abc323_a Weak Beats (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WeakBeats case1: expected [Yes] got [No] |
Failed |
IN9 188
OUT83 |
| × |
LcbWorldTourFinalsLiveCodeBench / atcoder / medium: abc323_c World Tour Finals (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WorldTourFinals case0: expected [0
1
1] got [1
1
1] |
Failed |
IN9 268
OUT505 |
Code Generation - LiveCodeBench v1 AtCoder (set 14)
3 passed3 failed1 skipped |
50.0% |
IN55 080
OUT2 010 |
| ✓ |
Lcb2x3yLiveCodeBench / atcoder / easy: abc324_b 2^x 3^y (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 152
OUT100 |
| ✓ |
LcbAllValuesEqualLiveCodeBench / atcoder / easy: abc324_a Same (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT112 |
| – |
LcbBeautifulPathLiveCodeBench / atcoder / hard: abc324_f Beautiful Path (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: float stdout: stdin driver compares stdout by exact trimmed string, cannot tolerance-match a floating-point ratio like 0.7500000000000000 |
Skipped |
IN0
OUT0 |
| × |
LcbErrorCorrectionLiveCodeBench / atcoder / medium: abc324_c Error Correction (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ErrorCorrection case0: expected [4
1 2 3 4] got [2
1 4] |
Failed |
IN9 268
OUT536 |
| × |
LcbJointTwoStringsLiveCodeBench / atcoder / hard: abc324_e Joint Two Strings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: JointTwoStrings case0 runtime error: List index -1 is out of range. The list has 4 element(s).
Values:
run.Success = false |
Failed |
IN9 202
OUT648 |
| × |
LcbSquarePermutationLiveCodeBench / atcoder / hard: abc324_d Square Permutation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SquarePermutation case0: expected [2] got [0] |
Failed |
IN9 165
OUT579 |
| ✓ |
LcbSurnameSanLiveCodeBench / atcoder / easy: abc325_a Takahashi san (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 144
OUT35 |
Code Generation - LiveCodeBench v1 AtCoder (set 15)
5 passed3 failed1 skipped |
62.5% |
IN101 710
OUT10 192 |
| ✓ |
Lcb326LikeNumberLiveCodeBench / atcoder / abc326_b: 3261 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 225
OUT119 |
| – |
LcbAbcPuzzleLiveCodeBench / atcoder / abc326_d: ABC Puzzle NOTE: output is NON-UNIQUE (any valid grid is accepted); exact string compare is unsafe -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: non-unique output: any valid grid is accepted, exact string compare unsafe |
Skipped |
IN0
OUT0 |
| ✓ |
LcbCarTrainTravelLiveCodeBench / atcoder / abc325_e: Our clients, please wait a moment1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN36 774
OUT5 272 |
| ✓ |
LcbGiftIntervalLiveCodeBench / atcoder / abc326_c: Peak1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 249
OUT218 |
| × |
LcbPrintingMachineLiveCodeBench / atcoder / abc325_d: Printing Machine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PrintingMachine case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:228: Unexpected character '≥'.
3:252: Unexpected character '∈'.
5:243: Unexpected character '≥'.
5:348: Char literals must contain exactly one character and close with '''.
7:69: Unexpected character '≥'.
9:101: Unexpected character '≤'.
11:70: Unexpected character '≤'.
11:173: Unexpected character '≥'.
11:302: Unexpected character '≤'.
11:322: Char literals must contain exactly one character and close with '''.
13:96: Unexpected character '≥'.
13:161: Unexpected character '—'.
13:256: Unexpected character '∈'.
13:343: Unexpected character '≤'.
13:365: Unexpected character '≥'.
16:56: Unexpected character '≤'.
18:494: Unexpected character '≥'.
20:188: Char literals must contain exactly one character and close with '''.
20:356: Unexpected character '→'.
20:427: Unexpected character '≤'.
20:435: Unexpected character '≥'.
20:590: Unexpected character '≤'.
20:602: Unexpected character '≥'.
20:906: Char literals must contain exactly one character and close with '''.
22:257: Unexpected character '—'.
24:158: Unexpected character '≤'.
28:69: Unexpected character '≤'.
30:5: Char literals must contain exactly one character and close with '''.
34:20: Unexpected character '#'.
36:155: Unexpected character '—'.
40:9: Unexpected character '#'.
48:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 286
OUT2 407 |
| ✓ |
LcbSensorClustersLiveCodeBench / atcoder / abc325_c: Sensors1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 310
OUT678 |
| × |
LcbSensorCostDilemmaLiveCodeBench / atcoder / abc325_f: Sensor Optimization Dilemma1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SensorCostDilemma case0: expected [17] got [15] |
Failed |
IN9 380
OUT1 071 |
| ✓ |
LcbStairsOrElevatorLiveCodeBench / atcoder / abc326_a: 2UP3DOWN1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 201
OUT99 |
| × |
LcbWorldMeetingLiveCodeBench / atcoder / abc325_b: World Meeting1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WorldMeeting case0: expected [8] got [6] |
Failed |
IN9 285
OUT328 |
Code Generation - LiveCodeBench v1 AtCoder (set 16)
6 passed1 failed1 skipped |
85.7% |
IN75 767
OUT4 166 |
| ✓ |
LcbAdjacentABLiveCodeBench / atcoder / abc327_a: ab1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 198
OUT125 |
| ✓ |
LcbGoodTupleLiveCodeBench / atcoder / abc327_d: Good Tuple Problem1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 279
OUT413 |
| – |
LcbMaximizeRatingLiveCodeBench / atcoder / abc327_e: Maximize Rating NOTE: output is a FLOAT with tolerance 1e-6; exact trimmed string compare is unsafe -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: floating-point output with 1e-6 tolerance; exact string compare unsafe |
Skipped |
IN0
OUT0 |
| ✓ |
LcbNotTooHardLiveCodeBench / atcoder / abc328_a: Not Too Hard1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 199
OUT118 |
| ✓ |
LcbNumberPlaceLiveCodeBench / atcoder / abc327_c: Number Place1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 297
OUT423 |
| ✓ |
LcbPerfectPowerAALiveCodeBench / atcoder / abc327_b: A^A1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 179
OUT1 426 |
| × |
LcbRepdigitDatesLiveCodeBench / atcoder / abc328_b: 11/111 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RepdigitDates case0: expected [13] got [12] |
Failed |
IN9 216
OUT216 |
| ✓ |
LcbSalaryExpectationLiveCodeBench / atcoder / abc326_e: Revenge of "The Salary of AtCoder Inc."1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN20 399
OUT1 445 |
Code Generation - LiveCodeBench v1 AtCoder (set 17)
5 passed3 failed |
62.5% |
IN96 801
OUT43 203 |
| ✓ |
LcbConsecutivePairsLiveCodeBench / atcoder / abc328_c: Consecutive1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 283
OUT255 |
| × |
LcbCountRepeatsLiveCodeBench / atcoder / abc329_c: Count xxx1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountRepeats case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 198
OUT397 |
| ✓ |
LcbElectionReportLiveCodeBench / atcoder / abc329_d: Election Quick Report1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN21 178
OUT2 810 |
| × |
LcbModuloMstLiveCodeBench / atcoder / abc328_e: Modulo MST1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ModuloMst case0: expected [33] got [328] |
Failed |
IN20 325
OUT684 |
| × |
LcbSecondLargestLiveCodeBench / atcoder / abc329_b: Next1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SecondLargest case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:15: Variable 'tokens' has type 'List<int>' but its initializer has type 'List<string>'.
8:27: 'int.Parse' expects a string argument, but it has type 'int'.
14:27: 'int.Parse' expects a string argument, but it has type 'int'.
Values:
run.Compiled = false |
Failed |
IN9 187
OUT139 |
| ✓ |
LcbSpreadCharsLiveCodeBench / atcoder / abc329_a: Spread1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT71 |
| ✓ |
LcbStampMatchLiveCodeBench / atcoder / abc329_e: Stamp1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 254
OUT38 616 |
| ✓ |
LcbTakeAbcLiveCodeBench / atcoder / abc328_d: Take ABC1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 201
OUT231 |
Code Generation - LiveCodeBench v1 AtCoder (set 18)
5 passed4 failed |
55.6% |
IN116 303
OUT5 256 |
| ✓ |
LcbColoredBallLiveCodeBench / atcoder / hard: Colored Ball [LCB id: abc329_f]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 247
OUT813 |
| ✓ |
LcbCountingLsLiveCodeBench / atcoder / medium: Counting Ls [LCB id: abc330_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 296
OUT297 |
| ✓ |
LcbCountingPassesLiveCodeBench / atcoder / easy: Counting Passes [LCB id: abc330_a]1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN31 620
OUT571 |
| × |
LcbEggPacksLiveCodeBench / atcoder / easy: Buy One Carton of Milk (egg packs) [LCB id: abc331_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: EggPacks case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 178
OUT247 |
| × |
LcbMexAndUpdateLiveCodeBench / atcoder / hard: Mex and Update [LCB id: abc330_e]1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MexAndUpdate case0: expected [4
3
6
5
0] got [4
3
6
5
5] |
Failed |
IN20 205
OUT616 |
| ✓ |
LcbMinimizeAbs1LiveCodeBench / atcoder / easy: Minimize Abs 1 [LCB id: abc330_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 240
OUT172 |
| × |
LcbMinimizeAbs2LiveCodeBench / atcoder / medium: Minimize Abs 2 [LCB id: abc330_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimizeAbs2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:24: Expected ')' to close the argument list, found 'best'.
Values:
run.Compiled = false |
Failed |
IN9 146
OUT2 092 |
| ✓ |
LcbSumGreaterThanMeLiveCodeBench / atcoder / medium: Sum of Numbers Greater Than Me [LCB id: abc331_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 188
OUT288 |
| × |
LcbTomorrowCalendarLiveCodeBench / atcoder / easy: Tomorrow [LCB id: abc331_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TomorrowCalendar case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:14: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 183
OUT160 |
Code Generation - LiveCodeBench v1 AtCoder (set 19)
5 passed3 failed |
62.5% |
IN87 149
OUT8 492 |
| ✓ |
LcbGlassAndMugLiveCodeBench / atcoder / easy: Glass and Mug [LCB id: abc332_b]1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN22 107
OUT2 263 |
| ✓ |
LcbOnlineShoppingLiveCodeBench / atcoder / easy: Online Shopping [LCB id: abc332_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 238
OUT165 |
| ✓ |
LcbPentagonDiagonalLiveCodeBench / atcoder / easy: Pentagon [LCB id: abc333_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 250
OUT267 |
| × |
LcbSetMealLiveCodeBench / atcoder / hard: Set Meal [LCB id: abc331_e]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SetMeal case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:78: Char literals must contain exactly one character and close with '''.
15:42: Unexpected character '≤'.
15:45: Unexpected character '×'.
15:57: Unexpected character '≤'.
15:69: Unexpected character '×'.
18:236: Char literals must contain exactly one character and close with '''.
24:52: Char literals must contain exactly one character and close with '''.
26:39: Char literals must contain exactly one character and close with '''.
28:103: Char literals must contain exactly one character and close with '''.
30:86: Unexpected character '×'.
30:107: Unexpected character '×'.
32:12: Unexpected character '≤'.
37:9: Unexpected character '#'.
37:16: Char literals must contain exactly one character and close with '''.
39:72: Unexpected character '≤'.
39:75: Unexpected character '×'.
39:84: Unexpected character '≤'.
39:87: Unexpected character '×'.
42:121: Char literals must contain exactly one character and close with '''.
44:195: Char literals must contain exactly one character and close with '''.
46:88: Unexpected character '→'.
48:96: Char literals must contain exactly one character and close with '''.
50:219: Char literals must contain exactly one character and close with '''.
50:278: Unexpected character '≥'.
52:2: Char literals must contain exactly one character and close with '''.
54:253: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 306
OUT1 390 |
| × |
LcbSwappingPuzzleLiveCodeBench / atcoder / hard: Swapping Puzzle [LCB id: abc332_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SwappingPuzzle case0: expected [3] got [-1] |
Failed |
IN9 364
OUT729 |
| ✓ |
LcbThreeThreesLiveCodeBench / atcoder / easy: Three Threes [LCB id: abc333_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 142
OUT65 |
| × |
LcbTilePatternLiveCodeBench / atcoder / hard: Tile Pattern [LCB id: abc331_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TilePattern case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
4:16: Unexpected character '×'.
4:58: Unexpected character '×'.
5:49: Unexpected character '×'.
7:45: Unexpected character '×'.
9:90: Unexpected character '×'.
9:150: Char literals must contain exactly one character and close with '''.
17:13: Unexpected character '×'.
21:49: Unexpected character '→'.
25:11: Unexpected character '×'.
38:33: Unexpected character '×'.
42:30: Unexpected character '×'.
47:44: Unexpected character '→'.
51:93: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 346
OUT1 711 |
| ✓ |
LcbTShirtsLiveCodeBench / atcoder / medium: T-Shirts [LCB id: abc332_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 396
OUT1 902 |
Code Generation - LiveCodeBench v1 AtCoder (set 2)
4 passed4 failed |
50.0% |
IN87 825
OUT6 631 |
| ✓ |
LcbAlmostEqualRearrangeLiveCodeBench / atcoder abc302_c / medium: Almost Equal (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 220
OUT1 691 |
| × |
LcbDashHealthLiveCodeBench / atcoder abc303_c / medium: Dash (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DashHealth case0: expected [Yes] got [No] |
Failed |
IN9 312
OUT332 |
| ✓ |
LcbDiscordBadMoodLiveCodeBench / atcoder abc303_b / easy: Discord (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 237
OUT211 |
| × |
LcbImpartialGiftLiveCodeBench / atcoder abc302_d / medium: Impartial Gift (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ImpartialGift case0: expected [8] got [25] |
Failed |
IN9 262
OUT316 |
| × |
LcbIsolationQueriesLiveCodeBench / atcoder abc302_e / hard: Isolation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: IsolationQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
31:27: Type 'List<int>' has no supported method named 'Remove' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 249
OUT338 |
| × |
LcbMergeSetLiveCodeBench / atcoder abc302_f / hard: Merge Set (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MergeSet case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:36: Unknown function 'Find'.
22:20: Unknown function 'Union'.
22:45: Unknown function 'Find'.
25:9: Unknown function 'Find'.
25:28: Unknown function 'Find'.
30:19: Unknown function 'Find'.
31:18: Unknown function 'Find'.
38:25: Unknown function 'Find'.
Values:
run.Compiled = false |
Failed |
IN9 304
OUT431 |
| ✓ |
LcbShiftVsCapsLockLiveCodeBench / atcoder abc303_d / medium: Shift vs. CapsLock (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN22 990
OUT3 145 |
| ✓ |
LcbSimilarStringLiveCodeBench / atcoder abc303_a / easy: Similar String (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 251
OUT167 |
Code Generation - LiveCodeBench v1 AtCoder (set 20)
4 passed3 failed1 skipped |
57.1% |
IN93 510
OUT15 852 |
| ✓ |
LcbChristmasColorGridLiveCodeBench / atcoder / hard: Christmas Color Grid 1 [LCB id: abc334_e]1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN27 130
OUT7 898 |
| ✓ |
LcbChristmasPresentLiveCodeBench / atcoder / easy: Christmas Present [LCB id: abc334_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 159
OUT67 |
| × |
LcbChristmasTreesLiveCodeBench / atcoder / medium: Christmas Trees [LCB id: abc334_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ChristmasTrees case1: expected [0] got [1] |
Failed |
IN9 236
OUT138 |
| ✓ |
LcbEraseLeavesLiveCodeBench / atcoder / medium: Erase Leaves [LCB id: abc333_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 238
OUT1 726 |
| ✓ |
LcbReindeerSleighLiveCodeBench / atcoder / medium: Reindeer and Sleigh [LCB id: abc334_d]1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN20 312
OUT487 |
| × |
LcbRepunitTrioLiveCodeBench / atcoder / medium: Repunit Trio [LCB id: abc333_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: RepunitTrio case2 runtime error: Integer overflow.
Values:
run.Success = false |
Failed |
IN9 180
OUT1 525 |
| × |
LcbSocks2LiveCodeBench / atcoder / medium: Socks 2 [LCB id: abc334_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Socks2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
46:19: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 255
OUT4 011 |
| – |
LcbTakahashiQuestLiveCodeBench / atcoder / hard: Takahashi Quest [LCB id: abc333_e]0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: non-unique output: any valid pick-up sequence achieving K_min is accepted, cannot value-compare against a single oracle line |
Skipped |
IN0
OUT0 |
Code Generation - LiveCodeBench v1 AtCoder (set 21)
8 passed1 skipped |
100.0% |
IN89 363
OUT5 521 |
| ✓ |
Lcb2023To2024LiveCodeBench / atcoder / abc335_a: 2023 -> 2024 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT53 |
| ✓ |
LcbCtzLiveCodeBench / atcoder / abc336_b: CTZ (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 176
OUT68 |
| ✓ |
LcbEvenDigitsLiveCodeBench / atcoder / abc336_c: Even Digits (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 233
OUT1 402 |
| ✓ |
LcbLongLoongLiveCodeBench / atcoder / abc336_a: Long Loong (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 176
OUT87 |
| – |
LcbLoongAndTakahashiLiveCodeBench / atcoder / abc335_d: Loong and Takahashi (Pattern B, stdin) [NON-UNIQUE output]0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbLoongTrackingLiveCodeBench / atcoder / abc335_c: Loong Tracking (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 242
OUT442 |
| ✓ |
LcbPyramidSequenceLiveCodeBench / atcoder / abc336_d: Pyramid (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN24 985
OUT3 124 |
| ✓ |
LcbScoreboardLiveCodeBench / atcoder / abc337_a: Scoreboard (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 195
OUT158 |
| ✓ |
LcbTetrahedralNumberLiveCodeBench / atcoder / abc335_b: Tetrahedral Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 207
OUT187 |
Code Generation - LiveCodeBench v1 AtCoder (set 22)
5 passed2 failed1 skipped |
71.4% |
IN64 426
OUT7 343 |
| – |
LcbBadJuiceLiveCodeBench / atcoder / abc337_e: Bad Juice (INTERACTIVE — not expressible as batch stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: SC# subset: interactive problem (two-way judge interaction + non-unique valid output) |
Skipped |
IN0
OUT0 |
| ✓ |
LcbCapitalizedLiveCodeBench / atcoder / abc338_a: Capitalized? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 143
OUT133 |
| × |
LcbCheatingGomokuLiveCodeBench / atcoder / abc337_d: Cheating Gomoku Narabe (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CheatingGomoku case1: expected [0] got [-1] |
Failed |
IN9 232
OUT609 |
| ✓ |
LcbExtendedAbcLiveCodeBench / atcoder / abc337_b: Extended ABC (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 198
OUT115 |
| ✓ |
LcbFrequencyLiveCodeBench / atcoder / abc338_b: Frequency (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 146
OUT189 |
| ✓ |
LcbIslandTourLiveCodeBench / atcoder / abc338_d: Island Tour (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 284
OUT3 749 |
| × |
LcbLeftoverRecipesLiveCodeBench / atcoder / abc338_c: Leftover Recipes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: LeftoverRecipes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
7:95: Char literals must contain exactly one character and close with '''.
13:140: Char literals must contain exactly one character and close with '''.
15:156: Char literals must contain exactly one character and close with '''.
17:922: Unexpected character '→'.
17:964: Char literals must contain exactly one character and close with '''.
17:1069: Unexpected character '→'.
17:1417: Unexpected character '→'.
17:1462: Unexpected character '⇒'.
19:89: Unexpected character '±'.
21:275: Char literals must contain exactly one character and close with '''.
23:201: Unexpected character '≤'.
25:21: Char literals must contain exactly one character and close with '''.
27:95: Unexpected character '→'.
27:133: Unexpected character '→'.
27:177: Unexpected character '→'.
27:220: Unexpected character '→'.
29:59: Unexpected character '±'.
31:67: Char literals must contain exactly one character and close with '''.
33:112: Unexpected character '≥'.
37:40: Unexpected character '—'.
37:123: Unexpected character '→'.
37:207: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 226
OUT2 296 |
| ✓ |
LcbLiningUp2LiveCodeBench / atcoder / abc337_c: Lining Up 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 197
OUT252 |
Code Generation - LiveCodeBench v1 AtCoder (set 23)
5 passed3 failed |
62.5% |
IN84 561
OUT7 524 |
| ✓ |
LcbAppendQueryLiveCodeBench / atcoder / abc340_b: Append (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 185
OUT196 |
| ✓ |
LcbArithmeticProgressionLiveCodeBench / atcoder / abc340_a: Arithmetic Progression (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 150
OUT198 |
| × |
LcbChordsLiveCodeBench / atcoder / abc338_e: Chords (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Chords case1: expected [No] got [Yes] |
Failed |
IN9 193
OUT257 |
| ✓ |
LcbLangtonsTakahashiLiveCodeBench / atcoder / abc339_b: Langton's Takahashi (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 255
OUT1 393 |
| × |
LcbNegativeTspLiveCodeBench / atcoder / abc338_f: Negative Traveling Salesman (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NegativeTsp case0: expected [-2] got [103] |
Failed |
IN20 185
OUT3 638 |
| ✓ |
LcbPerfectBusLiveCodeBench / atcoder / abc339_c: Perfect Bus (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 192
OUT942 |
| × |
LcbSynchronizedPlayersLiveCodeBench / atcoder / abc339_d: Synchronized Players (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SynchronizedPlayers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
33:9: Variable 'dr' has type 'int' but its initializer has type 'List<int>'.
34:9: Variable 'dc' has type 'int' but its initializer has type 'List<int>'.
55:29: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
56:29: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
Values:
run.Compiled = false |
Failed |
IN9 242
OUT840 |
| ✓ |
LcbTldLiveCodeBench / atcoder / abc339_a: TLD (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 159
OUT60 |
Code Generation - LiveCodeBench v1 AtCoder (set 24)
5 passed5 failed |
50.0% |
IN117 947
OUT16 960 |
| × |
LcbAlternatingStringLiveCodeBench / atcoder abc341_e / hard: Alternating String (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AlternatingString case0: expected [Yes
No
Yes
No] got [Yes
No
No
No] |
Failed |
IN9 263
OUT545 |
| × |
LcbBreakdownLiveCodeBench / atcoder abc341_f / hard: Breakdown (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Breakdown case0: expected [5] got [4] |
Failed |
IN34 765
OUT3 966 |
| ✓ |
LcbDivideAndDivideLiveCodeBench / atcoder abc340_c / medium: Divide and Divide (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 248
OUT9 318 |
| × |
LcbForeignExchangeLiveCodeBench / atcoder abc341_b / easy: Foreign Exchange (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ForeignExchange case0: expected [5] got [4] |
Failed |
IN9 301
OUT131 |
| ✓ |
LcbMancala2LiveCodeBench / atcoder abc340_e / hard: Mancala 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 303
OUT377 |
| ✓ |
LcbOnlyOneOfTwoLiveCodeBench / atcoder abc341_d / medium: Only one of two (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 187
OUT362 |
| ✓ |
LcbPrint341LiveCodeBench / atcoder abc341_a / easy: Print 341 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 148
OUT84 |
| × |
LcbSuperTakahashiBrosLiveCodeBench / atcoder abc340_d / hard: Super Takahashi Bros. (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SuperTakahashiBros case0: expected [350] got [-1] |
Failed |
IN9 292
OUT242 |
| ✓ |
LcbTakahashiGetsLostLiveCodeBench / atcoder abc341_c / medium: Takahashi Gets Lost (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 277
OUT1 681 |
| × |
LcbYayLiveCodeBench / atcoder abc342_a / easy: Yay! (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Yay case1: expected [1] got [2] |
Failed |
IN9 163
OUT254 |
Code Generation - LiveCodeBench v1 AtCoder (set 25)
4 passed3 failed2 skipped |
57.1% |
IN86 641
OUT5 745 |
| ✓ |
LcbAdjacencyMatrixLiveCodeBench / atcoder abc343_b / easy: Adjacency Matrix (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 254
OUT151 |
| ✓ |
LcbDiversityOfScoresLiveCodeBench / atcoder abc343_d / medium: Diversity of Scores (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 274
OUT377 |
| × |
LcbLastTrainLiveCodeBench / atcoder abc342_e / hard: Last Train (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: LastTrain case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:13: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 411
OUT3 625 |
| × |
LcbManyReplacementLiveCodeBench / atcoder abc342_c / medium: Many Replacement (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ManyReplacement case0: expected [recover] got [aecovea] |
Failed |
IN9 240
OUT242 |
| ✓ |
LcbPalindromicCubeLiveCodeBench / atcoder abc343_c / medium: 343 (palindromic cube) (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN20 068
OUT333 |
| – |
LcbSevenCubedLiveCodeBench / atcoder abc343_e / hard: 7x7x7 (Pattern B, stdin) Non-unique output: any valid 9-integer placement is accepted; exact-match oracle cannot judge the Yes case.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: Non-unique output: any valid tuple is accepted for the Yes case; exact-match oracle cannot judge |
Skipped |
IN0
OUT0 |
| × |
LcbSquarePairLiveCodeBench / atcoder abc342_d / medium: Square Pair (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SquarePair case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN20 155
OUT767 |
| ✓ |
LcbWhichIsAheadLiveCodeBench / atcoder abc342_b / easy: Which is ahead? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 239
OUT250 |
| – |
LcbWrongAnswerLiveCodeBench / atcoder abc343_a / easy: Wrong Answer (Pattern B, stdin) Non-unique output: ANY integer 0..9 not equal to A+B is accepted; exact-match oracle cannot judge it.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchSkipped: Non-unique output: any integer 0..9 != A+B is a valid answer; exact-match oracle cannot judge |
Skipped |
IN0
OUT0 |
Code Generation - LiveCodeBench v1 AtCoder (set 3)
5 passed4 failed |
55.6% |
IN117 588
OUT4 819 |
| ✓ |
LcbAbcdefgDistLiveCodeBench / atcoder abc305_b / easy: ABCDEFG (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 212
OUT169 |
| ✓ |
LcbCookiePickerLiveCodeBench / atcoder abc305_c / medium: Snuke the Cookie Picker (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 236
OUT338 |
| ✓ |
LcbFirstPlayerLiveCodeBench / atcoder abc304_a / easy: First Player (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 208
OUT245 |
| × |
LcbGiftFromStarsLiveCodeBench / atcoder abc303_e / hard: A Gift From the Stars (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GiftFromStars case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
54:21: Type 'List<int>' has no supported method named 'Remove' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN32 319
OUT1 335 |
| × |
LcbGoodGraphLiveCodeBench / atcoder abc304_e / hard: Good Graph (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GoodGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN20 634
OUT1 344 |
| × |
LcbPieceOfCakeLiveCodeBench / atcoder abc304_d / medium: A Piece of Cake (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PieceOfCake case0: expected [0 2] got [0 4] |
Failed |
IN9 258
OUT818 |
| × |
LcbSubscribersLiveCodeBench / atcoder abc304_b / easy: Subscribers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Subscribers case0: expected [20200000] got [20000000] |
Failed |
IN9 316
OUT77 |
| ✓ |
LcbVirusSpreadLiveCodeBench / atcoder abc304_c / medium: Virus (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 207
OUT392 |
| ✓ |
LcbWaterStationLiveCodeBench / atcoder abc305_a / easy: Water Station (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 198
OUT101 |
Code Generation - LiveCodeBench v1 AtCoder (set 4)
2 passed6 failed |
25.0% |
IN100 111
OUT8 105 |
| × |
LcbArtGalleryLiveCodeBench / atcoder abc305_e / hard: Art Gallery on Graph (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ArtGallery case0: expected [4
1 2 3 5] got [5
1 2 3 4 5] |
Failed |
IN9 254
OUT639 |
| ✓ |
LcbBase2LiveCodeBench / atcoder abc306_b / easy: Base 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 197
OUT100 |
| × |
LcbBestPerformancesLiveCodeBench / atcoder abc306_e / hard: Best Performances (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BestPerformances case0: expected [5
6
8
8
15
13
13
11
1
0] got [6
6
8
8
15
13
13
11
2
2] |
Failed |
IN35 550
OUT5 141 |
| × |
LcbCentersLiveCodeBench / atcoder abc306_c / medium: Centers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: Centers case0 runtime error: The dictionary does not contain the key 1. Use ContainsKey to check for it first.
Values:
run.Success = false |
Failed |
IN9 208
OUT750 |
| ✓ |
LcbEchoStringLiveCodeBench / atcoder abc306_a / easy: Echo (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 172
OUT85 |
| × |
LcbPoisonousFullLiveCodeBench / atcoder abc306_d / medium: Poisonous Full (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PoisonousFull case1: expected [0] got [-3] |
Failed |
IN9 259
OUT774 |
| × |
LcbSleepLogLiveCodeBench / atcoder abc305_d / hard: Sleep Log (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SleepLog case0: expected [480
0
960] got [2
0
3] |
Failed |
IN9 261
OUT449 |
| × |
LcbWeeklyRecordsLiveCodeBench / atcoder abc307_a / easy: Weekly Records (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WeeklyRecords case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:16: Variable 'parts' has type 'List<long>' but its initializer has type 'List<string>'.
12:36: 'long.Parse' expects a string argument, but it has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 210
OUT167 |
Code Generation - LiveCodeBench v1 AtCoder (set 5)
4 passed4 failed |
50.0% |
IN133 708
OUT13 143 |
| ✓ |
LcbDefaultPriceLiveCodeBench / atcoder abc308_b / easy: Default Price (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 213
OUT210 |
| × |
LcbDistinctAdjacentLiveCodeBench / atcoder abc307_e / hard: Distinct Adjacent (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DistinctAdjacent case0: expected [6] got [5] |
Failed |
IN9 214
OUT242 |
| × |
LcbIdealSheetLiveCodeBench / atcoder abc307_c / medium: Ideal Sheet (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: IdealSheet case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
84:51: Expected ')' to close the parenthesized expression, found '('.
Values:
run.Compiled = false |
Failed |
IN20 307
OUT1 924 |
| ✓ |
LcbMismatchedParensLiveCodeBench / atcoder abc307_d / medium: Mismatched Parentheses (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 186
OUT3 090 |
| ✓ |
LcbNewSchemeLiveCodeBench / atcoder abc308_a / easy: New Scheme (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 205
OUT181 |
| ✓ |
LcbRacecarPalindromeLiveCodeBench / atcoder abc307_b / easy: racecar (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 212
OUT2 734 |
| × |
LcbSnukeMazeLiveCodeBench / atcoder abc308_d / medium: Snuke Maze (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SnukeMaze case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
31:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN33 400
OUT1 889 |
| × |
LcbStandingsLiveCodeBench / atcoder abc308_c / medium: Standings (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Standings case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
43:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN33 971
OUT2 873 |
Code Generation - LiveCodeBench v1 AtCoder (set 6)
3 passed6 failed |
33.3% |
IN100 052
OUT15 874 |
| ✓ |
LcbAddEdgeMaxDistanceLiveCodeBench / atcoder / medium: abc309_d Add One Edge1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 357
OUT1 832 |
| ✓ |
LcbDrinkCouponLiveCodeBench / atcoder / easy: abc310_a Order Something Else1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 227
OUT166 |
| × |
LcbFamilyInsuranceLiveCodeBench / atcoder / hard: abc309_e Family and Insurance1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: FamilyInsurance case0 runtime error: List index 7 is out of range. The list has 7 element(s).
Values:
run.Success = false |
Failed |
IN9 217
OUT750 |
| × |
LcbMedicineDayLiveCodeBench / atcoder / medium: abc309_c Medicine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: MedicineDay case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 238
OUT243 |
| × |
LcbMexTriplesLiveCodeBench / atcoder / hard: abc308_e MEX1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MexTriples case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:17: Unexpected character '≥'.
20:22: Unexpected character '→'.
20:28: Unexpected character '≥'.
21:24: Unexpected character '→'.
37:5: Char literals must contain exactly one character and close with '''.
43:91: Char literals must contain exactly one character and close with '''.
46:172: Unexpected character '≤'.
48:356: Char literals must contain exactly one character and close with '''.
48:391: Char literals must contain exactly one character and close with '''.
48:435: Unexpected character '≤'.
60:113: Char literals must contain exactly one character and close with '''.
62:64: Char literals must contain exactly one character and close with '''.
79:4: Char literals must contain exactly one character and close with '''.
85:2: Char literals must contain exactly one character and close with '''.
93:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 257
OUT2 057 |
| ✓ |
LcbNineAdjacentLiveCodeBench / atcoder / easy: abc309_a Nine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 226
OUT142 |
| × |
LcbRotateOuterLiveCodeBench / atcoder / easy: abc309_b Rotate1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: RotateOuter case0 runtime error: List index 4 is out of range. The list has 4 element(s).
Values:
run.Success = false |
Failed |
IN9 316
OUT462 |
| × |
LcbStrictlySuperiorLiveCodeBench / atcoder / easy: abc310_b Strictly Superior1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: StrictlySuperior case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:60: Unexpected character '⊆'.
6:79: Char literals must contain exactly one character and close with '''.
8:9: Unexpected character '≠'.
8:21: Char literals must contain exactly one character and close with '''.
10:68: Char literals must contain exactly one character and close with '''.
11:14: Unexpected character '→'.
11:27: Unexpected character '✓'.
12:28: Unexpected character '→'.
13:13: Unexpected character '→'.
13:31: Unexpected character '≥'.
13:51: Unexpected character '→'.
14:45: Unexpected character '≠'.
16:38: Unexpected character '≠'.
16:121: Unexpected character '≤'.
16:130: Unexpected character '×'.
16:153: Unexpected character '²'.
16:155: Unexpected character '×'.
16:217: Char literals must contain exactly one character and close with '''.
20:93: Char literals must contain exactly one character and close with '''.
22:182: Char literals must contain exactly one character and close with '''.
26:29: Char literals must contain exactly one character and close with '''.
26:363: Unexpected character '→'.
28:50: Unexpected character '≤'.
28:61: Char literals must contain exactly one character and close with '''.
28:207: Char literals must contain exactly one character and close with '''.
30:78: Unexpected character '⊆'.
30:94: Unexpected character '⊆'.
32:23: Unexpected character '≠'.
32:51: Unexpected character '⊆'.
32:139: Unexpected character '⇒'.
40:171: Char literals must contain exactly one character and close with '''.
48:89: Unexpected character '≠'.
48:128: Unexpected character '≠'.
54:4: Char literals must contain exactly one character and close with '''.
74:4: Char literals must contain exactly one character and close with '''.
80:14: Unexpected character '→'.
96:61: Unexpected character '→'.
120:4: Char literals must contain exactly one character and close with '''.
166:5: Char literals must contain exactly one character and close with '''.
170:9: Unexpected character '#'.
180:223: Char literals must contain exactly one character and close with '''.
180:447: Char literals must contain exactly one character and close with '''.
180:1060: Unexpected character '—'.
184:17: Unexpected character '≠'.
192:4: Char literals must contain exactly one character and close with '''.
194:122: Unexpected character '≤'.
194:131: Char literals must contain exactly one character and close with '''.
200:81: Unexpected character '≥'.
214:2: Unexpected character '’'.
Values:
run.Compiled = false |
Failed |
IN9 356
OUT2 678 |
| × |
LcbVouchersDiscountLiveCodeBench / atcoder / hard: abc308_f Vouchers1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: VouchersDiscount case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
36:55: Expected ')' to close the parenthesized expression, found 'prof'.
Values:
run.Compiled = false |
Failed |
IN25 858
OUT7 544 |
Code Generation - LiveCodeBench v1 AtCoder (set 7)
4 passed3 failed1 skipped |
57.1% |
IN77 220
OUT6 592 |
| – |
LcbFindCycleLiveCodeBench / atcoder / medium: abc311_c Find it! (NON-UNIQUE output: any valid directed cycle is accepted; exact-match unreliable)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: non-unique output: any valid directed cycle is accepted, exact string match unreliable |
Skipped |
IN0
OUT0 |
| ✓ |
LcbFirstAbcLiveCodeBench / atcoder / easy: abc311_a First ABC1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 191
OUT189 |
| × |
LcbGridIceFloorLiveCodeBench / atcoder / medium: abc311_d Grid Ice Floor1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GridIceFloor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
47:38: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
49:32: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 349
OUT509 |
| × |
LcbMake10AgainLiveCodeBench / atcoder / hard: abc310_f Make 10 Again1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Make10Again case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN21 593
OUT4 789 |
| ✓ |
LcbNandRepeatedlyLiveCodeBench / atcoder / hard: abc310_e NAND repeatedly1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 296
OUT157 |
| × |
LcbPeacefulTeamsLiveCodeBench / atcoder / medium: abc310_d Peaceful Teams1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PeacefulTeams case0: expected [4] got [0] |
Failed |
IN9 273
OUT570 |
| ✓ |
LcbReversibleSticksLiveCodeBench / atcoder / medium: abc310_c Reversible1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 267
OUT174 |
| ✓ |
LcbVacationTogetherLiveCodeBench / atcoder / easy: abc311_b Vacation Together1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 251
OUT204 |
Code Generation - LiveCodeBench v1 AtCoder (set 8)
4 passed4 failed |
50.0% |
IN85 254
OUT9 977 |
| ✓ |
LcbBecomeStrongestLiveCodeBench / atcoder / easy: abc313_a To Be Saikyo1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 204
OUT135 |
| × |
LcbCansOpenersLiveCodeBench / atcoder / hard: abc312_f Cans and Openers1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CansOpeners case0: expected [27] got [130] |
Failed |
IN9 295
OUT473 |
| ✓ |
LcbChordCheckLiveCodeBench / atcoder / easy: abc312_a Chord1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 156
OUT81 |
| ✓ |
LcbCountBracketsLiveCodeBench / atcoder / medium: abc312_d Count Bracket Sequences1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 253
OUT333 |
| × |
LcbCuboidTangencyLiveCodeBench / atcoder / hard: abc312_e Tangency of Cuboids1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CuboidTangency case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
80:25: Type 'string' has no supported method named 'TrimEnd'.
Values:
run.Compiled = false |
Failed |
IN20 450
OUT3 402 |
| × |
LcbHolelessSquaresLiveCodeBench / atcoder / hard: abc311_e Defect1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: HolelessSquares case0 runtime error: List index 5 is out of range. The list has 3 element(s).
Values:
run.Success = false |
Failed |
IN9 324
OUT560 |
| ✓ |
LcbInvisibleHandLiveCodeBench / atcoder / medium: abc312_c Invisible Hand1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 212
OUT2 242 |
| × |
LcbTakCodeLiveCodeBench / atcoder / easy: abc312_b TaK Code1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: TakCode case0 runtime error: List index 20 is out of range. The list has 20 element(s).
Values:
run.Success = false |
Failed |
IN9 360
OUT2 751 |
Code Generation - LiveCodeBench v1 AtCoder (set 9)
5 passed3 failed1 skipped |
62.5% |
IN100 481
OUT10 326 |
| × |
LcbApproxEqualizationLiveCodeBench / atcoder / medium: abc313_c Approximate Equalization 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ApproxEqualization case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:16: Variable 'parts' has type 'List<long>' but its initializer has type 'List<string>'.
8:32: 'long.Parse' expects a string argument, but it has type 'long'.
15:34: 'long.Parse' expects a string argument, but it has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 191
OUT175 |
| ✓ |
LcbCertainGameExpectedLiveCodeBench / atcoder / hard: abc314_f A Certain Game (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN35 865
OUT7 593 |
| × |
LcbLowerUpperOpsLiveCodeBench / atcoder / medium: abc314_d LOWER (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: LowerUpperOps case0: expected [atcYber] got [atc9ber] |
Failed |
IN9 225
OUT406 |
| ✓ |
LcbPiTruncateLiveCodeBench / atcoder / easy: abc314_a 3.14 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 270
OUT193 |
| ✓ |
LcbRemoveVowelsLiveCodeBench / atcoder / easy: abc315_a tcdr (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 155
OUT90 |
| ✓ |
LcbRotateColoredSubsequenceLiveCodeBench / atcoder / medium: abc314_c Rotate Colored Subsequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 254
OUT368 |
| ✓ |
LcbRouletteFewestBetsLiveCodeBench / atcoder / easy: abc314_b Roulette (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 271
OUT1 266 |
| – |
LcbRoulettesExpectedLiveCodeBench / atcoder / hard: abc314_e Roulettes (Pattern B, stdin) FLAG: float output (expected value); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: SC# stdin driver: float/expected-value output needs tolerance comparison; exact string compare unsafe |
Skipped |
IN0
OUT0 |
| × |
LcbStrongestProgrammerLiveCodeBench / atcoder / medium: abc313_b Who is Saikyo? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: StrongestProgrammer case0: expected [1] got [-1] |
Failed |
IN9 250
OUT235 |
Code Generation - LiveCodeBench v1 Codeforces (set 1)
3 passed6 failed |
33.3% |
IN94 384
OUT7 228 |
| × |
LcbChemistryPalindromeLiveCodeBench / codeforces 1883_B / medium: Chemistry (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ChemistryPalindrome case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
7:27: Unexpected character '`'.
7:29: Unexpected character '`'.
7:84: Unexpected character '`'.
7:88: Unexpected character '`'.
9:38: Unexpected character '`'.
9:45: Unexpected character '`'.
9:69: Unexpected character '`'.
9:91: Unexpected character '`'.
9:98: Unexpected character '`'.
9:109: Unexpected character '`'.
9:122: Unexpected character '`'.
9:136: Unexpected character '`'.
11:14: Unexpected character '`'.
11:28: Unexpected character '`'.
11:41: Unexpected character '`'.
11:48: Unexpected character '`'.
11:54: Unexpected character '`'.
11:61: Unexpected character '`'.
11:85: Unexpected character '`'.
11:92: Unexpected character '`'.
11:121: Unexpected character '`'.
11:128: Unexpected character '`'.
11:140: Unexpected character '`'.
11:147: Unexpected character '`'.
11:176: Unexpected character '`'.
11:183: Unexpected character '`'.
11:202: Unexpected character '#'.
11:248: Unexpected character '`'.
11:250: Unexpected character '`'.
11:423: Unexpected character '#'.
11:432: Unexpected character '#'.
11:481: Unexpected character '#'.
11:563: Unexpected character '`'.
11:570: Unexpected character '`'.
13:130: Unexpected character '#'.
13:136: Unexpected character '≡'.
13:169: Unexpected character '#'.
13:175: Unexpected character '∈'.
13:190: Unexpected character '#'.
13:230: Unexpected character '→'.
13:232: Unexpected character '#'.
13:263: Unexpected character '→'.
13:265: Unexpected character '#'.
15:121: Char literals must contain exactly one character and close with '''.
21:104: Unexpected character '#'.
21:139: Unexpected character '≡'.
23:90: Char literals must contain exactly one character and close with '''.
25:106: Unexpected character '#'.
38:247: Unexpected character '≥'.
38:250: Unexpected character '≤'.
38:268: Unexpected character '≥'.
38:319: Unexpected character '≤'.
38:335: Unexpected character '≡'.
38:358: Unexpected character '≥'.
38:404: Unexpected character '≤'.
38:409: Unexpected character '≤'.
40:81: Unexpected character '≤'.
40:104: Unexpected character '≤'.
40:175: Unexpected character '≤'.
40:225: Unexpected character '→'.
40:240: Unexpected character '→'.
42:67: Unexpected character '≡'.
42:97: Unexpected character '≡'.
42:119: Unexpected character '—'.
44:27: Unexpected character '∃'.
44:30: Unexpected character '∈'.
44:49: Unexpected character '∃'.
44:52: Unexpected character '⊆'.
44:90: Unexpected character '≤'.
44:94: Unexpected character '≤'.
46:106: Char literals must contain exactly one character and close with '''.
48:26: Unexpected character 'Σ'.
48:65: Unexpected character '≤'.
50:137: Unexpected character '∈'.
52:17: Unexpected character 'Σ'.
52:72: Unexpected character 'Σ'.
59:9: Unexpected character '#'.
59:34: Unexpected character '#'.
59:132: Char literals must contain exactly one character and close with '''.
61:1: Unexpected character 'Σ'.
63:4: Unexpected character '’'.
63:183: Unexpected character '≥'.
63:212: Unexpected character '≥'.
65:9: Unexpected character 'Σ'.
65:19: Unexpected character '#'.
65:35: Unexpected character '#'.
66:1: Unexpected character '#'.
66:20: Unexpected character '#'.
66:32: Unexpected character '#'.
66:45: Unexpected character '#'.
66:64: Unexpected character '#'.
66:77: Unexpected character '#'.
67:6: Unexpected character 'Σ'.
68:13: Unexpected character 'Σ'.
68:45: Unexpected character '≤'.
68:82: Unexpected character 'Σ'.
71:7: Unexpected character '≥'.
71:15: Unexpected character '→'.
71:17: Unexpected character 'Σ'.
72:17: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 252
OUT2 812 |
| × |
LcbGameWithIntegersLiveCodeBench / codeforces 1899_A / easy: Game with Integers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GameWithIntegers case0: expected [First
Second
First
First
Second
First] got [Second
First
First
Second
First
Second] |
Failed |
IN9 257
OUT147 |
| × |
LcbGoodKidLiveCodeBench / codeforces 1873_B / easy: Good Kid (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GoodKid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:13: Unknown variable 'prod'. Declare it first, e.g. 'int prod = ...;'.
19:17: Unknown variable 'd'. Declare it first, e.g. 'int d = ...;'.
20:31: Unknown variable 'd'. Declare it first, e.g. 'int d = ...;'.
20:35: Unknown variable 'd'.
21:17: Unknown variable 'prod'. Declare it first, e.g. 'int prod = ...;'.
21:24: Unknown variable 'prod'.
21:31: Unknown variable 'd'.
23:17: Unknown variable 'prod'.
23:39: Unknown variable 'prod'.
Values:
run.Compiled = false |
Failed |
IN9 246
OUT223 |
| ✓ |
LcbOneDEraserLiveCodeBench / codeforces 1873_D / easy: 1D Eraser (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBench |
Passed |
IN9 242
OUT214 |
| ✓ |
LcbRaspberriesLiveCodeBench / codeforces 1883_C / medium: Raspberries (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Mediumsource:LiveCodeBench |
Passed |
IN9 250
OUT1 111 |
| × |
LcbShortSortLiveCodeBench / codeforces 1873_A / easy: Short Sort (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ShortSort case0: expected [YES
YES
YES
NO
NO
YES] got [NO
NO
NO
NO
NO
NO] |
Failed |
IN9 229
OUT156 |
| ✓ |
LcbTruckWeightDifferenceLiveCodeBench / codeforces 1899_B / hard: 250 Thousand Tons of TNT (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBench |
Passed |
IN20 356
OUT1 963 |
| × |
LcbYarikArrayAltParityLiveCodeBench / codeforces 1899_C / hard: Yarik and Array (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: YarikArrayAltParity case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
42:25: Type 'string' has no supported method named 'TrimEnd'.
Values:
run.Compiled = false |
Failed |
IN9 270
OUT345 |
| × |
LcbYarikMusicalNotesLiveCodeBench / codeforces 1899_D / hard: Yarik and Musical Notes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: YarikMusicalNotes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:48: Expected ')' to close the argument list, found '='.
Values:
run.Compiled = false |
Failed |
IN9 282
OUT257 |
Code Generation - LiveCodeBench v1 LeetCode (set 1)
4 passed5 failed |
44.4% |
IN82 636
OUT16 638 |
| ✓ |
LcbBuyChocoLiveCodeBench / leetcode / easy: buy-two-chocolates (LCB 2756)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 158
OUT157 |
| ✓ |
LcbColorTheArrayLiveCodeBench / leetcode / medium: number-of-adjacent-elements-with-the-same-color (LCB 2779)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 235
OUT292 |
| × |
LcbCountOfIntegersLiveCodeBench / leetcode / hard: count-of-integers (LCB 2757)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: count case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
10:37: Char literals must contain exactly one character and close with '''.
12:111: Char literals must contain exactly one character and close with '''.
14:160: Char literals must contain exactly one character and close with '''.
18:32: Unexpected character '→'.
20:66: Unexpected character '≈'.
20:80: Unexpected character '≈'.
22:209: Char literals must contain exactly one character and close with '''.
24:46: Char literals must contain exactly one character and close with '''.
30:189: Unexpected character '—'.
45:93: Char literals must contain exactly one character and close with '''.
59:211: Unexpected character '→'.
67:87: Char literals must contain exactly one character and close with '''.
93:116: Unexpected character '→'.
93:196: Unexpected character '→'.
93:205: Unexpected character '✓'.
95:35: Unexpected character '→'.
95:38: Unexpected character '✓'.
99:104: Unexpected character '≥'.
99:114: Unexpected character '≥'.
99:120: Unexpected character '≥'.
99:130: Unexpected character '≤'.
99:214: Unexpected character '→'.
129:2: Char literals must contain exactly one character and close with '''.
145:39: Char literals must contain exactly one character and close with '''.
147:47: Unexpected character '→'.
157:6: Unexpected character '`'.
157:33: Unexpected character '`'.
157:185: Char literals must contain exactly one character and close with '''.
159:54: Char literals must contain exactly one character and close with '''.
161:4: Char literals must contain exactly one character and close with '''.
163:48: Unexpected character '`'.
163:73: Unexpected character '`'.
163:86: Char literals must contain exactly one character and close with '''.
194:27: Unexpected character '`'.
194:38: Unexpected character '`'.
194:90: Unexpected character '`'.
194:110: Unexpected character '`'.
194:140: Unexpected character '→'.
194:178: Unexpected character '`'.
194:220: Unexpected character '`'.
194:275: Unexpected character '#'.
194:280: Unexpected character '#'.
194:292: Char literals must contain exactly one character and close with '''.
194:318: Char literals must contain exactly one character and close with '''.
196:2: Char literals must contain exactly one character and close with '''.
198:37: Unexpected character '`'.
198:56: Unexpected character '`'.
208:2: Char literals must contain exactly one character and close with '''.
218:64: Unexpected character '—'.
220:57: Char literals must contain exactly one character and close with '''.
222:77: Unexpected character '→'.
232:13: Unexpected character '`'.
232:35: Unexpected character '`'.
238:2: Char literals must contain exactly one character and close with '''.
362:35: Unexpected character '→'.
362:66: Unexpected character '→'.
362:116: Unexpected character '→'.
376:58: Unexpected character '→'.
376:114: Unexpected character '✓'.
378:32: Unexpected character '→'.
378:56: Unexpected character '→'.
378:59: Unexpected character '✓'.
384:6: Unexpected character '`'.
384:27: Unexpected character '`'.
384:102: Unexpected character '#'.
384:106: Char literals must contain exactly one character and close with '''.
386:20: Unexpected character '`'.
386:58: Unexpected character '`'.
386:61: Char literals must contain exactly one character and close with '''.
394:75: Unexpected character '`'.
394:118: Unexpected character '`'.
411:6: Unexpected character '`'.
411:21: Unexpected character '`'.
415:65: Unexpected character '→'.
425:33: Unexpected character '`'.
425:39: Unexpected character '`'.
427:35: Unexpected character '—'.
429:28: Char literals must contain exactly one character and close with '''.
437:2: Char literals must contain exactly one character and close with '''.
443:11: Unexpected character '`'.
443:33: Unexpected character '`'.
449:2: Char literals must contain exactly one character and close with '''.
451:22: Unexpected character '`'.
451:28: Un ... (message truncated) |
Failed |
IN9 202
OUT14 732 |
| ✓ |
LcbCountSeniorsLiveCodeBench / leetcode / easy: number-of-senior-citizens (LCB 2727)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 238
OUT81 |
| × |
LcbDistinctDifferenceArrayLiveCodeBench / leetcode / easy: find-the-distinct-difference-array (LCB 2777)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: distinctDifference case0: expected [-3, -1, 1, 3, 5] got [1,2,3,4,5] |
Failed |
IN9 179
OUT248 |
| × |
LcbMatrixSumLiveCodeBench / leetcode / medium: sum-in-a-matrix (LCB 2728)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: matrixSum case0: expected 15 got 21 |
Failed |
IN9 153
OUT140 |
| ✓ |
LcbMaximumOrLiveCodeBench / leetcode / medium: maximum-or (LCB 2730)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 165
OUT185 |
| × |
LcbMaxStrengthLiveCodeBench / leetcode / medium: maximum-strength-of-a-group (LCB 2754)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxStrength case0: expected 1350 got -1350 |
Failed |
IN9 144
OUT246 |
| × |
LcbMinExtraCharLiveCodeBench / leetcode / medium: extra-characters-in-a-string (LCB 2755)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minExtraChar case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
35:5: Unexpected character '—'.
Values:
run.Compiled = false |
Failed |
IN9 162
OUT557 |
Code Generation - LiveCodeBench v1 LeetCode (set 10)
4 passed5 failed |
44.4% |
IN83 020
OUT3 930 |
| × |
LcbCanBeEqualSwapLiveCodeBench / leetcode / easy: check-if-strings-can-be-made-equal-with-operations-i (id 2999)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: canBeEqual case0: expected true got false |
Failed |
IN9 184
OUT102 |
| ✓ |
LcbCanMakeSubsequenceCyclicLiveCodeBench / leetcode / medium: make-string-a-subsequence-using-cyclic-increments (id 3018)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 249
OUT139 |
| × |
LcbConstructProductMatrixLiveCodeBench / leetcode / medium: construct-product-matrix (id 3031)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<List<int>>>(run.ResultJson) == FromJson<List<List<int>>>(expected[i])
Context: constructProduct case0: expected [[24, 12], [8, 6]] got [[6,16],[36,96]] |
Failed |
IN9 198
OUT343 |
| × |
LcbFurthestDistanceFromOriginLiveCodeBench / leetcode / easy: furthest-point-from-origin (id 3019)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: furthestOrigin case0: expected 3 got 11 |
Failed |
IN9 196
OUT57 |
| ✓ |
LcbMaxBallPassingValueLiveCodeBench / leetcode / hard: maximize-value-of-function-in-a-ball-passing-game (id 3032)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 310
OUT123 |
| ✓ |
LcbMinAbsoluteDifferenceApartLiveCodeBench / leetcode / medium: minimum-absolute-difference-between-elements-with-constraint (id 3000)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 206
OUT129 |
| × |
LcbMinOperationsSubsequenceSumLiveCodeBench / leetcode / hard: minimum-operations-to-form-subsequence-with-target-sum (id 3025)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minOpsSubseqSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
57:14: Variable 'bit' is already declared in this scope or an enclosing one.
110:14: Variable 'bit' is already declared in this scope or an enclosing one.
Values:
run.Compiled = false |
Failed |
IN9 263
OUT2 041 |
| ✓ |
LcbNumberOfBeautifulIntegersLiveCodeBench / leetcode / hard: number-of-beautiful-integers-in-the-range (id 3017)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 184
OUT164 |
| × |
LcbStringTransformationWaysLiveCodeBench / leetcode / hard: string-transformation (id 3024)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: stringTransform case0: expected 2 got 1000000006 |
Failed |
IN9 230
OUT832 |
Code Generation - LiveCodeBench v1 LeetCode (set 11)
5 passed4 failed |
55.6% |
IN82 747
OUT3 089 |
| × |
LcbMaximumCompleteSubsetSumLiveCodeBench / leetcode / hard: maximum-element-sum-of-a-complete-subset-of-indices (id 3047)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxCompleteSubset case0: expected 16 got 45 |
Failed |
IN9 236
OUT376 |
| ✓ |
LcbMaximumOddBinaryNumberLiveCodeBench / leetcode / easy: maximum-odd-binary-number (id 3055)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 152
OUT138 |
| ✓ |
LcbMinArrayLengthAfterRemovalsLiveCodeBench / leetcode / medium: minimum-array-length-after-pair-removals (id 3081)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 210
OUT213 |
| ✓ |
LcbMinCostMakeStringsEqualLiveCodeBench / leetcode / medium: apply-operations-to-make-two-strings-equal (id 3033)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 283
OUT364 |
| × |
LcbMinimumRightShiftsLiveCodeBench / leetcode / easy: minimum-right-shifts-to-sort-the-array (id 3045)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minRightShifts case0: expected 2 got -1 |
Failed |
IN9 161
OUT254 |
| × |
LcbMinOperationsCollectElementsLiveCodeBench / leetcode / easy: minimum-operations-to-collect-elements (id 3044)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsCollect case0: expected 4 got 2 |
Failed |
IN9 172
OUT79 |
| × |
LcbMinOperationsSpecialNumberLiveCodeBench / leetcode / medium: minimum-operations-to-make-a-special-number (id 3046)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsSpecial case1: expected 3 got -1 |
Failed |
IN9 160
OUT340 |
| ✓ |
LcbNumberOfPointsCarsLiveCodeBench / leetcode / easy: points-that-intersect-with-cars (id 3034)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 171
OUT89 |
| ✓ |
LcbSplitArrayMaxSubarraysLiveCodeBench / leetcode / medium: split-array-into-maximum-number-of-subarrays (id 3080)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 202
OUT1 236 |
Code Generation - LiveCodeBench v1 LeetCode (set 12)
2 passed6 failed |
25.0% |
IN73 740
OUT2 545 |
| × |
LcbBeautifulTowersMaxSumLiveCodeBench / leetcode / medium: beautiful-towers-i (id 3114)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: beautifulTowers case0: expected 13 got 5 |
Failed |
IN9 266
OUT452 |
| × |
LcbCountSubMultisetsBoundedSumLiveCodeBench / leetcode / hard: count-of-sub-multisets-with-bounded-sum (id 3091)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countSubMultisets case0: expected 1 got 2 |
Failed |
IN9 293
OUT547 |
| ✓ |
LcbCountWaysHappyStudentsLiveCodeBench / leetcode / medium: happy-students (id 3104)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 202
OUT175 |
| × |
LcbLongestSubsequenceSumTargetLiveCodeBench / leetcode / medium: length-of-the-longest-subsequence-that-sums-to-target (id 3106)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestSubseqSum case0: expected 3 got 4 |
Failed |
IN9 202
OUT221 |
| × |
LcbMinOperationsMakeArrayEmptyLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-array-empty (id 3094)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsEmpty case1: expected -1 got 3 |
Failed |
IN9 177
OUT239 |
| × |
LcbMinSizeSubarrayInfiniteLiveCodeBench / leetcode / medium: minimum-size-subarray-in-infinite-array (id 3141)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minSizeSubarray case0: expected 2 got -1 |
Failed |
IN9 183
OUT410 |
| × |
LcbShortestBeautifulSubstringLiveCodeBench / leetcode / medium: shortest-and-lexicographically-smallest-beautiful-string (id 3150)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: shortestBeautiful case0: expected "11001" got "100011" |
Failed |
IN9 227
OUT356 |
| ✓ |
LcbSumIndicesKSetBitsLiveCodeBench / leetcode / easy: sum-of-values-at-indices-with-k-set-bits (id 3093)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 190
OUT145 |
Code Generation - LiveCodeBench v1 LeetCode (set 13)
5 passed4 failed |
55.6% |
IN82 352
OUT2 954 |
| ✓ |
LcbDivisibleSumsDifferenceLiveCodeBench / leetcode / easy: 3172 divisible-and-non-divisible-sums-difference (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 118
OUT93 |
| ✓ |
LcbLastVisitedIntegersLiveCodeBench / leetcode / easy: 3164 last-visited-integers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 202
OUT153 |
| × |
LcbMaxOrderedTripletValueLiveCodeBench / leetcode / easy: 3154 maximum-value-of-an-ordered-triplet-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxOrderedTripletValue case0: expected 77 got 42 |
Failed |
IN9 131
OUT194 |
| × |
LcbMaxSumOfSquaresLiveCodeBench / leetcode / hard: 3153 apply-operations-on-array-to-maximize-sum-of-squares (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxSumOfSquares case0: expected 261 got 13 |
Failed |
IN9 188
OUT293 |
| ✓ |
LcbMinChangesBinaryBeautifulLiveCodeBench / leetcode / medium: 3174 minimum-number-of-changes-to-make-binary-string-beautiful (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 124
OUT75 |
| × |
LcbMinEqualSumReplaceZerosLiveCodeBench / leetcode / medium: 3171 minimum-equal-sum-of-two-arrays-after-replacing-zeros (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MinEqualSumReplaceZeros case1: expected -1 got 5 |
Failed |
IN9 121
OUT232 |
| × |
LcbMinGroupsValidAssignmentLiveCodeBench / leetcode / medium: 3166 minimum-number-of-groups-to-create-a-valid-assignment (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinGroupsValidAssignment case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:47: Unexpected character '₁'.
3:51: Unexpected character 'ₘ'.
3:96: Unexpected character '⌊'.
3:100: Unexpected character '⌋'.
3:105: Unexpected character '⌈'.
3:109: Unexpected character '⌉'.
3:145: Unexpected character 'ᵢ'.
3:172: Unexpected character 'ᵢ'.
3:196: Unexpected character '⌊'.
3:200: Unexpected character '⌋'.
3:209: Unexpected character '⌈'.
3:213: Unexpected character '⌉'.
3:221: Unexpected character 'Σ'.
3:223: Unexpected character 'ᵢ'.
5:14: Unexpected character 'ᵢ'.
5:27: Unexpected character 'ᵢ'.
5:38: Unexpected character 'ᵢ'.
5:40: Unexpected character '∈'.
5:44: Unexpected character '·'.
5:49: Unexpected character '·'.
5:57: Unexpected character '→'.
5:61: Unexpected character '∈'.
5:106: Unexpected character 'ᵢ'.
5:187: Unexpected character '≤'.
5:191: Unexpected character '≤'.
5:524: Unexpected character 'ᵢ'.
7:108: Unexpected character '≥'.
7:118: Unexpected character '≤'.
7:137: Unexpected character 'Σ'.
7:149: Unexpected character 'Σ'.
7:170: Unexpected character '≤'.
7:172: Unexpected character '≤'.
9:427: Unexpected character 'Σ'.
9:429: Unexpected character 'ᵢ'.
9:481: Unexpected character '#'.
11:16: Unexpected character '≥'.
11:72: Unexpected character '—'.
11:90: Unexpected character '≥'.
11:105: Unexpected character '≥'.
11:344: Char literals must contain exactly one character and close with '''.
13:43: Unexpected character '≥'.
19:4: Char literals must contain exactly one character and close with '''.
21:151: Char literals must contain exactly one character and close with '''.
37:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 153
OUT1 598 |
| ✓ |
LcbMinProcessingTimeLiveCodeBench / leetcode / medium: 3151 minimum-processing-time (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 193
OUT204 |
| ✓ |
LcbSumDistinctSquaresLiveCodeBench / leetcode / easy: 3163 subarrays-distinct-element-sum-of-squares-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 122
OUT112 |
Code Generation - LiveCodeBench v1 LeetCode (set 14)
8 passed1 failed |
88.9% |
IN82 225
OUT3 874 |
| ✓ |
LcbFindChampionILiveCodeBench / leetcode / easy: 3188 find-champion-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 150
OUT78 |
| ✓ |
LcbFindKOrLiveCodeBench / leetcode / easy: 3183 find-the-k-or-of-an-array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 141
OUT137 |
| ✓ |
LcbFindWordsContainingLiveCodeBench / leetcode / easy: 3194 find-words-containing-character (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 098
OUT78 |
| ✓ |
LcbMaxBalancedSubseqSumLiveCodeBench / leetcode / hard: 3184 maximum-balanced-subsequence-sum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 159
OUT214 |
| × |
LcbMaximumXorProductLiveCodeBench / leetcode / medium: 3192 maximum-xor-product (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaximumXorProduct case0: expected 98 got 60 |
Failed |
IN9 121
OUT415 |
| ✓ |
LcbMaxStrongPairXorLiveCodeBench / leetcode / easy: 3193 maximum-strong-pair-xor-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT157 |
| ✓ |
LcbMinMountainTripletSumLiveCodeBench / leetcode / easy: 3176 minimum-sum-of-mountain-triplets-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 118
OUT287 |
| ✓ |
LcbMinOpsMaximizeLastLiveCodeBench / leetcode / medium: 3190 minimum-operations-to-maximize-last-elements-in-arrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 164
OUT2 413 |
| ✓ |
LcbSeparateBlackWhiteBallsLiveCodeBench / leetcode / medium: 3195 separate-black-and-white-balls (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 125
OUT95 |
Code Generation - LiveCodeBench v1 LeetCode (set 15)
4 passed4 failed |
50.0% |
IN73 170
OUT13 264 |
| × |
LcbCountBeautifulSubstringsLiveCodeBench / leetcode / medium: 3210 count-beautiful-substrings-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountBeautifulSubstrings case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'beautifulSubstrings' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, BeautifulSubstrings.
Values:
run.Compiled = false |
Failed |
IN9 145
OUT211 |
| ✓ |
LcbDistributeCandiesILiveCodeBench / leetcode / easy: 3199 distribute-candies-among-children-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 086
OUT135 |
| ✓ |
LcbFindCommonElementsLiveCodeBench / leetcode / easy: 3206 find-common-elements-between-two-arrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 161
OUT137 |
| × |
LcbGoodStringsLeetLiveCodeBench / leetcode / medium: 3200 number-of-strings-which-can-be-rearranged-to-contain-substring (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GoodStringsLeet case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:36: Unexpected character '`'.
3:38: Unexpected character '`'.
8:19: Unexpected character '≥'.
8:24: Unexpected character '≥'.
8:29: Unexpected character '≥'.
8:37: Char literals must contain exactly one character and close with '''.
14:14: Unexpected character '—'.
14:24: Char literals must contain exactly one character and close with '''.
16:26: Unexpected character '−'.
20:47: Unexpected character '≤'.
20:92: Unexpected character '≥'.
20:100: Unexpected character '≥'.
20:108: Unexpected character '≥'.
22:12: Unexpected character '≤'.
25:25: Unexpected character '∪'.
25:29: Unexpected character '∪'.
27:11: Unexpected character '∩'.
27:13: Unexpected character '∩'.
27:25: Char literals must contain exactly one character and close with '''.
30:1: Unexpected character '−'.
30:7: Unexpected character '−'.
30:13: Unexpected character '−'.
31:5: Unexpected character '∩'.
31:13: Unexpected character '∩'.
31:21: Unexpected character '∩'.
32:1: Unexpected character '−'.
32:5: Unexpected character '∩'.
32:7: Unexpected character '∩'.
37:16: Unexpected character '∈'.
38:5: Unexpected character '∩'.
39:5: Unexpected character '∩'.
39:17: Unexpected character '∈'.
40:5: Unexpected character '∩'.
40:17: Unexpected character '∈'.
41:5: Unexpected character '∩'.
41:7: Unexpected character '∩'.
41:25: Unexpected character '∈'.
43:17: Unexpected character '−'.
43:20: Unexpected character '·'.
43:26: Unexpected character '−'.
43:37: Unexpected character '·'.
43:66: Unexpected character '·'.
43:88: Unexpected character '·'.
43:99: Unexpected character '−'.
43:110: Unexpected character '·'.
46:8: Unexpected character '−'.
46:11: Unexpected character '·'.
46:17: Unexpected character '−'.
46:20: Unexpected character '·'.
46:33: Unexpected character '·'.
46:42: Unexpected character '·'.
46:44: Unexpected character '·'.
46:54: Unexpected character '−'.
46:61: Unexpected character '−'.
46:64: Unexpected character '·'.
48:95: Unexpected character '—'.
48:154: Unexpected character '·'.
48:164: Unexpected character '—'.
50:19: Unexpected character '→'.
50:49: Char literals must contain exactly one character and close with '''.
52:75: Unexpected character '≥'.
62:11: Unexpected character '·'.
Values:
run.Compiled = false |
Failed |
IN9 117
OUT2 095 |
| ✓ |
LcbMakeThreeStringsEqualLiveCodeBench / leetcode / easy: 3207 make-three-strings-equal (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 148
OUT143 |
| × |
LcbMaxFrequencyScoreLiveCodeBench / leetcode / hard: 3196 apply-operations-to-maximize-frequency-score (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxFrequencyScore case0: expected 3 got 2 |
Failed |
IN9 129
OUT186 |
| × |
LcbMinCoinsForFruitsLiveCodeBench / leetcode / medium: 3209 minimum-number-of-coins-for-fruits (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinCoinsForFruits case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:38: Unexpected character '`'.
3:45: Unexpected character '`'.
3:61: Unexpected character '`'.
3:63: Unexpected character '`'.
3:80: Unexpected character '`'.
3:82: Unexpected character '`'.
5:42: Unexpected character '`'.
5:48: Unexpected character '`'.
5:93: Unexpected character '`'.
5:95: Unexpected character '`'.
7:13: Unexpected character '`'.
7:15: Unexpected character '`'.
7:36: Unexpected character '`'.
7:46: Unexpected character '`'.
7:61: Unexpected character '`'.
7:72: Unexpected character '`'.
7:80: Unexpected character '`'.
7:82: Unexpected character '`'.
7:114: Unexpected character '`'.
7:118: Unexpected character '`'.
7:131: Unexpected character '`'.
7:135: Unexpected character '`'.
9:38: Unexpected character '`'.
9:40: Unexpected character '`'.
9:48: Unexpected character '`'.
9:58: Unexpected character '`'.
9:74: Unexpected character '`'.
9:76: Unexpected character '`'.
9:91: Unexpected character '`'.
9:100: Unexpected character '`'.
9:124: Unexpected character '`'.
9:126: Unexpected character '`'.
9:147: Unexpected character '`'.
9:151: Unexpected character '`'.
11:5: Unexpected character '`'.
11:10: Unexpected character '`'.
11:39: Unexpected character '`'.
11:44: Unexpected character '`'.
11:210: Unexpected character '—'.
11:273: Unexpected character '`'.
11:275: Unexpected character '`'.
11:296: Unexpected character '`'.
11:299: Unexpected character '`'.
13:20: Unexpected character '`'.
13:26: Unexpected character '`'.
13:62: Unexpected character '`'.
13:64: Unexpected character '`'.
13:88: Unexpected character '`'.
13:90: Unexpected character '`'.
13:100: Unexpected character '`'.
13:110: Unexpected character '`'.
13:137: Unexpected character '`'.
13:151: Unexpected character '`'.
13:229: Unexpected character '`'.
13:231: Unexpected character '`'.
13:239: Unexpected character '`'.
13:247: Unexpected character '`'.
13:254: Char literals must contain exactly one character and close with '''.
13:337: Unexpected character '`'.
13:375: Unexpected character '`'.
13:380: Char literals must contain exactly one character and close with '''.
15:107: Unexpected character '`'.
15:113: Unexpected character '`'.
15:144: Unexpected character '`'.
15:149: Unexpected character '`'.
15:162: Char literals must contain exactly one character and close with '''.
17:40: Unexpected character '`'.
17:46: Unexpected character '`'.
17:93: Unexpected character '`'.
17:95: Unexpected character '`'.
17:100: Unexpected character '`'.
17:102: Unexpected character '`'.
17:122: Unexpected character '`'.
17:124: Unexpected character '`'.
17:153: Char literals must contain exactly one character and close with '''.
20:1: Unexpected character '`'.
20:37: Unexpected character '`'.
22:12: Unexpected character '`'.
22:18: Unexpected character '`'.
22:59: Unexpected character '`'.
22:61: Unexpected character '`'.
22:66: Unexpected character '`'.
22:68: Unexpected character '`'.
22:96: Unexpected character '`'.
22:98: Unexpected character '`'.
22:127: Unexpected character '`'.
22:130: Unexpected character '`'.
22:156: Unexpected character '`'.
22:158: Unexpected character '`'.
22:164: Unexpected character '`'.
22:168: Unexpected character '≤'.
22:171: Unexpected character '`'.
22:189: Unexpected character '`'.
22:220: Unexpected character '`'.
22:226: Unexpected character '`'.
22:228: Unexpected character '`'.
22:233: Unexpected character '`'.
22:252: Unexpected character '`'.
22:259: Char literals must contain exactly one character and close with '''.
24:23: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 150
OUT5 493 |
| ✓ |
LcbPalindromeRearrangeQueriesLiveCodeBench / leetcode / hard: 3203 palindrome-rearrangement-queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 234
OUT4 864 |
Code Generation - LiveCodeBench v1 LeetCode (set 16)
4 passed5 failed |
44.4% |
IN82 354
OUT5 763 |
| × |
LcbCompleteSubstringsLiveCodeBench / leetcode / hard: count-complete-substrings (id 3223)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: completeSub case0 runtime error: String index -1 is out of range. The string has 6 character(s).
Values:
run.Success = false |
Failed |
IN9 145
OUT353 |
| ✓ |
LcbCountTestedDevicesLiveCodeBench / leetcode / easy: count-tested-devices-after-test-operations (id 3220)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT106 |
| ✓ |
LcbFindPeaksLiveCodeBench / leetcode / easy: find-the-peaks (id 3221)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 117
OUT101 |
| × |
LcbGoodPartitionsCountLiveCodeBench / leetcode / hard: count-the-number-of-good-partitions (id 3212)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: goodPart case1: expected 1 got 8 |
Failed |
IN9 118
OUT295 |
| × |
LcbInfectionSequencesLiveCodeBench / leetcode / hard: count-the-number-of-infection-sequences (id 3224)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: infectionSeq case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:200: Unexpected character '—'.
11:142: Char literals must contain exactly one character and close with '''.
13:563: Char literals must contain exactly one character and close with '''.
15:29: Char literals must contain exactly one character and close with '''.
21:2: Char literals must contain exactly one character and close with '''.
48:21: Char literals must contain exactly one character and close with '''.
107:156: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 217
OUT3 803 |
| ✓ |
LcbLexSmallestSwapLiveCodeBench / leetcode / medium: make-lexicographically-smallest-array-by-swapping-elements (id 3219)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 174
OUT467 |
| ✓ |
LcbLongestSubarrayKFreqLiveCodeBench / leetcode / medium: length-of-longest-subarray-with-at-most-k-frequency (id 3225)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 133
OUT214 |
| × |
LcbMaxNonDecreasingLengthLiveCodeBench / leetcode / hard: find-maximum-non-decreasing-array-length (id 3211)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxNonDec case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 157
OUT212 |
| × |
LcbSubarraysMaxKTimesLiveCodeBench / leetcode / medium: count-subarrays-where-max-element-appears-at-least-k-times (id 3213)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: subMaxK case0: expected 6 got 11 |
Failed |
IN9 101
OUT212 |
Code Generation - LiveCodeBench v1 LeetCode (set 17)
5 passed4 failed |
55.6% |
IN82 456
OUT4 505 |
| × |
LcbDoubleModExpLiveCodeBench / leetcode / medium: double-modular-exponentiation (id 3234)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: doubleModExp case1 runtime error: Pow result overflowed the target numeric type.
Values:
run.Success = false |
Failed |
IN9 150
OUT209 |
| × |
LcbEqualindromicCostLiveCodeBench / leetcode / medium: minimum-cost-to-make-array-equalindromic (id 3229)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: equalindromic case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
18:28: Expected ')' to close the argument list, found 'res'.
Values:
run.Compiled = false |
Failed |
IN9 202
OUT792 |
| × |
LcbMaxPartitionsOpsLiveCodeBench / leetcode / hard: maximize-the-number-of-partitions-after-operations (id 3233)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxPartOps case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:6: Expected a name, found '='.
Values:
run.Compiled = false |
Failed |
IN9 166
OUT1 356 |
| × |
LcbMaxSetSizeRemovalsLiveCodeBench / leetcode / medium: maximum-size-of-a-set-after-removals (id 3228)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSetSize case1: expected 5 got 6 |
Failed |
IN9 131
OUT207 |
| ✓ |
LcbMinCoinsToAddLiveCodeBench / leetcode / medium: minimum-number-of-coins-to-be-added (id 3231)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 163
OUT117 |
| ✓ |
LcbMinCostConvertStringLiveCodeBench / leetcode / medium: minimum-cost-to-convert-string-i (id 3235)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 234
OUT602 |
| ✓ |
LcbMinimumNumberGameLiveCodeBench / leetcode / easy: minimum-number-game (id 3226)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 139
OUT100 |
| ✓ |
LcbMissingAndRepeatedLiveCodeBench / leetcode / easy: find-missing-and-repeated-values (id 3227)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 153
OUT992 |
| ✓ |
LcbRemoveAlmostEqualLiveCodeBench / leetcode / medium: remove-adjacent-almost-equal-characters (id 3230)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 118
OUT130 |
Code Generation - LiveCodeBench v1 LeetCode (set 18)
5 passed3 failed |
62.5% |
IN73 246
OUT1 976 |
| ✓ |
LcbBeautifulIndicesLiveCodeBench / leetcode / medium: find-beautiful-indices-in-the-given-array-i (id 3245)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 174
OUT267 |
| ✓ |
LcbBitwiseOrTrailingZerosLiveCodeBench / leetcode / easy: check-if-bitwise-or-has-trailing-zeros (id 3246)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 159
OUT87 |
| ✓ |
LcbMaxFrequencyElementsLiveCodeBench / leetcode / easy: count-elements-with-maximum-frequency (id 3242)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 093
OUT158 |
| × |
LcbMaxNumberPricesSumLiveCodeBench / leetcode / medium: maximum-number-that-sum-of-the-prices-is-less-than-or-equal-to-k (id 3240)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxNumberPrices case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 193
OUT260 |
| × |
LcbMinArrayLengthModLiveCodeBench / leetcode / medium: minimize-length-of-array-using-operations (id 3244)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minArrayLen case0: expected 1 got 2 |
Failed |
IN9 156
OUT292 |
| ✓ |
LcbMinOpsXYEqualLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-x-and-y-equal (id 3239)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 133
OUT294 |
| ✓ |
LcbMissingSeqPrefixSumLiveCodeBench / leetcode / easy: smallest-missing-integer-greater-than-sequential-prefix-sum (id 3236)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 132
OUT161 |
| × |
LcbPowerfulIntegersLiveCodeBench / leetcode / hard: count-the-number-of-powerful-integers (id 3243)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: powerfulInt case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
27:5: Expected an expression, found 'else'.
Values:
run.Compiled = false |
Failed |
IN9 206
OUT457 |
Code Generation - LiveCodeBench v1 LeetCode (set 19)
4 passed5 failed |
44.4% |
IN82 287
OUT1 795 |
| ✓ |
LcbAreaOfMaxDiagonalLiveCodeBench / leetcode / easy: maximum-area-of-longest-diagonal-rectangle (id 3251)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 130
OUT151 |
| ✓ |
LcbCountMatchingSubarraysLiveCodeBench / leetcode / medium: number-of-subarrays-that-match-a-pattern-i (id 3269)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 203
OUT245 |
| × |
LcbIncremovableSubarrayCountLiveCodeBench / leetcode / easy: count-the-number-of-incremovable-subarrays-i (id 3252)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: incremovable case0: expected 10 got 8 |
Failed |
IN9 123
OUT336 |
| × |
LcbLargestPerimeterLiveCodeBench / leetcode / medium: find-polygon-with-the-largest-perimeter (id 3262)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: largestPerimeter case1: expected 12 got -1 |
Failed |
IN9 189
OUT127 |
| × |
LcbLongestSpecialThriceLiveCodeBench / leetcode / medium: find-longest-special-substring-that-occurs-thrice-i (id 3267)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: longestSpecial case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:45: Operator '+' cannot be applied to 'string' and 'int'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 117
OUT307 |
| × |
LcbMaximumGoodSubarraySumLiveCodeBench / leetcode / medium: maximum-good-subarray-sum (id 3265)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxGoodSub case2: expected -6 got 0 |
Failed |
IN9 134
OUT151 |
| ✓ |
LcbMinimumCostThreeSubarraysLiveCodeBench / leetcode / easy: divide-an-array-into-subarrays-with-minimum-cost-i (id 3263)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 098
OUT150 |
| ✓ |
LcbMinOpsXorEqualKLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-array-xor-equal-to-k (id 3249)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 141
OUT135 |
| × |
LcbMinOrAfterOperationsLiveCodeBench / leetcode / hard: minimize-or-of-remaining-elements-using-operations (id 3261)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOr case0: expected 3 got 1073741823 |
Failed |
IN9 152
OUT193 |
Code Generation - LiveCodeBench v1 LeetCode (set 2)
4 passed5 failed |
44.4% |
IN82 512
OUT5 707 |
| ✓ |
LcbCircularGameLosersLiveCodeBench / leetcode / easy: find-the-losers-of-the-circular-game (LCB 2791)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 186
OUT1 469 |
| × |
LcbCollectingChocolatesLiveCodeBench / leetcode / medium: collecting-chocolates (LCB 2810)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minCost case0: expected 13 got 36 |
Failed |
IN9 184
OUT127 |
| ✓ |
LcbLongestSemiRepetitiveLiveCodeBench / leetcode / medium: find-the-longest-semi-repetitive-substring (LCB 2786)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 161
OUT176 |
| ✓ |
LcbMinLengthRemoveSubstrLiveCodeBench / leetcode / easy: minimum-string-length-after-removing-substrings (LCB 2800)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 129
OUT149 |
| × |
LcbNeighboringXorLiveCodeBench / leetcode / medium: neighboring-bitwise-xor (LCB 2792)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: doesValidArrayExist case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'doesValidArrayExist' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, DoesValidArrayExist.
Values:
run.Compiled = false |
Failed |
IN9 189
OUT52 |
| ✓ |
LcbPaintWallsLiveCodeBench / leetcode / hard: painting-the-walls (LCB 2808)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 195
OUT2 998 |
| × |
LcbPowerOfHeroesLiveCodeBench / leetcode / hard: power-of-heroes (LCB 2784)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: sumOfPower case0 runtime error: List index 4 is out of range. The list has 4 element(s).
Values:
run.Success = false |
Failed |
IN9 156
OUT277 |
| × |
LcbPunishmentNumberLiveCodeBench / leetcode / medium: find-the-punishment-number-of-an-integer (LCB 2802)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: punishmentNumber case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:5: Function 'CanPartition' must return 'int', but this return statement has type 'bool'.
36:13: The if condition must be 'bool', but it has type 'int'.
Values:
run.Compiled = false |
Failed |
IN9 149
OUT268 |
| × |
LcbSemiOrderedPermutationLiveCodeBench / leetcode / easy: semi-ordered-permutation (LCB 2785)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: semiOrdered case0: expected 2 got 1 |
Failed |
IN9 163
OUT191 |
Code Generation - LiveCodeBench v1 LeetCode (set 20)
3 passed5 failed |
37.5% |
IN73 165
OUT1 998 |
| ✓ |
LcbCanSortArrayBySetBitsLiveCodeBench / leetcode / medium: find-if-array-can-be-sorted (id 3291)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 117
OUT275 |
| ✓ |
LcbCountPrefixSuffixPairsLiveCodeBench / leetcode / easy: count-prefix-and-suffix-pairs-i (id 3309)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 133
OUT135 |
| × |
LcbEarliestSecondToMarkLiveCodeBench / leetcode / medium: earliest-second-to-mark-indices-i (id 3292)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: earliestSecond case0: expected 8 got -1 |
Failed |
IN9 194
OUT421 |
| ✓ |
LcbFlowerGameLiveCodeBench / leetcode / medium: alice-and-bob-playing-flower-game (id 3279)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 149
OUT111 |
| × |
LcbLastNonEmptyStringLiveCodeBench / leetcode / medium: apply-operations-to-make-string-empty (id 3308)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: lastNonEmpty case0: expected "ba" got "" |
Failed |
IN9 101
OUT272 |
| × |
LcbMaxSelectedConsecutiveLiveCodeBench / leetcode / hard: maximize-consecutive-elements-in-an-array-after-modification (id 3298)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSelected case0: expected 3 got 2 |
Failed |
IN9 148
OUT324 |
| × |
LcbMaxSubsetPowerPatternLiveCodeBench / leetcode / medium: find-the-maximum-number-of-elements-in-subset (id 3299)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSubsetPower case0: expected 3 got 1 |
Failed |
IN9 195
OUT309 |
| × |
LcbMinTimeRevertWordLiveCodeBench / leetcode / medium: minimum-time-to-revert-word-to-initial-state-i (id 3297)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minTimeRevert case0: expected 2 got 7 |
Failed |
IN9 128
OUT151 |
Code Generation - LiveCodeBench v1 LeetCode (set 21)
7 passed1 failed |
87.5% |
IN73 182
OUT6 812 |
| ✓ |
LcbAntOnBoundaryLiveCodeBench / leetcode / easy: ant-on-the-boundary (id 3311)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 156
OUT70 |
| ✓ |
LcbCountKeyChangesLiveCodeBench / leetcode / easy: number-of-changing-keys (id 3312)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 126
OUT110 |
| ✓ |
LcbDistributeTwoArraysLiveCodeBench / leetcode / easy: distribute-elements-into-two-arrays-i (id 3347)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 180
OUT963 |
| ✓ |
LcbLongestCommonPrefixDigitsLiveCodeBench / leetcode / medium: find-the-length-of-the-longest-common-prefix (id 3329)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 192
OUT151 |
| ✓ |
LcbMaxOperationsSameScoreLiveCodeBench / leetcode / easy: maximum-number-of-operations-with-the-same-score-i (id 3320)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 126
OUT120 |
| × |
LcbMaxPalindromesAfterOpsLiveCodeBench / leetcode / medium: maximum-palindromes-after-operations (id 3317)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxPalindromes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:26: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '('.
Values:
run.Compiled = false |
Failed |
IN9 169
OUT5 221 |
| ✓ |
LcbMinOpsExceedThresholdLiveCodeBench / leetcode / easy: minimum-operations-to-exceed-threshold-value-i (id 3331)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 109
OUT56 |
| ✓ |
LcbSplitArrayDistinctLiveCodeBench / leetcode / easy: split-the-array (id 3324)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 124
OUT121 |
Code Generation - LiveCodeBench v1 LeetCode (set 3)
4 passed4 failed |
50.0% |
IN73 151
OUT1 363 |
| × |
LcbGcdTraversalLiveCodeBench / leetcode / hard: greatest-common-divisor-traversal (LCB 2827)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: canTraverseAllPairs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:15: Expected ';' after the variable declaration, found '['.
Values:
run.Compiled = false |
Failed |
IN9 171
OUT597 |
| × |
LcbIsFascinatingLiveCodeBench / leetcode / easy: check-if-the-number-is-fascinating (LCB 2824)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: isFascinating case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:13: Operator '!' requires 'bool', but the operand has type 'void'.
Values:
run.Compiled = false |
Failed |
IN9 138
OUT108 |
| × |
LcbKAvoidingArrayLiveCodeBench / leetcode / medium: determine-the-minimum-sum-of-a-k-avoiding-array (LCB 2811)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimumSum case0: expected 18 got 15 |
Failed |
IN9 125
OUT139 |
| ✓ |
LcbMaxAchievableNumberLiveCodeBench / leetcode / easy: find-the-maximum-achievable-number (LCB 2812)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 139
OUT27 |
| ✓ |
LcbMinCostEqualCharsLiveCodeBench / leetcode / medium: minimum-cost-to-make-all-characters-equal (LCB 2817)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 167
OUT91 |
| × |
LcbMinimizeStringLengthLiveCodeBench / leetcode / easy: minimize-string-length (LCB 2825)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimizedStringLength case1: expected 3 got 4 |
Failed |
IN9 156
OUT132 |
| ✓ |
LcbRemoveTrailingZerosLiveCodeBench / leetcode / easy: remove-trailing-zeros-from-a-string (LCB 2819)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 109
OUT64 |
| ✓ |
LcbSmallestPalindromeLiveCodeBench / leetcode / easy: lexicographically-smallest-palindrome (LCB 2816)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 146
OUT205 |
Code Generation - LiveCodeBench v1 LeetCode (set 4)
5 passed4 failed |
55.6% |
IN83 007
OUT2 353 |
| ✓ |
LcbBeautifulPairsLiveCodeBench / leetcode / easy: number-of-beautiful-pairs (id 2831)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 191
OUT242 |
| × |
LcbCountZeroRequestServersLiveCodeBench / leetcode / medium: count-zero-request-servers (id 2833)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: countServers case1: expected [0, 1] got [1,1] |
Failed |
IN9 290
OUT418 |
| × |
LcbLongestEqualSubarrayLiveCodeBench / leetcode / medium: find-the-longest-equal-subarray (id 2832)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestEqualSubarray case0: expected 3 got 5 |
Failed |
IN9 195
OUT258 |
| ✓ |
LcbMakeIntegerZeroLiveCodeBench / leetcode / medium: minimum-operations-to-make-the-integer-zero (id 2837)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 182
OUT185 |
| × |
LcbMaximumSumQueriesLiveCodeBench / leetcode / hard: maximum-sum-queries (id 2839)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maximumSumQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:26: Type 'List<long>' has no supported method named 'Slice' with 1 argument(s).
66:9: Unknown variable 'nums1'.
66:20: Unknown variable 'nums1'.
70:9: Unknown variable 'nums1'.
70:20: Unknown variable 'nums1'.
74:9: Unknown variable 'nums2'.
74:20: Unknown variable 'nums2'.
78:9: Unknown variable 'nums2'.
78:20: Unknown variable 'nums2'.
Values:
run.Compiled = false |
Failed |
IN9 342
OUT580 |
| ✓ |
LcbRelocateMarblesLiveCodeBench / leetcode / medium: relocate-marbles (id 2834)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 268
OUT253 |
| × |
LcbSmallestStringOpLiveCodeBench / leetcode / medium: lexicographically-smallest-string-after-substring-operation (id 2828)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: smallestString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:36: Expected ')' to close the StringBuilder constructor. Use 'new StringBuilder()'., found 's'.
Values:
run.Compiled = false |
Failed |
IN9 207
OUT220 |
| ✓ |
LcbSumOfSquaresSpecialLiveCodeBench / leetcode / easy: sum-of-squares-of-special-elements (id 2844)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 156
OUT94 |
| ✓ |
LcbValueOfPartitionLiveCodeBench / leetcode / medium: find-the-value-of-the-partition (id 2845)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 176
OUT103 |
Code Generation - LiveCodeBench v1 LeetCode (set 5)
4 passed5 failed |
44.4% |
IN82 916
OUT2 459 |
| × |
LcbCountCompleteSubarraysLiveCodeBench / leetcode / medium: count-complete-subarrays-in-an-array (id 2856)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countCompleteSubarrays case0: expected 4 got 8 |
Failed |
IN9 161
OUT329 |
| × |
LcbDecrementalConcatLiveCodeBench / leetcode / medium: decremental-string-concatenation (id 2854)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minimizeConcatenatedLength case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:46: List constructor expects a List or HashSet to copy, but got 'int'.
23:13: Variable 'firstCh' has type 'int' but its initializer has type 'char'.
24:13: Variable 'lastCh' has type 'int' but its initializer has type 'char'.
Values:
run.Compiled = false |
Failed |
IN9 273
OUT708 |
| × |
LcbLongestEvenOddThresholdLiveCodeBench / leetcode / easy: longest-even-odd-subarray-with-threshold (id 2866)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestEvenOdd case0: expected 3 got 2 |
Failed |
IN9 245
OUT234 |
| × |
LcbLongestNewStringLiveCodeBench / leetcode / medium: construct-the-longest-new-string (id 2850)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestString case0: expected 12 got 18 |
Failed |
IN9 197
OUT172 |
| ✓ |
LcbMaximumJumpsLiveCodeBench / leetcode / medium: maximum-number-of-jumps-to-reach-the-last-index (id 2855)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 244
OUT202 |
| ✓ |
LcbMaxStringPairsLiveCodeBench / leetcode / easy: find-maximum-number-of-string-pairs (id 2847)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 195
OUT133 |
| × |
LcbSpecialPermutationsLiveCodeBench / leetcode / medium: special-permutations (id 2848)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: specialPerm case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:54: Expected ')' to close the argument list, found 'val'.
Values:
run.Compiled = false |
Failed |
IN9 178
OUT388 |
| ✓ |
LcbSumImbalanceNumbersLiveCodeBench / leetcode / hard: sum-of-imbalance-numbers-of-all-subarrays (id 2849)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 200
OUT180 |
| ✓ |
LcbTotalDistanceTraveledLiveCodeBench / leetcode / easy: total-distance-traveled (id 2857)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 223
OUT113 |
Code Generation - LiveCodeBench v1 LeetCode (set 6)
5 passed3 failed |
62.5% |
IN73 684
OUT2 931 |
| ✓ |
LcbContinuousSubarraysLiveCodeBench / leetcode / medium: continuous-subarrays (id 2868)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 186
OUT155 |
| ✓ |
LcbEmployeesMetTargetLiveCodeBench / leetcode / easy: number-of-employees-who-met-the-target (id 2876)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 200
OUT59 |
| × |
LcbGoodSubarraySplitsLiveCodeBench / leetcode / medium: ways-to-split-array-into-good-subarrays (id 2867)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: goodSubarraySplits case0: expected 3 got 6 |
Failed |
IN9 181
OUT219 |
| ✓ |
LcbLongestAlternatingLiveCodeBench / leetcode / easy: longest-alternating-subarray (id 2870)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 265
OUT234 |
| ✓ |
LcbLongestNonDecreasingLiveCodeBench / leetcode / medium: longest-non-decreasing-subarray-from-two-arrays (id 2869)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 264
OUT306 |
| ✓ |
LcbMaxArrayValueMergeLiveCodeBench / leetcode / medium: largest-element-in-an-array-after-merge-operations (id 2872)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 193
OUT152 |
| × |
LcbPrimePairsSumLiveCodeBench / leetcode / medium: prime-pairs-with-target-sum (id 2873)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: findPrimePairs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:14: Unexpected character '`'.
1:21: Unexpected character '`'.
1:32: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 195
OUT508 |
| × |
LcbShortestThreeStringsLiveCodeBench / leetcode / medium: shortest-string-that-contains-three-strings (id 2877)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minimumString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
78:61: Expected an expression, found end of file.
Values:
run.Compiled = false |
Failed |
IN9 200
OUT1 298 |
Code Generation - LiveCodeBench v1 LeetCode (set 7)
3 passed6 failed |
33.3% |
IN82 444
OUT5 413 |
| × |
LcbCheckArrayZeroableLiveCodeBench / leetcode / medium: apply-operations-to-make-all-array-elements-equal-to-zero (id 2878)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: CheckArrayZeroable case0: expected true got false |
Failed |
IN9 155
OUT245 |
| × |
LcbFaultyKeyboardLiveCodeBench / leetcode / easy: faulty-keyboard (id 2886)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: FaultyKeyboard case0: expected "rtsng" got "gnrts" |
Failed |
IN9 115
OUT193 |
| ✓ |
LcbLongestValidSubstringForbiddenLiveCodeBench / leetcode / hard: length-of-the-longest-valid-substring (id 2884)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 139
OUT206 |
| × |
LcbMinBeautifulSubstringsLiveCodeBench / leetcode / medium: partition-string-into-minimum-beautiful-substrings (id 2883)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinBeautifulSubstrings case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
63:11: Type 'StringBuilder' has no supported method named 'Insert' with 2 argument(s). Supported methods: Append, AppendLine, Clear, ToString.
Values:
run.Compiled = false |
Failed |
IN9 149
OUT3 189 |
| × |
LcbMinChangesKSemiPalindromesLiveCodeBench / leetcode / hard: minimum-changes-to-make-k-semi-palindromes (id 2879)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinChangesKSemiPalindromes case1: expected 2 got 3 |
Failed |
IN9 178
OUT618 |
| × |
LcbMinIndexValidSplitLiveCodeBench / leetcode / medium: minimum-index-of-a-valid-split (id 2888)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: MinIndexValidSplit case0 runtime error: The dictionary does not contain the key 0. Use ContainsKey to check for it first.
Values:
run.Success = false |
Failed |
IN9 213
OUT249 |
| ✓ |
LcbSortVowelsInStringLiveCodeBench / leetcode / medium: sort-vowels-in-a-string (id 2887)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 141
OUT285 |
| ✓ |
LcbSplitWordsBySeparatorLiveCodeBench / leetcode / easy: split-strings-by-separator (id 2881)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 179
OUT92 |
| × |
LcbWaysSumOfPowersLiveCodeBench / leetcode / medium: ways-to-express-an-integer-as-sum-of-powers (id 2882)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: WaysSumOfPowers case0: expected 1 got 0 |
Failed |
IN9 175
OUT336 |
Code Generation - LiveCodeBench v1 LeetCode (set 8)
5 passed4 failed |
55.6% |
IN82 568
OUT5 020 |
| × |
LcbCanSplitArrayMLiveCodeBench / leetcode / medium: check-if-it-is-possible-to-split-array (id 2916)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: CanSplitArrayM case1: expected false got true |
Failed |
IN9 199
OUT3 328 |
| ✓ |
LcbCountInterestingSubarraysLiveCodeBench / leetcode / medium: count-of-interesting-subarrays (id 2915)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 186
OUT233 |
| ✓ |
LcbCountPairsSumLessTargetLiveCodeBench / leetcode / easy: count-pairs-whose-sum-is-less-than-target (id 2917)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 146
OUT129 |
| ✓ |
LcbIsArrayGoodLiveCodeBench / leetcode / easy: check-if-array-is-good (id 2892)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 194
OUT224 |
| ✓ |
LcbMaxBeautyAfterOpLiveCodeBench / leetcode / medium: maximum-beauty-of-an-array-after-applying-operation (id 2891)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 179
OUT152 |
| × |
LcbMaxIncreasingGroupsLiveCodeBench / leetcode / hard: maximum-number-of-groups-with-increasing-length (id 2919)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaxIncreasingGroups case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:5: Not all code paths in function 'maxIncreasingGroups' return a value.
Values:
run.Compiled = false |
Failed |
IN9 173
OUT202 |
| ✓ |
LcbMaxPairSumEqualDigitLiveCodeBench / leetcode / easy: max-pair-sum-in-an-array (id 2902)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 139
OUT208 |
| × |
LcbMaxScoreVisitPositionsLiveCodeBench / leetcode / medium: visit-array-positions-to-maximize-score (id 2893)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxScoreVisitPositions case1: expected 20 got 18 |
Failed |
IN9 186
OUT323 |
| × |
LcbMinSecondsCircularEqualLiveCodeBench / leetcode / medium: minimum-seconds-to-equalize-a-circular-array (id 2920)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinSecondsCircularEqual case0: expected 1 got 0 |
Failed |
IN9 166
OUT221 |
Code Generation - LiveCodeBench v1 LeetCode (set 9)
6 passed2 failed |
75.0% |
IN73 483
OUT5 037 |
| ✓ |
LcbAccountBalanceRoundedLiveCodeBench / leetcode / easy: account-balance-after-rounded-purchase (id 2955)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 152
OUT87 |
| ✓ |
LcbCountPairsDistanceKLiveCodeBench / leetcode / medium: count-pairs-of-points-with-distance-k (id 2953)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 206
OUT156 |
| × |
LcbCountSteppingNumbersLiveCodeBench / leetcode / hard: count-stepping-numbers-in-range (id 2921)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountSteppingNumbers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'countSteppingNumbers' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, CountSteppingNumbers, F, dp.
Values:
run.Compiled = false |
Failed |
IN9 158
OUT592 |
| ✓ |
LcbCountSymmetricIntegersLiveCodeBench / leetcode / easy: count-symmetric-integers (id 2998)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 151
OUT192 |
| ✓ |
LcbIsAcronymOfWordsLiveCodeBench / leetcode / easy: check-if-a-string-is-an-acronym-of-words (id 2977)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 165
OUT82 |
| × |
LcbMaximizeSalesmanProfitLiveCodeBench / leetcode / medium: maximize-the-profit-as-the-salesman (id 2979)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaximizeSalesmanProfit case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'maximizeTheProfit' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MaximizeTheProfit.
Values:
run.Compiled = false |
Failed |
IN9 215
OUT162 |
| ✓ |
LcbMaxSumAlmostUniqueLiveCodeBench / leetcode / medium: maximum-sum-of-almost-unique-subarray (id 2954)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 210
OUT370 |
| ✓ |
LcbMinTimeSumAtMostXLiveCodeBench / leetcode / hard: minimum-time-to-make-array-sum-at-most-x (id 2952)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 226
OUT3 396 |
Code Generation - LiveCodeBench v2 AtCoder (set 1)
5 passed3 failed |
62.5% |
IN73 816
OUT13 085 |
| ✓ |
LcbABCSumQueryLiveCodeBench / atcoder abc344_c / medium: A+B+C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 263
OUT1 367 |
| ✓ |
LcbCeilDivTenLiveCodeBench / atcoder abc345_b / easy: Integer Division Returns (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT87 |
| ✓ |
LcbDelimiterReverseLiveCodeBench / atcoder abc344_b / easy: Delimiter (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 196
OUT104 |
| × |
LcbGomamayoSequenceLiveCodeBench / atcoder abc346_d / medium: Gomamayo Sequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GomamayoSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:75: Unexpected character '≤'.
3:79: Unexpected character '≤'.
8:21: Unexpected character '→'.
9:32: Unexpected character '→'.
9:42: Char literals must contain exactly one character and close with '''.
10:104: Unexpected character '→'.
10:151: Unexpected character '→'.
10:192: Unexpected character '→'.
10:239: Char literals must contain exactly one character and close with '''.
12:177: Char literals must contain exactly one character and close with '''.
16:111: Char literals must contain exactly one character and close with '''.
24:74: Char literals must contain exactly one character and close with '''.
26:278: Char literals must contain exactly one character and close with '''.
28:21: Char literals must contain exactly one character and close with '''.
30:152: Unexpected character '∈'.
30:183: Unexpected character '∈'.
34:4: Char literals must contain exactly one character and close with '''.
35:6: Unexpected character '≤'.
36:49: Char literals must contain exactly one character and close with '''.
39:6: Unexpected character '≥'.
41:151: Unexpected character '≥'.
44:9: Unexpected character '≤'.
45:9: Unexpected character '≥'.
47:13: Unexpected character '≥'.
49:10: Unexpected character '≤'.
51:203: Char literals must contain exactly one character and close with '''.
53:42: Unexpected character '∈'.
53:62: Unexpected character '∈'.
55:18: Unexpected character '≤'.
55:49: Unexpected character '≥'.
68:22: Unexpected character '≤'.
69:24: Unexpected character '≥'.
75:26: Unexpected character '≤'.
75:28: Unexpected character '≤'.
75:43: Unexpected character '≥'.
75:66: Unexpected character '≥'.
Values:
run.Compiled = false |
Failed |
IN9 259
OUT2 439 |
| ✓ |
LcbInsertOrEraseLiveCodeBench / atcoder abc344_e / hard: Insert or Erase (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 251
OUT394 |
| ✓ |
LcbSpoilerRemoveLiveCodeBench / atcoder abc344_a / easy: Spoiler (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 151
OUT62 |
| × |
LcbStringBagsLiveCodeBench / atcoder abc344_d / hard: String Bags (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: StringBags case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 299
OUT505 |
| × |
LcbTilingGridLiveCodeBench / atcoder abc345_d / hard: Tiling (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TilingGrid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
35:10: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed |
IN9 248
OUT8 127 |
Code Generation - LiveCodeBench v2 AtCoder (set 2)
2 passed5 failed |
28.6% |
IN90 845
OUT8 995 |
| ✓ |
LcbAdjacentProductLiveCodeBench / atcoder abc346_a / easy: Adjacent Product (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 174
OUT241 |
| ✓ |
LcbDivisibleQuotientsLiveCodeBench / atcoder abc347_a / easy: Divisible (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 184
OUT199 |
| × |
LcbIdealHolidaysLiveCodeBench / atcoder abc347_c / medium: Ideal Holidays (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: IdealHolidays case0: expected [Yes] got [No] |
Failed |
IN9 220
OUT254 |
| × |
LcbPaintGridLiveCodeBench / atcoder abc346_e / hard: Paint (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PaintGrid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:42: Expected a type name after 'new', found 'char'.
Values:
run.Compiled = false |
Failed |
IN22 566
OUT3 536 |
| × |
LcbPianoSubstringLiveCodeBench / atcoder abc346_b / easy: Piano (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PianoSubstring case1: expected [No] got [Yes] |
Failed |
IN22 285
OUT4 272 |
| × |
LcbSetAddQueryLiveCodeBench / atcoder abc347_e / hard: Set Add Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SetAddQuery case0: expected [6 2 2] got [1 1 0] |
Failed |
IN9 249
OUT332 |
| × |
LcbSigmaMissingSumLiveCodeBench / atcoder abc346_c / medium: Sigma (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: SigmaMissingSum case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 167
OUT161 |
Code Generation - LiveCodeBench v2 AtCoder (set 3)
6 passed2 failed |
75.0% |
IN184 298
OUT12 144 |
| × |
LcbAirportCodeLiveCodeBench / atcoder / abc349_c: Airport Code (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AirportCode case2: expected [No] got [Yes] |
Failed |
IN9 292
OUT353 |
| ✓ |
LcbColorfulBeansLiveCodeBench / atcoder / abc348_c: Colorful Beans (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 281
OUT194 |
| ✓ |
LcbDistinctSubstringsLiveCodeBench / atcoder / abc347_b: Substring (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT103 |
| ✓ |
LcbFarthestPointLiveCodeBench / atcoder / abc348_b: Farthest Point (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 333
OUT321 |
| × |
LcbMedicinesOnGridLiveCodeBench / atcoder / abc348_d: Medicines on Grid (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MedicinesOnGrid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:58: Unexpected character '≤'.
5:64: Unexpected character '≤'.
5:217: Char literals must contain exactly one character and close with '''.
7:131: Char literals must contain exactly one character and close with '''.
13:64: Char literals must contain exactly one character and close with '''.
15:141: Char literals must contain exactly one character and close with '''.
15:380: Unexpected character '#'.
15:381: Char literals must contain exactly one character and close with '''.
15:405: Char literals must contain exactly one character and close with '''.
15:412: Char literals must contain exactly one character and close with '''.
15:419: Char literals must contain exactly one character and close with '''.
18:25: Unexpected character '≤'.
18:100: Unexpected character '≥'.
20:131: Char literals must contain exactly one character and close with '''.
22:91: Unexpected character '—'.
22:314: Unexpected character '≤'.
22:429: Unexpected character '≥'.
22:485: Unexpected character '→'.
22:504: Unexpected character '≤'.
22:546: Char literals must contain exactly one character and close with '''.
22:799: Char literals must contain exactly one character and close with '''.
24:21: Char literals must contain exactly one character and close with '''.
26:2: Char literals must contain exactly one character and close with '''.
28:104: Unexpected character '≤'.
28:260: Unexpected character '≤'.
28:326: Unexpected character '≤'.
28:343: Unexpected character '—'.
30:57: Unexpected character '→'.
30:73: Unexpected character '≤'.
32:54: Unexpected character '→'.
32:74: Unexpected character '→'.
32:82: Unexpected character '≤'.
34:39: Unexpected character '≤'.
34:50: Unexpected character '→'.
36:138: Unexpected character '×'.
38:70: Unexpected character '≠'.
38:217: Unexpected character '→'.
42:18: Unexpected character '#'.
48:319: Unexpected character '≤'.
48:362: Unexpected character '≈'.
68:8: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 512
OUT3 058 |
| ✓ |
LcbMinimizeSumOfDistancesLiveCodeBench / atcoder / abc348_e: Minimize Sum of Distances (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN119 190
OUT7 946 |
| ✓ |
LcbPenaltyKickLiveCodeBench / atcoder / abc348_a: Penalty Kick (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 226
OUT86 |
| ✓ |
LcbZeroSumGameLiveCodeBench / atcoder / abc349_a: Zero Sum Game (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 272
OUT83 |
Code Generation - LiveCodeBench v2 AtCoder (set 4)
4 passed1 failed2 skipped |
80.0% |
IN58 438
OUT2 474 |
| ✓ |
LcbCommencementLiveCodeBench / atcoder / abc349_b: Commencement (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 215
OUT217 |
| ✓ |
LcbDentistAokiLiveCodeBench / atcoder / abc350_b: Dentist Aoki (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 279
OUT166 |
| ✓ |
LcbDivideIntervalLiveCodeBench / atcoder / abc349_d: Divide Interval (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN21 192
OUT915 |
| ✓ |
LcbPastABCsLiveCodeBench / atcoder / abc350_a: Past ABCs (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 297
OUT104 |
| – |
LcbSortPermutationLiveCodeBench / atcoder / abc350_c: Sort (Pattern B, stdin) TRIAGE: 0 public tests in dump + non-unique output (any valid swap sequence accepted) — cannot oracle-compare.0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: no public tests in dump; output non-unique (any valid swap sequence is accepted) — exact-compare oracle inapplicable, coordinator triage |
Skipped |
IN0
OUT0 |
| – |
LcbTowardZeroLiveCodeBench / atcoder / abc350_e: Toward 0 (Pattern B, stdin) TRIAGE: float output (absolute/relative error <= 1e-6) — RunLcbStdin does exact string compare; needs tolerance.0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: float output requires error tolerance (<=1e-6); RunLcbStdin does exact string compare — coordinator triage |
Skipped |
IN0
OUT0 |
| × |
LcbWeightedTicLiveCodeBench / atcoder / abc349_e: Weighted Tic (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WeightedTic case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:34: Expected ')' to close the argument list, found '{'.
Values:
run.Compiled = false |
Failed |
IN9 455
OUT1 072 |
Code Generation - LiveCodeBench v2 AtCoder (set 5)
3 passed5 failed |
37.5% |
IN137 324
OUT8 658 |
| ✓ |
LcbBottomNinthLiveCodeBench / atcoder / abc351_a: The bottom of the ninth (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN20 468
OUT278 |
| × |
LcbCliqueConnectLiveCodeBench / atcoder / abc352_e: Clique Connect (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CliqueConnect case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:90: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:94: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:116: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:120: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:29: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:30: Unexpected character '\'.
5:42: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:71: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:73: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:133: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:135: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:174: Char literals must contain exactly one character and close with '''.
7:175: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:179: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:200: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:204: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:279: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:285: Unexpected character '\'.
7:294: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:36: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:42: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:56: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:58: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:78: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:83: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:113: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:115: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:128: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:130: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:144: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:148: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:154: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:157: Unexpected character '\'.
9:164: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:297: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:301: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:409: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:413: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:422: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:428: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:435: Char literals must contain exactly one character and close with '''.
11:296: Char literals must contain exactly one character and close with '''.
13:10: Char literals must contain exactly one character and close with '''.
22:74: Char literals must contain exactly one character and close with '''.
24:45: Char literals must contain ex ... (message truncated) |
Failed |
IN9 341
OUT1 601 |
| ✓ |
LcbDoubleSumMaxDiffLiveCodeBench / atcoder / abc351_f: Double Sum (Pattern B, stdin)1 AI query
3 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN37 353
OUT2 454 |
| × |
LcbGridAndMagnetLiveCodeBench / atcoder / abc351_d: Grid and Magnet (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GridAndMagnet case0: expected [9] got [4] |
Failed |
IN9 296
OUT927 |
| × |
LcbJumpDistanceSumLiveCodeBench / atcoder / abc351_e: Jump Distance Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: JumpDistanceSum case0: expected [3] got [26] |
Failed |
IN9 297
OUT262 |
| × |
LcbMergeTheBallsLiveCodeBench / atcoder / abc351_c: Merge the balls (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MergeTheBalls case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:47: Expected ')' to close the argument list, found 'int'.
Values:
run.Compiled = false |
Failed |
IN9 275
OUT179 |
| ✓ |
LcbNewFriendsLiveCodeBench / atcoder / abc350_d: New Friends (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN32 990
OUT2 758 |
| × |
LcbSpotTheDifferenceLiveCodeBench / atcoder / abc351_b: Spot the Difference (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SpotTheDifference case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
26:15: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 304
OUT199 |
Code Generation - LiveCodeBench v2 AtCoder (set 6)
6 passed1 failed |
85.7% |
IN75 757
OUT7 462 |
| ✓ |
LcbAmusementParkLiveCodeBench / atcoder / abc353_b: AtCoder Amusement Park (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 286
OUT173 |
| ✓ |
LcbAtcoderLineLiveCodeBench / atcoder / abc352_a: AtCoder Line (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 239
OUT1 070 |
| ✓ |
LcbBuildingsTallerLiveCodeBench / atcoder / abc353_a: Buildings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 213
OUT147 |
| × |
LcbPermutationSubsequenceLiveCodeBench / atcoder / abc352_d: Permutation Subsequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PermutationSubsequence case2: expected [5] got [4] |
Failed |
IN9 301
OUT3 930 |
| ✓ |
LcbSigmaRemainderSumLiveCodeBench / atcoder / abc353_c: Sigma Problem (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN20 135
OUT328 |
| ✓ |
LcbStandingOnShouldersLiveCodeBench / atcoder / abc352_c: Standing On The Shoulders (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 312
OUT1 649 |
| ✓ |
LcbTypingBackspaceLiveCodeBench / atcoder / abc352_b: Typing (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 271
OUT165 |
Code Generation - LiveCodeBench v2 AtCoder (set 7)
2 passed4 failed |
33.3% |
IN55 858
OUT5 661 |
| × |
LcbAnotherSigmaProblemLiveCodeBench / atcoder / abc353_d: Another Sigma Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AnotherSigmaProblem case0: expected [2044] got [507] |
Failed |
IN9 320
OUT253 |
| ✓ |
LcbAtcoderJanken2LiveCodeBench / atcoder / abc354_b: AtCoder Janken 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 304
OUT116 |
| × |
LcbAtcoderMagicsLiveCodeBench / atcoder / abc354_c: AtCoder Magics (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AtcoderMagics case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:16: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed |
IN9 373
OUT2 120 |
| ✓ |
LcbExponentialPlantLiveCodeBench / atcoder / abc354_a: Exponential Plant (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 257
OUT77 |
| × |
LcbRemovePairsLiveCodeBench / atcoder / abc354_e: Remove Pairs (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RemovePairs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
39:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 350
OUT2 872 |
| × |
LcbYetAnotherSigmaProblemLiveCodeBench / atcoder / abc353_e: Yet Another Sigma Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: YetAnotherSigmaProblem case0: expected [4] got [0] |
Failed |
IN9 254
OUT223 |
Code Generation - LiveCodeBench v2 AtCoder (set 8)
3 passed2 failed1 skipped |
60.0% |
IN46 533
OUT3 477 |
| × |
LcbAtcoderWallpaperLiveCodeBench / atcoder / abc354_d: AtCoder Wallpaper (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AtcoderWallpaper case0: expected [10] got [5] |
Failed |
IN9 365
OUT1 687 |
| ✓ |
LcbBingo2LiveCodeBench / atcoder / abc355_c: Bingo 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 378
OUT354 |
| – |
LcbGuessTheSumLiveCodeBench / atcoder / abc355_e: Guess the Sum (Pattern B, stdin) — INTERACTIVE, cannot run in the batch harness0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: SC# subset: interactive problem (adaptive query/response dialogue; not expressible as a single stdin->stdout Solve) |
Skipped |
IN0
OUT0 |
| ✓ |
LcbIntersectingIntervalsLiveCodeBench / atcoder / abc355_d: Intersecting Intervals (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 235
OUT1 120 |
| ✓ |
LcbPiano2LiveCodeBench / atcoder / abc355_b: Piano 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 303
OUT252 |
| × |
LcbWhoAteTheCakeLiveCodeBench / atcoder / abc355_a: Who Ate the Cake? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WhoAteTheCake case1: expected [-1] got [4] |
Failed |
IN9 252
OUT64 |
Code Generation - LiveCodeBench v2 LeetCode (set 1)
4 passed5 failed |
44.4% |
IN82 585
OUT1 735 |
| ✓ |
LcbCountSubstringsStartEndCharLiveCodeBench / leetcode / medium: count-substrings-starting-and-ending-with-given-character (LCB 3337)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 105
OUT67 |
| ✓ |
LcbMaximumHappinessSumLiveCodeBench / leetcode / medium: maximize-happiness-of-selected-children (LCB 3351)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 200
OUT103 |
| × |
LcbMaximumStrengthSubarraysLiveCodeBench / leetcode / hard: maximum-strength-of-k-disjoint-subarrays (LCB 3313)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maximumStrength case0: expected 22 got -4611686018427387880 |
Failed |
IN9 255
OUT328 |
| ✓ |
LcbMaxLengthTwoOccurrencesLiveCodeBench / leetcode / easy: maximum-length-substring-with-two-occurrences (LCB 3349)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 087
OUT195 |
| × |
LcbMinDeletionsKSpecialLiveCodeBench / leetcode / medium: minimum-deletions-to-make-string-k-special (LCB 3360)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimumDeletions case0: expected 3 got 1 |
Failed |
IN9 160
OUT449 |
| ✓ |
LcbMinimumBoxesApplesLiveCodeBench / leetcode / easy: apple-redistribution-into-boxes (LCB 3334)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 190
OUT105 |
| × |
LcbMinMovesPickKOnesLiveCodeBench / leetcode / hard: minimum-moves-to-pick-k-ones (LCB 3327)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumMoves case0: expected 3 got 2 |
Failed |
IN9 314
OUT383 |
| × |
LcbShortestUncommonSubstringsLiveCodeBench / leetcode / medium: shortest-uncommon-substring-in-an-array (LCB 3356)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: shortestSubstrings case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:9: Expected '(' to declare a function or '=' to declare a global variable after 'LIST OF', found 'FILES'. Global variables must have an initializer.
Values:
run.Compiled = false |
Failed |
IN9 168
OUT11 |
| × |
LcbSubstringInReverseLiveCodeBench / leetcode / easy: existence-of-a-substring-in-a-string-and-its-reverse (LCB 3353)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: isSubstringPresent case1: expected true got false |
Failed |
IN9 106
OUT94 |
Code Generation - LiveCodeBench v2 LeetCode (set 2)
8 passed1 failed |
88.9% |
IN82 482
OUT11 369 |
| ✓ |
LcbCountAlternatingSubarraysLiveCodeBench / leetcode / medium: count-alternating-subarrays (LCB 3374)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 118
OUT131 |
| ✓ |
LcbHarshadDigitSumLiveCodeBench / leetcode / easy: harshad-number (LCB 3371)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 117
OUT108 |
| × |
LcbLexSmallestWithinDistanceLiveCodeBench / leetcode / medium: lexicographically-smallest-string-after-operations-with-constraint (LCB 3346)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: getSmallestString case0: expected "aaaz" got "zbbz" |
Failed |
IN9 213
OUT352 |
| ✓ |
LcbLongestCommonSuffixQueriesLiveCodeBench / leetcode / hard: longest-common-suffix-queries (LCB 3376)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 212
OUT3 247 |
| ✓ |
LcbLongestMonotonicSubarrayLiveCodeBench / leetcode / easy: longest-strictly-increasing-or-decreasing-subarray (LCB 3372)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 126
OUT269 |
| ✓ |
LcbMinimizeManhattanDistanceLiveCodeBench / leetcode / hard: minimize-manhattan-distances (LCB 3344)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 149
OUT774 |
| ✓ |
LcbMinOpsMedianKLiveCodeBench / leetcode / medium: minimum-operations-to-make-median-of-array-equal-to-k (LCB 3387)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 189
OUT5 381 |
| ✓ |
LcbMinOpsSumAtLeastKLiveCodeBench / leetcode / medium: apply-operations-to-make-sum-of-array-ge-k (LCB 3328)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 144
OUT206 |
| ✓ |
LcbMostFrequentIdsLiveCodeBench / leetcode / medium: most-frequent-ids (LCB 3363)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 214
OUT901 |
Code Generation - LiveCodeBench v2 LeetCode (set 3)
3 passed6 failed |
33.3% |
IN82 832
OUT7 502 |
| × |
LcbKthSmallestAmountLiveCodeBench 3375 / leetcode / hard: kth-smallest-amount-with-single-denomination-combination (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: kthSmallestAmount case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
35:9: Cannot assign a value of type 'long' to variable 'k' of type 'int'.
1:6: Not all code paths in function 'findKthSmallest' return a value.
Values:
run.Compiled = false |
Failed |
IN9 189
OUT264 |
| × |
LcbLatestTimeReplaceLiveCodeBench 3361 / leetcode / easy: latest-time-you-can-obtain-after-replacing-characters (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: latestTimeReplace case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
23:20: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 213
OUT235 |
| ✓ |
LcbMaximumPrimeDifferenceLiveCodeBench 3373 / leetcode / medium: maximum-prime-difference (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 158
OUT192 |
| × |
LcbMinimizeStringValueLiveCodeBench 3354 / leetcode / medium: replace-question-marks-in-string-to-minimize-its-value (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minimizeStringValue case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:10: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '('.
Values:
run.Compiled = false |
Failed |
IN9 225
OUT744 |
| × |
LcbMinSumValuesDividingLiveCodeBench 3364 / leetcode / hard: minimum-sum-of-values-by-dividing-array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: minSumValuesDividing case0 runtime error: List index 4 is out of range. The list has 4 element(s).
Values:
run.Success = false |
Failed |
IN9 277
OUT434 |
| × |
LcbShortestSubarrayOrKLiveCodeBench 3381 / leetcode / easy: shortest-subarray-with-or-at-least-k-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: shortestSubarrayOrK case2 runtime error: List index 2 is out of range. The list has 2 element(s).
Values:
run.Success = false |
Failed |
IN9 181
OUT145 |
| ✓ |
LcbSumOfEncryptedIntsLiveCodeBench 3367 / leetcode / easy: find-the-sum-of-encrypted-integers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 159
OUT198 |
| ✓ |
LcbSumOfPowerSubsequencesLiveCodeBench 3345 / leetcode / hard: find-the-sum-of-the-power-of-all-subsequences (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 174
OUT2 473 |
| × |
LcbUnmarkedSumArrayLiveCodeBench 3306 / leetcode / medium: mark-elements-on-array-by-performing-queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: unmarkedSumArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:7: Expected a name, found '='.
Values:
run.Compiled = false |
Failed |
IN9 256
OUT2 817 |
Code Generation - LiveCodeBench v2 LeetCode (set 4)
3 passed6 failed |
33.3% |
IN82 587
OUT2 042 |
| × |
LcbAddedIntegerLiveCodeBench 3397 / leetcode / easy: find-the-integer-added-to-array-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: addedInteger case0: expected 3 got 9 |
Failed |
IN9 202
OUT96 |
| × |
LcbMedianUniquenessArrayLiveCodeBench 3362 / leetcode / hard: find-the-median-of-the-uniqueness-array (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: medianUniquenessArray case0: expected 1 got 3 |
Failed |
IN9 222
OUT439 |
| × |
LcbMinimumArrayEndLiveCodeBench 3394 / leetcode / medium: minimum-array-end (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumArrayEnd case1: expected 15 got 7 |
Failed |
IN9 166
OUT93 |
| × |
LcbMinimumLevelsGainPointsLiveCodeBench 3355 / leetcode / medium: minimum-levels-to-gain-more-points (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimumLevelsGainPoints case0: expected 1 got -1 |
Failed |
IN9 225
OUT188 |
| ✓ |
LcbNumberOfSpecialCharsLiveCodeBench 3408 / leetcode / easy: count-the-number-of-special-characters-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 121
OUT150 |
| ✓ |
LcbScoreOfStringLiveCodeBench 3379 / leetcode / easy: score-of-a-string (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 172
OUT106 |
| × |
LcbSubarraysBoundaryMaxLiveCodeBench 3382 / leetcode / hard: find-the-number-of-subarrays-where-boundary-elements-are-maximum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: subarraysBoundaryMax case0: expected 6 got 7 |
Failed |
IN9 127
OUT247 |
| × |
LcbSumOfSubsequencePowersLiveCodeBench 3316 / leetcode / hard: find-the-sum-of-subsequence-powers (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: sumOfSubsequencePowers case0: expected 4 got 16 |
Failed |
IN9 182
OUT470 |
| ✓ |
LcbValidWordLiveCodeBench 3396 / leetcode / easy: valid-word (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 170
OUT253 |
Code Generation - LiveCodeBench v2 LeetCode (set 5)
5 passed4 failed |
55.6% |
IN82 723
OUT2 552 |
| ✓ |
LcbCanMakeSquareColorLiveCodeBench / leetcode / easy: make-a-square-with-the-same-color1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 217
OUT221 |
| ✓ |
LcbIsArraySpecialILiveCodeBench / leetcode / easy: special-array-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 127
OUT95 |
| ✓ |
LcbMakeWordKPeriodicLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-word-k-periodic1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 197
OUT181 |
| × |
LcbMinAnagramLengthLiveCodeBench / leetcode / medium: minimum-length-of-anagram-concatenation1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinAnagramLength case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minAnagramLength' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, Gcd, MinAnagramLength.
Values:
run.Compiled = false |
Failed |
IN9 131
OUT206 |
| × |
LcbMinCostEqualizeArrayLiveCodeBench / leetcode / hard: minimum-cost-to-equalize-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinCostEqualizeArray case0: expected 15 got 7 |
Failed |
IN9 261
OUT365 |
| × |
LcbMysticDungeonEnergyLiveCodeBench / leetcode / medium: taking-maximum-energy-from-the-mystic-dungeon1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MysticDungeonEnergy case0: expected 3 got 5 |
Failed |
IN9 188
OUT180 |
| ✓ |
LcbNumberOfRightTrianglesLiveCodeBench / leetcode / medium: right-triangles1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 223
OUT444 |
| × |
LcbNumberOfStableArraysLiveCodeBench / leetcode / medium: find-all-possible-stable-binary-arrays-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: NumberOfStableArrays case0: expected 2 got 0 |
Failed |
IN9 217
OUT749 |
| ✓ |
LcbPermutationDifferenceLiveCodeBench / leetcode / easy: permutation-difference-between-two-strings1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 162
OUT111 |
Code Generation - LiveCodeBench v2 LeetCode (set 6)
4 passed5 failed |
44.4% |
IN82 717
OUT11 979 |
| × |
LcbMaxSumNonAdjacentQueriesLiveCodeBench / leetcode / hard: maximum-sum-of-subsequence-with-non-adjacent-elements1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaxSumNonAdjacentQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'maximumSumSubsequence' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MaximumSumSubsequence.
Values:
run.Compiled = false |
Failed |
IN9 228
OUT333 |
| × |
LcbMinBalancedPartitionLiveCodeBench / leetcode / medium: minimum-substring-partition-of-equal-character-frequency1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinBalancedPartition case0: expected 3 got 0 |
Failed |
IN9 125
OUT454 |
| ✓ |
LcbNumberOfGoodPairsILiveCodeBench / leetcode / easy: find-the-number-of-good-pairs-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 207
OUT123 |
| ✓ |
LcbOccurrencesOfElementLiveCodeBench / leetcode / medium: find-occurrences-of-an-element-in-an-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 205
OUT154 |
| × |
LcbProductsBigArrayLiveCodeBench / leetcode / hard: find-products-of-elements-of-big-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ProductsBigArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:66: Char literals must contain exactly one character and close with '''.
14:54: Unexpected character '✓'.
22:246: Unexpected character '→'.
22:415: Char literals must contain exactly one character and close with '''.
24:126: Unexpected character '—'.
24:279: Char literals must contain exactly one character and close with '''.
26:160: Char literals must contain exactly one character and close with '''.
30:65: Unexpected character '≤'.
34:58: Char literals must contain exactly one character and close with '''.
44:116: Char literals must contain exactly one character and close with '''.
48:76: Unexpected character '≈'.
48:102: Char literals must contain exactly one character and close with '''.
48:222: Unexpected character '#'.
48:248: Unexpected character '#'.
54:99: Unexpected character '→'.
56:48: Unexpected character '→'.
56:78: Unexpected character '→'.
58:11: Unexpected character '→'.
64:71: Unexpected character 'Σ'.
66:169: Char literals must contain exactly one character and close with '''.
70:96: Unexpected character '≤'.
78:80: Unexpected character '≥'.
81:21: Unexpected character '→'.
81:102: Unexpected character '→'.
81:182: Unexpected character '≈'.
86:21: Unexpected character '→'.
89:85: Char literals must contain exactly one character and close with '''.
96:29: Unexpected character 'Σ'.
104:59: Unexpected character 'Σ'.
106:4: Char literals must contain exactly one character and close with '''.
121:4: Char literals must contain exactly one character and close with '''.
125:2: Char literals must contain exactly one character and close with '''.
127:83: Char literals must contain exactly one character and close with '''.
145:93: Unexpected character '→'.
145:272: Unexpected character '→'.
145:276: Unexpected character '→'.
145:280: Unexpected character '→'.
145:383: Unexpected character '→'.
147:87: Unexpected character '→'.
147:105: Unexpected character '✓'.
147:125: Unexpected character '→'.
147:135: Unexpected character '✓'.
155:22: Unexpected character '→'.
159:22: Unexpected character '#'.
163:2: Char literals must contain exactly one character and close with '''.
167:135: Unexpected character '→'.
169:33: Char literals must contain exactly one character and close with '''.
189:23: Unexpected character '→'.
207:14: Unexpected character 'Σ'.
229:2: Char literals must contain exactly one character and close with '''.
243:114: Unexpected character '✓'.
243:129: Unexpected character '✓'.
245:125: Unexpected character '✓'.
245:139: Unexpected character '✓'.
253:2: Char literals must contain exactly one character and close with '''.
273:2: Char literals must contain exactly one character and close with '''.
294:38: Unexpected character '→'.
336:20: Unexpected character '#'.
338:2: Char literals must contain exactly one character and close with '''.
348:2: Char literals must contain exactly one character and close with '''.
386:26: Unexpected character '≥'.
394:2: Char literals must contain exactly one character and close with '''.
595:96: Unexpected character '✓'.
597:27: Unexpected character '→'.
597:124: Unexpected character '✓'.
597:209: Char literals must contain exactly one character and close with '''.
599:46: Unexpected character '→'.
599:56: Unexpected character '✓'.
609:2: Char literals must contain exactly one character and close with '''.
619:2: Char literals must contain exactly one character and close with '''.
627:2: Char literals must contain exactly one character and close with '''.
637:2: Char literals must contain exactly one character and close with '''.
649:2: Char literals must contain exactly one character and close with '''.
665:2: Char literals must contain exactly one character and close with '''.
673:2: Char literals must contain exactly one character and close with '''.
687:2: Char literals must contain exactly one character and close with '''.
707:2: Char literals must contain exactly one character and close wit ... (message truncated) |
Failed |
IN9 310
OUT9 617 |
| ✓ |
LcbStringCompressionIIILiveCodeBench / leetcode / medium: string-compression-iii1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 149
OUT122 |
| × |
LcbSumDigitDifferencesLiveCodeBench / leetcode / medium: sum-of-digit-differences-of-all-pairs1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: SumDigitDifferences case0: expected 4 got 0 |
Failed |
IN9 147
OUT250 |
| × |
LcbWaysToReachStairLiveCodeBench / leetcode / hard: find-number-of-ways-to-reach-the-k-th-stair1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WaysToReachStair case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:5: Function 'CountWays' must return 'int', but this return statement has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 204
OUT805 |
| ✓ |
LcbXorAppearingTwiceLiveCodeBench / leetcode / easy: find-the-xor-of-numbers-which-appear-twice1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 142
OUT121 |
Code Generation - LiveCodeBench v3 AtCoder (set 1)
4 passed3 failed |
57.1% |
IN64 954
OUT4 750 |
| × |
LcbKeysCombinationsLiveCodeBench / atcoder / medium: abc356_c Keys (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: KeysCombinations case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:40: Type 'string' has no supported method named 'Split'.
11:52: Unknown variable 'System'.
Values:
run.Compiled = false |
Failed |
IN9 392
OUT399 |
| ✓ |
LcbMaskedPopcountLiveCodeBench / atcoder / medium: abc356_d Masked Popcount (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 229
OUT508 |
| × |
LcbMaxMinFloorSumLiveCodeBench / atcoder / hard: abc356_e Max/Min (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaxMinFloorSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:40: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:46: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:51: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:52: Unexpected character '\'.
3:59: Unexpected character '\'.
3:74: Unexpected character '\'.
3:88: Unexpected character '\'.
3:95: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:70: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:81: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:88: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:93: Unexpected character '\'.
5:101: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:112: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:113: Unexpected character '\'.
5:129: Unexpected character '\'.
5:136: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:37: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:38: Unexpected character '\'.
7:49: Unexpected character '\'.
7:65: Unexpected character '\'.
7:72: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:7: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:12: Unexpected character '\'.
9:18: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:24: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:29: Unexpected character '\'.
9:38: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:51: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:52: Unexpected character '\'.
9:68: Unexpected character '\'.
9:75: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:58: Char literals must contain exactly one character and close with '''.
13:26: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:32: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:51: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:52: Unexpected character '\'.
13:64: Unexpected character '\'.
13:72: Unexpected character '\'.
13:85: Unexpected character '\'.
13:97: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:16: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:22: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:41: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:44: Unexpected character '\'.
14:50: Unexpected character '\'.
14:67: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:75: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:76: Unexpected character '\'.
14:88: Unexpected character '\'.
14:99: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
16:45: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
16:46: Unexpected character '\'.
16:57: Unexpected character '\'.
16:68: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
18:57: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
18:62: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
18:102: Unexpected character '≤'.
... (message truncated) |
Failed |
IN9 216
OUT2 960 |
| ✓ |
LcbNutrientsLiveCodeBench / atcoder / easy: abc356_b Nutrients (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 293
OUT213 |
| ✓ |
LcbSanitizeHandsLiveCodeBench / atcoder / easy: abc357_a Sanitize Hands (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 268
OUT151 |
| × |
LcbSierpinskiCarpetLiveCodeBench / atcoder / medium: abc357_c Sierpinski carpet (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SierpinskiCarpet case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:13: Unknown variable 'rr'. Declare it first, e.g. 'int rr = ...;'.
19:21: Unknown variable 'cc'. Declare it first, e.g. 'int cc = ...;'.
19:29: Unknown variable 'lev'. Declare it first, e.g. 'int lev = ...;'.
19:38: Unknown variable 'ok'. Declare it first, e.g. 'int ok = ...;'.
20:20: Unknown variable 'lev'.
22:17: Unknown variable 'dr'. Declare it first, e.g. 'int dr = ...;'.
22:22: Unknown variable 'rr'.
22:30: Unknown variable 'dc'. Declare it first, e.g. 'int dc = ...;'.
22:35: Unknown variable 'cc'.
23:21: Unknown variable 'dr'.
23:32: Unknown variable 'dc'.
23:43: Unknown variable 'ok'. Declare it first, e.g. 'int ok = ...;'.
24:17: Unknown variable 'rr'. Declare it first, e.g. 'int rr = ...;'.
24:22: Unknown variable 'rr'.
24:30: Unknown variable 'cc'. Declare it first, e.g. 'int cc = ...;'.
24:35: Unknown variable 'cc'.
24:43: Unknown variable 'lev'. Declare it first, e.g. 'int lev = ...;'.
24:43: Unknown variable 'lev'.
26:17: Unknown variable 'ok'.
Values:
run.Compiled = false |
Failed |
IN9 321
OUT277 |
| ✓ |
LcbSubsegmentReverseLiveCodeBench / atcoder / easy: abc356_a Subsegment Reverse (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 235
OUT242 |
Code Generation - LiveCodeBench v3 AtCoder (set 2)
4 passed3 failed |
57.1% |
IN117 573
OUT9 386 |
| × |
LcbAlphabetTilesLiveCodeBench / atcoder / hard: abc358_e Alphabet Tiles (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AlphabetTiles case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
68:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN34 106
OUT2 260 |
| × |
LcbConcatModuloLiveCodeBench / atcoder / medium: abc357_d 88888888 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ConcatModulo case0: expected [55555] got [998234353] |
Failed |
IN9 280
OUT359 |
| × |
LcbFunctionalGraphReachLiveCodeBench / atcoder / hard: abc357_e Reachability in Functional Graph (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: FunctionalGraphReach case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 268
OUT487 |
| ✓ |
LcbPopcornStandsLiveCodeBench / atcoder / medium: abc358_c Popcorn (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 292
OUT317 |
| ✓ |
LcbSouvenirsLiveCodeBench / atcoder / medium: abc358_d Souvenirs (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN37 073
OUT5 513 |
| ✓ |
LcbTicketCounterLiveCodeBench / atcoder / easy: abc358_b Ticket Counter (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 333
OUT221 |
| ✓ |
LcbUppercaseLowercaseLiveCodeBench / atcoder / easy: abc357_b Uppercase and Lowercase (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 221
OUT229 |
Code Generation - LiveCodeBench v3 AtCoder (set 3)
2 passed5 failed |
28.6% |
IN75 536
OUT4 272 |
| × |
LcbAvoidKPalindromeLiveCodeBench / atcoder abc359_d / hard: Avoid K Palindrome (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AvoidKPalindrome case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN20 124
OUT698 |
| ✓ |
LcbCountTakahashiLiveCodeBench / atcoder abc359_a / easy: Count Takahashi (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 179
OUT76 |
| × |
LcbCouplesBetweenLiveCodeBench / atcoder abc359_b / easy: Couples (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CouplesBetween case0: expected [2] got [1] |
Failed |
IN9 192
OUT670 |
| × |
LcbMoveItBoxesLiveCodeBench / atcoder abc360_c / medium: Move It (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MoveItBoxes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:139: Char literals must contain exactly one character and close with '''.
12:153: Char literals must contain exactly one character and close with '''.
16:367: Char literals must contain exactly one character and close with '''.
18:418: Unexpected character '≤'.
18:423: Unexpected character '∩'.
20:217: Unexpected character '—'.
22:45: Char literals must contain exactly one character and close with '''.
24:261: Char literals must contain exactly one character and close with '''.
28:60: Unexpected character '→'.
28:112: Unexpected character 'Σ'.
30:49: Unexpected character 'σ'.
30:108: Unexpected character 'σ'.
30:130: Char literals must contain exactly one character and close with '''.
32:119: Unexpected character '≠'.
32:176: Unexpected character '→'.
36:25: Char literals must contain exactly one character and close with '''.
38:206: Unexpected character '→'.
40:141: Char literals must contain exactly one character and close with '''.
42:93: Char literals must contain exactly one character and close with '''.
44:66: Unexpected character '↔'.
44:102: Unexpected character '≠'.
46:76: Unexpected character '≥'.
46:182: Unexpected character '#'.
46:264: Unexpected character 'Σ'.
46:275: Unexpected character '−'.
46:282: Unexpected character '#'.
48:125: Char literals must contain exactly one character and close with '''.
50:85: Unexpected character '−'.
50:91: Unexpected character '≥'.
50:105: Unexpected character '#'.
50:206: Unexpected character '≥'.
50:346: Unexpected character '≠'.
50:563: Char literals must contain exactly one character and close with '''.
52:244: Unexpected character '−'.
54:70: Unexpected character '≥'.
54:176: Unexpected character '−'.
54:204: Unexpected character 'Σ'.
54:288: Unexpected character '⇒'.
54:358: Unexpected character '−'.
54:383: Unexpected character '≥'.
54:442: Unexpected character '—'.
54:471: Unexpected character '−'.
54:624: Unexpected character '−'.
54:736: Unexpected character '−'.
54:743: Unexpected character 'Σ'.
54:748: Unexpected character '−'.
54:763: Unexpected character '−'.
54:766: Unexpected character '✓'.
54:840: Unexpected character '—'.
54:853: Unexpected character '—'.
56:134: Unexpected character '−'.
56:148: Unexpected character '−'.
56:180: Unexpected character '✓'.
58:76: Unexpected character '−'.
58:78: Unexpected character '−'.
58:83: Unexpected character '✓'.
62:88: Unexpected character '−'.
62:90: Unexpected character 'Σ'.
66:9: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 232
OUT2 416 |
| × |
LcbTileDistance2TollLiveCodeBench / atcoder abc359_c / medium: Tile Distance 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TileDistance2Toll case0: expected [5] got [8] |
Failed |
IN9 322
OUT112 |
| × |
LcbWaterTankFillLiveCodeBench / atcoder abc359_e / hard: Water Tank (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WaterTankFill case0: expected [4 5 13 14 26] got [2 3 4 5 6] |
Failed |
IN9 319
OUT241 |
| ✓ |
LcbWelcomeAtCoderLandLiveCodeBench / atcoder abc358_a / easy: Welcome to AtCoder Land (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 168
OUT59 |
Code Generation - LiveCodeBench v3 AtCoder (set 4)
4 passed3 failed |
57.1% |
IN75 691
OUT13 649 |
| × |
LcbGhostAntsPassLiveCodeBench / atcoder abc360_d / medium: Ghost Ants (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GhostAntsPass case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:39: Char literals must contain exactly one character and close with '''.
14:24: Unexpected character '≤'.
16:74: Char literals must contain exactly one character and close with '''.
22:34: Unexpected character '→'.
23:34: Unexpected character '→'.
25:26: Unexpected character '→'.
25:37: Unexpected character '→'.
27:11: Unexpected character '≥'.
27:21: Unexpected character '≥'.
27:187: Unexpected character '≤'.
27:415: Unexpected character '≤'.
29:87: Char literals must contain exactly one character and close with '''.
34:53: Unexpected character '≤'.
36:67: Unexpected character '⟺'.
36:78: Unexpected character '≤'.
36:139: Unexpected character '≤'.
36:258: Unexpected character '≤'.
36:312: Unexpected character '≤'.
36:399: Unexpected character '≤'.
42:246: Unexpected character '≤'.
44:97: Unexpected character '≤'.
44:192: Unexpected character '≤'.
46:140: Unexpected character '→'.
46:155: Unexpected character '→'.
46:176: Unexpected character '≤'.
48:215: Unexpected character '≤'.
52:2: Char literals must contain exactly one character and close with '''.
65:64: Char literals must contain exactly one character and close with '''.
99:67: Unexpected character '≤'.
99:74: Unexpected character '→'.
99:94: Unexpected character '✓'.
101:21: Unexpected character '→'.
101:24: Unexpected character '≤'.
101:31: Unexpected character '→'.
164:5: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 302
OUT2 866 |
| × |
LcbGoStonePuzzleLiveCodeBench / atcoder abc361_d / hard: Go Stone Puzzle (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: GoStonePuzzle case1 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 288
OUT3 176 |
| ✓ |
LcbHealthyBreakfastLiveCodeBench / atcoder abc360_a / easy: A Healthy Breakfast (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 228
OUT61 |
| ✓ |
LcbInsertAfterKLiveCodeBench / atcoder abc361_a / easy: Insert (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 177
OUT149 |
| ✓ |
LcbIntersectCuboidsLiveCodeBench / atcoder abc361_b / medium: Intersection of Cuboids (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 209
OUT396 |
| × |
LcbRandomSwapsBallsLiveCodeBench / atcoder abc360_e / hard: Random Swaps of Balls (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RandomSwapsBalls case0: expected [499122178] got [2] |
Failed |
IN20 260
OUT4 826 |
| ✓ |
LcbVerticalReadingPairLiveCodeBench / atcoder abc360_b / easy: Vertical Reading (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 227
OUT2 175 |
Code Generation - LiveCodeBench v3 AtCoder (set 5)
3 passed3 failed1 skipped |
50.0% |
IN66 328
OUT10 232 |
| ✓ |
LcbBuyAPenLiveCodeBench / atcoder / easy: abc362_a Buy a Pen (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 205
OUT131 |
| × |
LcbCountArithmeticSubsequencesLiveCodeBench / atcoder / hard: abc362_e Count Arithmetic Subsequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountArithmeticSubsequences case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:17: Unexpected character '≥'.
7:90: Char literals must contain exactly one character and close with '''.
15:66: Unexpected character '≤'.
15:132: Unexpected character '≈'.
15:199: Unexpected character '→'.
17:98: Unexpected character '#'.
17:166: Char literals must contain exactly one character and close with '''.
25:54: Unexpected character '≤'.
25:62: Unexpected character '≤'.
29:108: Unexpected character '≥'.
31:392: Char literals must contain exactly one character and close with '''.
33:31: Unexpected character '≥'.
39:122: Char literals must contain exactly one character and close with '''.
39:206: Char literals must contain exactly one character and close with '''.
39:304: Unexpected character '#'.
39:343: Unexpected character '≈'.
42:8: Unexpected character '→'.
43:321: Unexpected character '≈'.
43:335: Unexpected character '≈'.
58:273: Unexpected character '≈'.
58:307: Unexpected character '→'.
58:317: Unexpected character '≈'.
58:487: Unexpected character '≤'.
60:87: Unexpected character '±'.
68:4: Char literals must contain exactly one character and close with '''.
72:15: Unexpected character '#'.
74:9: Unexpected character '#'.
82:44: Unexpected character '—'.
84:19: Unexpected character '→'.
84:49: Char literals must contain exactly one character and close with '''.
90:32: Unexpected character '→'.
91:61: Unexpected character '→'.
94:52: Unexpected character '→'.
103:174: Char literals must contain exactly one character and close with '''.
105:2: Char literals must contain exactly one character and close with '''.
107:19: Unexpected character '≥'.
111:4: Char literals must contain exactly one character and close with '''.
131:4: Char literals must contain exactly one character and close with '''.
133:91: Unexpected character '–'.
145:4: Char literals must contain exactly one character and close with '''.
204:148: Char literals must contain exactly one character and close with '''.
220:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 263
OUT4 137 |
| × |
LcbMakeThemNarrowLiveCodeBench / atcoder / medium: abc361_c Make Them Narrow (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: MakeThemNarrow case1 runtime error: List index 6 is out of range. The list has 6 element(s).
Values:
run.Success = false |
Failed |
IN9 215
OUT893 |
| ✓ |
LcbPerfectPowerCountLiveCodeBench / atcoder / hard: abc361_f x = a^b (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 158
OUT3 010 |
| ✓ |
LcbShortestPath3LiveCodeBench / atcoder / hard: abc362_d Shortest Path 3 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 282
OUT588 |
| – |
LcbSumZeroSequenceLiveCodeBench / atcoder / medium: abc362_c Sum = 0 (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbTreeHamiltonPathLiveCodeBench / atcoder / hard: abc361_e Tree and Hamilton Path 2 (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TreeHamiltonPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
56:26: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN20 205
OUT1 473 |
Code Generation - LiveCodeBench v3 AtCoder (set 6)
3 passed3 failed1 skipped |
50.0% |
IN55 524
OUT7 674 |
| × |
LcbAvoidKPalindrome2LiveCodeBench / atcoder / medium: abc363_c Avoid K Palindrome 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AvoidKPalindrome2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
17:31: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
run.Compiled = false |
Failed |
IN9 271
OUT1 107 |
| ✓ |
LcbCursedDollLiveCodeBench / atcoder / easy: abc363_b Japanese Cursed Doll (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 241
OUT209 |
| × |
LcbNthPalindromeLiveCodeBench / atcoder / medium: abc363_d Palindromic Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NthPalindrome case1: expected [0] got [1] |
Failed |
IN9 181
OUT376 |
| – |
LcbPalindromicExpressionLiveCodeBench / atcoder / hard: abc363_f Palindromic Expression (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbPilingUpLiveCodeBench / atcoder / easy: abc363_a Piling Up (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 261
OUT56 |
| ✓ |
LcbRightTriangleCheckLiveCodeBench / atcoder / easy: abc362_b Right Triangle (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 211
OUT297 |
| × |
LcbSinkingLandLiveCodeBench / atcoder / hard: abc363_e Sinking Land (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SinkingLand case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:16: Assignment target must be a variable or an element like 'items[0]'. Assigning to struct fields is not supported.
Values:
run.Compiled = false |
Failed |
IN9 359
OUT5 629 |
Code Generation - LiveCodeBench v3 AtCoder (set 7)
1 passed5 failed |
16.7% |
IN119 315
OUT17 195 |
| × |
LcbGluttonTakahashiLiveCodeBench / atcoder / easy: Glutton Takahashi (abc364_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GluttonTakahashi case1: expected [Yes] got [No] |
Failed |
IN9 193
OUT88 |
| ✓ |
LcbGridWalkLiveCodeBench / atcoder / easy: Grid Walk (abc364_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 373
OUT403 |
| × |
LcbKthClosestPointLiveCodeBench / atcoder / hard: K — k-th closest point (abc364_d)1 AI query
2 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: KthClosestPoint case0: expected [7
3
13] got [1
3
4] |
Failed |
IN33 242
OUT568 |
| × |
LcbMaximumGluttonLiveCodeBench / atcoder / hard: Maximum Glutton (abc364_e)1 AI query
2 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MaximumGlutton case0: expected [3] got [2] |
Failed |
IN38 062
OUT10 461 |
| × |
LcbMinimumGluttonLiveCodeBench / atcoder / medium: Minimum Glutton (abc364_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimumGlutton case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:6: Char literals must contain exactly one character and close with '''.
13:185: Unexpected character '—'.
13:282: Char literals must contain exactly one character and close with '''.
17:142: Char literals must contain exactly one character and close with '''.
17:184: Char literals must contain exactly one character and close with '''.
25:345: Unexpected character '≤'.
25:399: Char literals must contain exactly one character and close with '''.
25:533: Unexpected character '≤'.
25:695: Unexpected character '≥'.
25:710: Char literals must contain exactly one character and close with '''.
25:768: Char literals must contain exactly one character and close with '''.
27:89: Char literals must contain exactly one character and close with '''.
27:171: Unexpected character '∃'.
27:269: Unexpected character '→'.
29:16: Char literals must contain exactly one character and close with '''.
29:190: Char literals must contain exactly one character and close with '''.
33:369: Char literals must contain exactly one character and close with '''.
33:457: Char literals must contain exactly one character and close with '''.
35:111: Unexpected character '≥'.
91:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 200
OUT1 979 |
| × |
LcbRangeConnectMstLiveCodeBench / atcoder / hard: Range Connect MST (abc364_f)1 AI query
1 tool callcategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RangeConnectMst case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:13: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN20 245
OUT3 696 |
Code Generation - LiveCodeBench v3 AtCoder (set 8)
4 passed1 failed |
80.0% |
IN57 014
OUT11 041 |
| × |
LcbAtcoderJanken3LiveCodeBench / atcoder / medium: AtCoder Janken 3 (abc365_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AtcoderJanken3 case0: expected [5] got [6] |
Failed |
IN9 269
OUT421 |
| ✓ |
LcbLeapYearLiveCodeBench / atcoder / easy: Leap Year (abc365_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 243
OUT95 |
| ✓ |
LcbSecondBestLiveCodeBench / atcoder / easy: Second Best (abc365_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT247 |
| ✓ |
LcbTransportationExpensesLiveCodeBench / atcoder / medium: Transportation Expenses (abc365_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 235
OUT1 675 |
| ✓ |
LcbXorSigmaProblemLiveCodeBench / atcoder / hard: Xor Sigma Problem (abc365_e)1 AI query
1 tool callcategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN20 092
OUT8 603 |
Code Generation - LiveCodeBench v3 LeetCode (set 1)
6 passed2 failed |
75.0% |
IN73 393
OUT2 502 |
| ✓ |
LcbChildWithBallLiveCodeBench / leetcode / easy: find-the-child-who-has-the-ball-after-k-seconds (id 3450)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 181
OUT82 |
| ✓ |
LcbCompleteDayPairsLiveCodeBench / leetcode / easy: count-pairs-that-form-a-complete-day-i (id 3421)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 140
OUT161 |
| ✓ |
LcbCountDaysWithoutMeetingsLiveCodeBench / leetcode / medium: count-days-without-meetings (id 3430)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 198
OUT202 |
| ✓ |
LcbMaxTotalDamageLiveCodeBench / leetcode / medium: maximum-total-damage-with-spell-casting (id 3437)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 160
OUT258 |
| × |
LcbMaxTotalRewardLiveCodeBench / leetcode / medium: maximum-total-reward-using-operations-i (id 3442)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxTotalReward case1: expected 11 got 7 |
Failed |
IN9 174
OUT64 |
| ✓ |
LcbMinimumChairsLiveCodeBench / leetcode / easy: minimum-number-of-chairs-in-a-waiting-room (id 3426)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT90 |
| ✓ |
LcbSubarrayOrClosestKLiveCodeBench / leetcode / hard: find-subarray-with-bitwise-or-closest-to-k (id 3436)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 182
OUT146 |
| × |
LcbValueAfterKSecondsLiveCodeBench / leetcode / medium: find-the-n-th-value-after-k-seconds (id 3422)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ValueAfterKSeconds case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:18: Unexpected character '`'.
3:20: Unexpected character '`'.
3:32: Unexpected character '`'.
3:34: Unexpected character '`'.
3:97: Unexpected character '`'.
3:117: Unexpected character '`'.
3:141: Unexpected character '`'.
3:148: Unexpected character '`'.
3:156: Unexpected character '`'.
3:158: Unexpected character '`'.
5:45: Char literals must contain exactly one character and close with '''.
7:10: Unexpected character '`'.
7:23: Unexpected character '`'.
9:26: Unexpected character '→'.
9:28: Unexpected character '`'.
9:41: Unexpected character '`'.
10:14: Unexpected character '`'.
10:28: Unexpected character '`'.
11:14: Unexpected character '`'.
11:29: Unexpected character '`'.
12:14: Unexpected character '`'.
12:29: Unexpected character '`'.
13:14: Unexpected character '`'.
13:29: Unexpected character '`'.
13:31: Unexpected character '✓'.
13:51: Unexpected character '→'.
15:4: Unexpected character '`'.
15:11: Unexpected character '`'.
17:81: Char literals must contain exactly one character and close with '''.
19:19: Unexpected character '→'.
20:22: Unexpected character '→'.
21:23: Unexpected character '→'.
25:40: Unexpected character '✓'.
26:40: Unexpected character '✓'.
32:19: Char literals must contain exactly one character and close with '''.
34:2: Char literals must contain exactly one character and close with '''.
40:17: Unexpected character '≡'.
44:11: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 195
OUT1 499 |
Code Generation - LiveCodeBench v3 LeetCode (set 2)
4 passed4 failed |
50.0% |
IN73 563
OUT7 027 |
| ✓ |
LcbEncryptedStringLiveCodeBench / leetcode / easy: find-the-encrypted-string (id 3468)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 127
OUT109 |
| × |
LcbMaxTotalCostAlternatingLiveCodeBench / leetcode / medium: maximize-total-cost-of-alternating-subarrays (id 3464)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxTotalCostAlternating case0: expected 10 got 8 |
Failed |
IN9 218
OUT175 |
| × |
LcbMaxTriangleHeightLiveCodeBench / leetcode / easy: maximum-height-of-a-triangle (id 3469)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxTriangleHeight case0: expected 3 got 2 |
Failed |
IN9 184
OUT103 |
| × |
LcbMaxValidSubsequenceLiveCodeBench / leetcode / medium: find-the-maximum-length-of-valid-subsequence-i (id 3490)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxValidSubsequence case1: expected 6 got 7 |
Failed |
IN9 185
OUT203 |
| ✓ |
LcbMinDiameterMergeTreesLiveCodeBench / leetcode / hard: find-minimum-diameter-after-merging-two-trees (id 3439)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 284
OUT2 000 |
| ✓ |
LcbMinimumAreaCoverOnesLiveCodeBench / leetcode / medium: find-the-minimum-area-to-cover-all-ones-i (id 3461)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 130
OUT252 |
| ✓ |
LcbMinimumAverageLiveCodeBench / leetcode / easy: minimum-average-of-smallest-and-largest-elements (id 3471)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 185
OUT135 |
| × |
LcbPeaksInArrayLiveCodeBench / leetcode / hard: peaks-in-array (id 3438)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PeaksInArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
44:16: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 250
OUT4 050 |
Code Generation - LiveCodeBench v3 LeetCode (set 3)
2 passed5 failed1 skipped |
28.6% |
IN64 797
OUT3 801 |
| × |
LcbBitChangesEqualLiveCodeBench / leetcode / easy: 3508 number-of-bit-changes-to-make-two-integers-equal1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minChanges case2: expected -1 got 1 |
Failed |
IN9 223
OUT102 |
| × |
LcbCountEqualXYSubmatricesLiveCodeBench / leetcode / medium: 3492 count-submatrices-with-equal-frequency-of-x-and-y1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfSubmatrices case0: expected 3 got 2 |
Failed |
IN9 238
OUT219 |
| × |
LcbMaxOpsMoveOnesLiveCodeBench / leetcode / medium: 3493 maximum-number-of-operations-to-move-ones-to-the-end1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxOperations case0: expected 4 got 7 |
Failed |
IN9 220
OUT125 |
| × |
LcbMinCostConstructStringLiveCodeBench / leetcode / hard: 3482 construct-string-with-minimum-cost1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minimumCost case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:13: Expected a name, found '='.
Values:
run.Compiled = false |
Failed |
IN9 334
OUT306 |
| × |
LcbMinCostCutCakeLiveCodeBench / leetcode / medium: 3494 minimum-cost-for-cutting-cake-i1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimumCostCake case0: expected 13 got 10 |
Failed |
IN9 350
OUT271 |
| ✓ |
LcbSmallestStringAfterSwapLiveCodeBench / leetcode / easy: 3484 lexicographically-smallest-string-after-a-swap1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 216
OUT274 |
| – |
LcbValidBinaryStringsLiveCodeBench / leetcode / medium: 3453 generate-binary-strings-without-adjacent-zeros0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbVowelsGameLiveCodeBench / leetcode / medium: 3462 vowels-game-in-a-string1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 216
OUT2 504 |
Code Generation - LiveCodeBench v3 LeetCode (set 4)
4 passed4 failed |
50.0% |
IN74 297
OUT3 848 |
| × |
LcbAlternatingGroupsIIILiveCodeBench / leetcode / hard: 3527 alternating-groups-iii1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: numberOfAlternatingGroups case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'numberOfAlternatingGroups' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, NumberOfAlternatingGroupsHelper.
Values:
run.Compiled = false |
Failed |
IN9 403
OUT899 |
| ✓ |
LcbDigitGameWinLiveCodeBench / leetcode / easy: 3515 find-if-digit-game-can-be-won1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 257
OUT136 |
| × |
LcbDominantOnesSubstringsLiveCodeBench / leetcode / medium: 3479 count-the-number-of-substrings-with-dominant-ones1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: numberOfSubstrings case0 runtime error: List index -1 is out of range. The list has 3 element(s).
Values:
run.Success = false |
Failed |
IN9 187
OUT336 |
| × |
LcbMinOpsArrayToTargetLiveCodeBench / leetcode / hard: 3454 minimum-operations-to-make-array-equal-to-target1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumOperations case1: expected 5 got 2 |
Failed |
IN9 216
OUT146 |
| ✓ |
LcbNonSpecialCountLiveCodeBench / leetcode / medium: 3507 find-the-count-of-numbers-which-are-not-special1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 241
OUT1 110 |
| × |
LcbRectangleCornerReachableLiveCodeBench / leetcode / hard: 3478 check-if-the-rectangle-corner-is-reachable1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: canReachCorner case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
28:14: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 392
OUT785 |
| ✓ |
LcbShortestDistanceQueriesLiveCodeBench / leetcode / medium: 3517 shortest-distance-after-road-addition-queries-i1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 333
OUT325 |
| ✓ |
LcbSnakeFinalPositionLiveCodeBench / leetcode / easy: 3533 snake-in-matrix1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 268
OUT111 |
Code Generation - LiveCodeBench v3 LeetCode (set 5)
5 passed3 failed |
62.5% |
IN73 510
OUT9 161 |
| ✓ |
LcbAlternatingGroupsCircleLiveCodeBench / leetcode / easy: alternating-groups-i (LCB id 3463)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 180
OUT139 |
| ✓ |
LcbClearDigitsLiveCodeBench / leetcode / easy: clear-digits (LCB id 3447)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 127
OUT132 |
| × |
LcbCountInversionPermutationsLiveCodeBench / leetcode / hard: count-the-number-of-inversions (LCB id 3460)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: countInversionPermutations case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:19: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed |
IN9 282
OUT7 643 |
| × |
LcbCountMonotonicPairsLiveCodeBench / leetcode / hard: find-the-count-of-monotonic-pairs-i (LCB id 3535)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countMonotonicPairs case0: expected 4 got 5 |
Failed |
IN9 244
OUT395 |
| ✓ |
LcbFirstPlayerWinKGamesLiveCodeBench / leetcode / medium: find-the-first-player-to-win-k-games-in-a-row (LCB id 3413)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 228
OUT335 |
| × |
LcbMaxGoodSubsequenceLengthLiveCodeBench / leetcode / medium: find-the-maximum-length-of-a-good-subsequence-i (LCB id 3456)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxGoodSubsequenceLength case0: expected 4 got 2 |
Failed |
IN9 167
OUT181 |
| ✓ |
LcbMinOpsBinaryToOneLiveCodeBench / leetcode / medium: minimum-operations-to-make-binary-array-elements-equal-to-one-i (LCB id 3475)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 161
OUT244 |
| ✓ |
LcbMinOpsDivisibleThreeLiveCodeBench / leetcode / easy: find-minimum-operations-to-make-all-elements-divisible-by-three (LCB id 3476)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 121
OUT92 |
Code Generation - LiveCodeBench v3 LeetCode (set 6)
3 passed5 failed |
37.5% |
IN73 675
OUT4 800 |
| × |
LcbCoinGameWinnerLiveCodeBench / leetcode / easy: find-the-winning-player-in-coin-game (LCB id 3511)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: coinGameWinner case0: expected "Alice" got "Bob" |
Failed |
IN9 176
OUT103 |
| ✓ |
LcbCountSubarraysAndKLiveCodeBench / leetcode / hard: number-of-subarrays-with-and-value-of-k (LCB id 3466)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 146
OUT125 |
| ✓ |
LcbCountWinningPlayersLiveCodeBench / leetcode / easy: find-the-number-of-winning-players (LCB id 3519)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 278
OUT267 |
| × |
LcbMaxPointsEnemyBattlesLiveCodeBench / leetcode / medium: maximum-points-after-enemy-battles (LCB id 3264)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxPointsEnemyBattles case0 runtime error: Division by zero.
Values:
run.Success = false |
Failed |
IN9 243
OUT235 |
| × |
LcbMinChangesEqualDiffsLiveCodeBench / leetcode / medium: minimum-array-changes-to-make-differences-equal (LCB id 3498)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minChangesEqualDiffs case0: expected 2 got 0 |
Failed |
IN9 197
OUT268 |
| ✓ |
LcbMinFlipsPalindromicGridLiveCodeBench / leetcode / medium: minimum-number-of-flips-to-make-binary-grid-palindromic-i (LCB id 3526)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 192
OUT251 |
| × |
LcbMinLengthAfterOpsLiveCodeBench / leetcode / medium: minimum-length-of-string-after-operations (LCB id 3455)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minLengthAfterOps case0: expected 5 got 9 |
Failed |
IN9 164
OUT177 |
| × |
LcbTimeTakenMarkNodesLiveCodeBench / leetcode / hard: time-taken-to-mark-all-nodes (LCB id 3532)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: timeTakenMarkNodes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:108: Char literals must contain exactly one character and close with '''.
9:278: Char literals must contain exactly one character and close with '''.
13:82: Char literals must contain exactly one character and close with '''.
13:317: Char literals must contain exactly one character and close with '''.
15:152: Char literals must contain exactly one character and close with '''.
17:103: Char literals must contain exactly one character and close with '''.
17:333: Char literals must contain exactly one character and close with '''.
23:4: Char literals must contain exactly one character and close with '''.
25:221: Char literals must contain exactly one character and close with '''.
31:164: Char literals must contain exactly one character and close with '''.
36:23: Char literals must contain exactly one character and close with '''.
38:371: Char literals must contain exactly one character and close with '''.
40:57: Char literals must contain exactly one character and close with '''.
44:17: Unexpected character '#'.
56:59: Unexpected character '—'.
56:64: Char literals must contain exactly one character and close with '''.
62:4: Char literals must contain exactly one character and close with '''.
64:148: Unexpected character '✓'.
66:84: Unexpected character '✓'.
66:150: Unexpected character '✓'.
66:199: Unexpected character '✓'.
78:8: Unexpected character '#'.
90:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 279
OUT3 374 |
Code Generation - LiveCodeBench v4 AtCoder (set 1)
4 passed4 failed1 skipped |
50.0% |
IN73 909
OUT6 585 |
| × |
LcbBallsAndBagQueryLiveCodeBench / atcoder / medium: abc366_c Balls and Bag Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BallsAndBagQuery case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
20:31: The left operand of '??' must be nullable or null, but it has type 'int'.
39:25: Type 'string' has no supported method named 'TrimEnd'.
Values:
run.Compiled = false |
Failed |
IN9 213
OUT264 |
| × |
LcbCuboidSumQueryLiveCodeBench / atcoder / medium: abc366_d Cuboid Sum Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: CuboidSumQuery case0 runtime error: Cannot parse '1 2' as long.
Values:
run.Success = false |
Failed |
IN9 323
OUT365 |
| ✓ |
LcbCutTrailingZerosLiveCodeBench / atcoder / easy: abc367_b Cut .0 (Pattern B, stdin) NOTE (triage): float-formatting output — model must print X with no trailing zeros and no trailing decimal point. Compared as a trimmed string. Flagged for coordinator.1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 205
OUT165 |
| ✓ |
LcbElection2LiveCodeBench / atcoder / easy: abc366_a Election 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 219
OUT114 |
| × |
LcbManhattanEllipseLiveCodeBench / atcoder / hard: abc366_e Manhattan Multifocal Ellipse (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ManhattanEllipse case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:56: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:62: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:70: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:71: Unexpected character '\'.
1:95: Unexpected character '\'.
1:100: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:127: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:141: Unexpected character '\'.
1:146: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:150: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 236
OUT4 694 |
| × |
LcbMaximumCompositionLiveCodeBench / atcoder / hard: abc366_f Maximum Composition (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaximumComposition case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 240
OUT488 |
| ✓ |
LcbShoutEverydayLiveCodeBench / atcoder / easy: abc367_a Shout Everyday (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 200
OUT115 |
| ✓ |
LcbVerticalWritingLiveCodeBench / atcoder / easy: abc366_b Vertical Writing (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 273
OUT380 |
| – |
LcbXorNeighborsLiveCodeBench / atcoder / hard: abc366_g XOR Neighbors (Pattern B, stdin) NOTE (triage): non-unique answer — any valid assignment is accepted, so exact-string comparison against the sample output will reject other correct solutions. Flagged for coordinator.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
Code Generation - LiveCodeBench v4 AtCoder (set 2)
2 passed6 failed |
25.0% |
IN106 991
OUT24 227 |
| × |
LcbAddMultiplyQueriesLiveCodeBench / atcoder / hard: abc368_g Add and Multiply Queries (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AddMultiplyQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:24: Expected ';' after the variable declaration, found ','.
Values:
run.Compiled = false |
Failed |
IN20 291
OUT14 160 |
| × |
LcbCardCutLiveCodeBench / atcoder / easy: abc368_a Cut (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CardCut case0: expected [3 4 5 1 2] got [4 5 1 2 3] |
Failed |
IN9 219
OUT118 |
| ✓ |
LcbDecreaseTwoMaxLiveCodeBench / atcoder / easy: abc368_b Decrease 2 max elements (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 204
OUT285 |
| ✓ |
LcbEnumerateSequencesLiveCodeBench / atcoder / medium: abc367_c Enumerate Sequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 216
OUT406 |
| × |
LcbPedometerLiveCodeBench / atcoder / medium: abc367_d Pedometer (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Pedometer case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:35: Unexpected character '≠'.
5:120: Unexpected character '—'.
9:35: Unexpected character '—'.
9:94: Char literals must contain exactly one character and close with '''.
11:215: Unexpected character '—'.
15:140: Char literals must contain exactly one character and close with '''.
17:2: Char literals must contain exactly one character and close with '''.
19:38: Unexpected character '≤'.
21:14: Unexpected character '≡'.
21:50: Unexpected character '≡'.
21:223: Unexpected character '≡'.
21:645: Unexpected character '≡'.
21:652: Unexpected character '−'.
23:22: Unexpected character '≡'.
23:45: Unexpected character '≠'.
25:98: Unexpected character '≡'.
29:15: Unexpected character 'Σ'.
29:20: Unexpected character '²'.
29:21: Unexpected character '−'.
31:8: Unexpected character '≥'.
39:20: Unexpected character '≤'.
39:62: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 269
OUT1 098 |
| × |
LcbPermuteKTimesLiveCodeBench / atcoder / hard: abc367_e Permute K times (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PermuteKTimes case0: expected [7 2 3 5 1 9 3] got [7 2 3 5 1 9 11] |
Failed |
IN20 199
OUT624 |
| × |
LcbRearrangeQueryLiveCodeBench / atcoder / hard: abc367_f Rearrange Query (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RearrangeQuery case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
56:40: Expected ')' to close the parenthesized expression, found 'L'.
Values:
run.Compiled = false |
Failed |
IN20 295
OUT3 061 |
| × |
LcbSumXorKLiveCodeBench / atcoder / hard: abc367_g Sum of (XOR^K or 0) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SumXorK case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:10: Unexpected character '≡'.
9:136: Char literals must contain exactly one character and close with '''.
11:35: Unexpected character '×'.
11:106: Unexpected character '≈'.
11:137: Unexpected character '≤'.
11:148: Char literals must contain exactly one character and close with '''.
13:32: Char literals must contain exactly one character and close with '''.
17:62: Unexpected character '⊆'.
17:150: Unexpected character '≥'.
19:51: Unexpected character '∈'.
19:97: Unexpected character '≡'.
27:121: Unexpected character '→'.
27:134: Unexpected character '→'.
29:19: Unexpected character '×'.
29:23: Unexpected character '≈'.
29:30: Unexpected character '×'.
29:59: Unexpected character '×'.
33:148: Unexpected character '≡'.
35:32: Unexpected character 'Σ'.
35:44: Unexpected character '·'.
35:131: Unexpected character '≡'.
35:164: Unexpected character '≥'.
35:283: Unexpected character '≥'.
37:13: Unexpected character 'Σ'.
37:33: Unexpected character '·'.
39:368: Unexpected character '→'.
39:474: Unexpected character 'χ'.
39:550: Unexpected character 'ω'.
41:162: Unexpected character '·'.
41:163: Unexpected character 'ω'.
41:181: Unexpected character 'χ'.
41:190: Unexpected character '±'.
41:214: Unexpected character 'ω'.
41:223: Unexpected character 'π'.
41:238: Char literals must contain exactly one character and close with '''.
41:543: Unexpected character '…'.
41:596: Char literals must contain exactly one character and close with '''.
43:68: Unexpected character '≤'.
45:54: Unexpected character '≡'.
45:94: Unexpected character 'Σ'.
45:105: Unexpected character 'χ'.
46:2: Char literals must contain exactly one character and close with '''.
48:47: Unexpected character 'χ'.
50:108: Unexpected character '←'.
50:124: Unexpected character 'χ'.
50:132: Unexpected character 'χ'.
50:134: Unexpected character '±'.
52:33: Unexpected character 'χ'.
52:34: Unexpected character '·'.
54:5: Char literals must contain exactly one character and close with '''.
58:113: Unexpected character '≤'.
58:180: Unexpected character '—'.
58:242: Unexpected character '→'.
58:359: Char literals must contain exactly one character and close with '''.
60:17: Char literals must contain exactly one character and close with '''.
62:7: Unexpected character 'χ'.
62:80: Unexpected character 'χ'.
62:89: Unexpected character 'χ'.
62:328: Unexpected character 'Σ'.
62:332: Unexpected character 'δ'.
62:339: Unexpected character 'χ'.
62:382: Unexpected character 'δ'.
66:38: Unexpected character '·'.
66:88: Unexpected character 'Σ'.
66:111: Unexpected character 'Σ'.
66:146: Unexpected character '·'.
66:198: Unexpected character 'Σ'.
66:221: Unexpected character 'Σ'.
68:19: Unexpected character 'Σ'.
68:37: Unexpected character '·'.
72:67: Unexpected character 'Σ'.
72:131: Unexpected character 'ω'.
72:166: Unexpected character 'ω'.
72:182: Unexpected character 'λ'.
72:196: Unexpected character '∏'.
72:200: Unexpected character 'λ'.
72:201: Unexpected character 'ω'.
72:284: Unexpected character '·'.
72:366: Unexpected character '⟨'.
72:381: Unexpected character '⟩'.
72:390: Unexpected character 'ω'.
72:394: Unexpected character '·'.
72:471: Unexpected character 'ω'.
72:481: Unexpected character 'ω'.
72:516: Unexpected character 'Σ'.
72:525: Unexpected character '·'.
72:529: Unexpected character 'ω'.
72:538: Unexpected character 'ω'.
72:545: Unexpected character '·'.
72:562: Unexpected character 'Σ'.
72:569: Unexpected character 'ω'.
72:579: Unexpected character 'ω'.
74:33: Unexpected character 'Σ'.
74:50: Unexpected character 'ω'.
74:65: Unexpected character 'ω'.
74:84: Unexpected character 'ω'.
74:90: Unexpected character 'π'.
74:158: Unexpected character 'ω'.
74:166: Unexpected character 'ω'.
74:198: Unexpected character 'ζ'.
74:409: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 298
OUT4 475 |
Code Generation - LiveCodeBench v4 AtCoder (set 3)
2 passed7 failed |
22.2% |
IN83 493
OUT7 022 |
| ✓ |
LcbArrangeArithmeticLiveCodeBench / atcoder / abc369_a: 369 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 211
OUT908 |
| × |
LcbAsFarAsPossibleLiveCodeBench / atcoder / abc369_g: As far as possible (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AsFarAsPossible case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:24: Unexpected character '→'.
8:92: Unexpected character '×'.
8:162: Unexpected character '∪'.
10:63: Unexpected character '∪'.
10:113: Unexpected character '∈'.
10:171: Unexpected character '·'.
12:11: Unexpected character 'Σ'.
12:19: Unexpected character '·'.
14:41: Unexpected character '⊆'.
14:144: Unexpected character '—'.
16:224: Unexpected character '≥'.
18:37: Unexpected character '≠'.
18:119: Unexpected character '→'.
20:118: Char literals must contain exactly one character and close with '''.
22:457: Char literals must contain exactly one character and close with '''.
24:235: Char literals must contain exactly one character and close with '''.
26:58: Unexpected character '→'.
26:255: Char literals must contain exactly one character and close with '''.
28:201: Unexpected character '≥'.
28:430: Unexpected character '≥'.
28:452: Char literals must contain exactly one character and close with '''.
30:48: Char literals must contain exactly one character and close with '''.
32:190: Unexpected character '·'.
34:190: Unexpected character '≤'.
34:258: Unexpected character '—'.
34:433: Unexpected character '≤'.
34:442: Char literals must contain exactly one character and close with '''.
46:145: Unexpected character '≥'.
60:31: Unexpected character '≥'.
60:46: Unexpected character '≤'.
Values:
run.Compiled = false |
Failed |
IN9 370
OUT1 363 |
| × |
LcbBonusExpLiveCodeBench / atcoder / abc369_d: Bonus EXP (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BonusExp case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:16: Variable 'parts' has type 'List<long>' but its initializer has type 'List<string>'.
8:31: 'long.Parse' expects a string argument, but it has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 275
OUT296 |
| × |
LcbCountArithmeticSubarraysLiveCodeBench / atcoder / abc369_c: Count Arithmetic Subarrays (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountArithmeticSubarrays case0: expected [8] got [9] |
Failed |
IN9 256
OUT221 |
| × |
LcbDividingGameLiveCodeBench / atcoder / abc368_f: Dividing Game (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DividingGame case1: expected [Bruno] got [Anna] |
Failed |
IN9 260
OUT180 |
| × |
LcbMinimumSteinerTreeLiveCodeBench / atcoder / abc368_d: Minimum Steiner Tree (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimumSteinerTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:18: Expected '(' to declare a function or '=' to declare a global variable after 'Please implement', found 'the'. Global variables must have an initializer.
Values:
run.Compiled = false |
Failed |
IN9 218
OUT43 |
| × |
LcbPiano3LiveCodeBench / atcoder / abc369_b: Piano 3 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Piano3 case0: expected [11] got [20] |
Failed |
IN9 308
OUT173 |
| ✓ |
LcbTrainDelayLiveCodeBench / atcoder / abc368_e: Train Delay (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 342
OUT3 658 |
| × |
LcbTripleAttackLiveCodeBench / atcoder / abc368_c: Triple Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: TripleAttack case0 runtime error: List index 3 is out of range. The list has 3 element(s).
Values:
run.Success = false |
Failed |
IN9 253
OUT180 |
Code Generation - LiveCodeBench v4 AtCoder (set 4)
2 passed6 failed |
25.0% |
IN86 948
OUT39 002 |
| × |
LcbAvoidKPartitionLiveCodeBench / atcoder / abc370_e: Avoid K Partition (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AvoidKPartition case0: expected [2] got [5] |
Failed |
IN9 318
OUT376 |
| ✓ |
LcbBinaryAlchemyLiveCodeBench / atcoder / abc370_b: Binary Alchemy (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 250
OUT224 |
| × |
LcbCakeDivisionLiveCodeBench / atcoder / abc370_f: Cake Division (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CakeDivision case0: expected [13 1] got [24 4] |
Failed |
IN9 369
OUT2 748 |
| × |
LcbCrossExplosionLiveCodeBench / atcoder / abc370_d: Cross Explosion (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CrossExplosion case0: expected [2] got [8] |
Failed |
IN9 346
OUT1 162 |
| × |
LcbDivisibleByThreeLiveCodeBench / atcoder / abc370_g: Divisible by 3 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DivisibleByThree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:76: Unexpected character '≤'.
3:84: Unexpected character 'σ'.
3:89: Unexpected character '≡'.
3:106: Unexpected character 'σ'.
6:1: Unexpected character 'σ'.
6:46: Unexpected character 'σ'.
6:76: Unexpected character 'σ'.
6:81: Unexpected character '≡'.
9:8: Unexpected character '≡'.
9:19: Unexpected character 'σ'.
9:26: Unexpected character '≡'.
9:54: Unexpected character '≢'.
9:73: Unexpected character '≠'.
9:98: Unexpected character 'σ'.
9:102: Unexpected character '≡'.
9:120: Unexpected character 'σ'.
9:141: Unexpected character 'σ'.
9:146: Unexpected character '∏'.
9:147: Unexpected character 'σ'.
9:169: Unexpected character '≡'.
9:195: Unexpected character '≡'.
10:8: Unexpected character '≡'.
10:19: Unexpected character 'σ'.
10:67: Unexpected character '→'.
10:79: Unexpected character '≡'.
10:100: Unexpected character '→'.
10:110: Unexpected character '≡'.
10:133: Char literals must contain exactly one character and close with '''.
11:53: Unexpected character '≡'.
13:49: Unexpected character '→'.
13:81: Unexpected character '≡'.
14:37: Unexpected character '→'.
14:57: Unexpected character '≡'.
16:9: Unexpected character '≡'.
16:19: Unexpected character 'σ'.
16:25: Unexpected character '≡'.
18:6: Unexpected character '≡'.
18:16: Unexpected character 'σ'.
18:22: Unexpected character '≡'.
18:34: Unexpected character '≡'.
18:42: Unexpected character '≡'.
20:10: Unexpected character 'σ'.
20:39: Unexpected character '≡'.
20:47: Unexpected character '≥'.
20:82: Unexpected character '≥'.
20:89: Unexpected character 'σ'.
20:126: Char literals must contain exactly one character and close with '''.
22:4: Unexpected character 'σ'.
22:8: Unexpected character '≡'.
22:17: Unexpected character '⟺'.
22:19: Unexpected character '∃'.
22:40: Unexpected character '≡'.
22:70: Unexpected character '≡'.
22:89: Unexpected character '≡'.
24:49: Unexpected character '≤'.
24:134: Unexpected character '≤'.
24:204: Unexpected character 'Σ'.
24:345: Char literals must contain exactly one character and close with '''.
30:75: Unexpected character 'σ'.
30:92: Unexpected character '≤'.
30:133: Unexpected character '#'.
30:215: Unexpected character 'Σ'.
30:248: Unexpected character '—'.
30:267: Unexpected character 'Σ'.
30:383: Unexpected character 'Σ'.
30:399: Unexpected character '·'.
30:483: Unexpected character '—'.
30:492: Char literals must contain exactly one character and close with '''.
32:15: Unexpected character 'Σ'.
32:28: Unexpected character '≤'.
34:60: Unexpected character 'Σ'.
34:132: Unexpected character '·'.
34:316: Char literals must contain exactly one character and close with '''.
38:2: Char literals must contain exactly one character and close with '''.
39:41: Unexpected character 'σ'.
41:77: Unexpected character 'Σ'.
42:13: Unexpected character 'Σ'.
52:2: Char literals must contain exactly one character and close with '''.
75:51: Unexpected character '≤'.
87:4: Char literals must contain exactly one character and close with '''.
95:55: Unexpected character '≈'.
111:4: Char literals must contain exactly one character and close with '''.
113:82: Unexpected character 'Σ'.
131:57: Char literals must contain exactly one character and close with '''.
145:4: Char literals must contain exactly one character and close with '''.
153:18: Unexpected character '→'.
173:4: Char literals must contain exactly one character and close with '''.
248:48: Unexpected character '≥'.
256:4: Char literals must contain exactly one character and close with '''.
258:32: Unexpected character 'σ'.
258:57: Unexpected character 'σ'.
262:25: Unexpected character '→'.
262:46: Unexpected character 'σ'.
264:136: Unexpected character 'σ'.
268:41: Unexpected character 'σ'.
286:4: Char literals must contain exactly one character and close with '''.
318:4: Char literals must contain exactly one character and close with '''.
330:4: Char literals must contain exactly one character and close with ... (message truncated) |
Failed |
IN9 194
OUT23 436 |
| ✓ |
LcbRaiseBothHandsLiveCodeBench / atcoder / abc370_a: Raise Both Hands (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 265
OUT112 |
| × |
LcbSightseeingTourLiveCodeBench / atcoder / abc369_e: Sightseeing Tour (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SightseeingTour case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
179:19: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed |
IN21 931
OUT10 704 |
| × |
LcbWordLadderLiveCodeBench / atcoder / abc370_c: Word Ladder (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WordLadder case0: expected [3
acbe
acbc
bcbc] got [3
bdbe
bcbe
bcbc
] |
Failed |
IN9 275
OUT240 |
Code Generation - LiveCodeBench v4 AtCoder (set 5)
3 passed6 failed |
33.3% |
IN83 455
OUT7 157 |
| × |
LcbCountABCAgainLiveCodeBench / atcoder / medium: abc372_c Count ABC Again (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountABCAgain case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:25: Type 'string' has no supported method named 'TrimEnd'.
Values:
run.Compiled = false |
Failed |
IN9 241
OUT448 |
| × |
LcbDistinctSubarraySumLiveCodeBench / atcoder / hard: abc371_e I Hate Sigma Problems (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DistinctSubarraySum case0: expected [8] got [6] |
Failed |
IN9 209
OUT235 |
| × |
LcbKthLargestConnectedLiveCodeBench / atcoder / hard: abc372_e K (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: KthLargestConnected case0: expected [2
1
-1
4
2
-1] got [1
2
-1
1
3
-1] |
Failed |
IN9 279
OUT475 |
| × |
LcbLexSmallestPermLiveCodeBench / atcoder / hard: abc371_g Lexicographically Smallest Permutation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: LexSmallestPerm case0 runtime error: List index 6 is out of range. The list has 6 element(s).
Values:
run.Success = false |
Failed |
IN9 248
OUT549 |
| × |
LcbMakeIsomorphicLiveCodeBench / atcoder / medium: abc371_c Make Isomorphic (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MakeIsomorphic case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:16: Variable 'parts' has type 'string' but its initializer has type 'List<string>'.
Values:
run.Compiled = false |
Failed |
IN9 402
OUT253 |
| ✓ |
LcbMiddleBrotherLiveCodeBench / atcoder / easy: abc371_a Jiro (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 228
OUT185 |
| × |
LcbNarrowRoadLiveCodeBench / atcoder / hard: abc371_f Takahashi in Narrow Road (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: NarrowRoad case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:119: Char literals must contain exactly one character and close with '''.
7:53: Char literals must contain exactly one character and close with '''.
11:180: Unexpected character '±'.
13:228: Unexpected character '±'.
17:235: Char literals must contain exactly one character and close with '''.
23:72: Char literals must contain exactly one character and close with '''.
33:110: Unexpected character '±'.
37:94: Unexpected character '—'.
39:130: Char literals must contain exactly one character and close with '''.
43:116: Char literals must contain exactly one character and close with '''.
49:174: Unexpected character '—'.
51:156: Unexpected character '≠'.
55:378: Unexpected character '#'.
55:405: Char literals must contain exactly one character and close with '''.
59:209: Unexpected character '—'.
61:82: Unexpected character 'Σ'.
61:100: Char literals must contain exactly one character and close with '''.
63:43: Unexpected character '≤'.
63:178: Unexpected character '≠'.
63:219: Unexpected character '→'.
65:40: Unexpected character '≥'.
67:159: Unexpected character '#'.
67:174: Unexpected character '≤'.
67:194: Unexpected character '#'.
67:202: Unexpected character '≤'.
73:80: Unexpected character '—'.
75:676: Char literals must contain exactly one character and close with '''.
77:39: Unexpected character '≥'.
77:49: Unexpected character '#'.
77:109: Unexpected character '≥'.
77:116: Unexpected character '≤'.
77:186: Unexpected character '≥'.
79:91: Unexpected character '≥'.
79:394: Char literals must contain exactly one character and close with '''.
81:58: Unexpected character '—'.
85:4: Char literals must contain exactly one character and close with '''.
95:41: Unexpected character '→'.
95:55: Unexpected character '≥'.
95:65: Unexpected character '≤'.
95:119: Unexpected character '≥'.
95:151: Unexpected character '≤'.
96:19: Unexpected character '→'.
96:22: Unexpected character '≥'.
96:30: Unexpected character '≤'.
96:73: Unexpected character '≤'.
103:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 289
OUT4 229 |
| ✓ |
LcbOneDCountryLiveCodeBench / atcoder / medium: abc371_d 1D Country (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 281
OUT555 |
| ✓ |
LcbTaroNameLiveCodeBench / atcoder / easy: abc371_b Taro (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 278
OUT228 |
Code Generation - LiveCodeBench v4 AtCoder (set 6)
4 passed3 failed1 skipped |
57.1% |
IN88 081
OUT17 231 |
| × |
LcbAxByLessCLiveCodeBench / atcoder / hard: abc372_g Ax + By < C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AxByLessC case0: expected [2
0] got [1
0] |
Failed |
IN9 304
OUT380 |
| ✓ |
LcbDeleteDotsLiveCodeBench / atcoder / easy: abc372_a delete . (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 153
OUT62 |
| × |
LcbDiminishingKnapsackLiveCodeBench / atcoder / hard: abc373_f Knapsack with Diminishing Values (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DiminishingKnapsack case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:54: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:58: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:70: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:74: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:85: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:93: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:120: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:124: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:158: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:163: Unexpected character '\'.
3:180: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:230: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:231: Unexpected character '\'.
3:240: Unexpected character '\'.
3:250: Unexpected character '\'.
3:256: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:264: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:269: Unexpected character '\'.
3:275: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:45: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:53: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:92: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:96: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:131: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:137: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
6:1: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
6:2: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
6:3: Unexpected character '\'.
6:62: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
6:63: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:39: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:67: Unexpected character '\'.
8:73: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:127: Unexpected character '—'.
10:156: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:160: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:211: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:215: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:248: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:261: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
12:121: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
12:125: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
12:158: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
12:171: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:45: Char literals must contain exactly one character and close with '''.
14:186: Char literals must contain exactly one character and close with '''.
16:54: Char literals must contain exactly one character and close with '''.
18:64: Unexpected character '≤'.
18:68: Unexpected character '≤'.
18:99: Unexpected char ... (message truncated) |
Failed |
IN9 245
OUT3 613 |
| – |
LcbNoCrossMatchingLiveCodeBench / atcoder / hard: abc373_g No Cross Matching (Pattern B, stdin) NOTE: non-unique answer — any non-crossing perfect matching R is valid; the exact-match oracle may reject a different valid R.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbOneDKeyboardLiveCodeBench / atcoder / easy: abc373_b 1D Keyboard (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 246
OUT149 |
| ✓ |
LcbSeptemberLengthsLiveCodeBench / atcoder / easy: abc373_a September (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 183
OUT74 |
| × |
LcbTeleportingWalksLiveCodeBench / atcoder / hard: abc372_f Teleporting Takahashi 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TeleportingWalks case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
4:38: Unexpected character '→'.
6:97: Unexpected character '→'.
8:72: Unexpected character '≤'.
12:91: Unexpected character '≈'.
14:78: Unexpected character '→'.
18:15: Unexpected character '—'.
18:83: Char literals must contain exactly one character and close with '''.
27:25: Unexpected character 'Σ'.
33:111: Unexpected character '→'.
35:60: Unexpected character '∈'.
37:40: Unexpected character 'Σ'.
39:20: Char literals must contain exactly one character and close with '''.
43:9: Unexpected character '≤'.
45:79: Unexpected character '≤'.
45:84: Unexpected character '≤'.
47:20: Unexpected character '∉'.
49:110: Unexpected character '∈'.
51:150: Char literals must contain exactly one character and close with '''.
55:325: Unexpected character '→'.
57:14: Char literals must contain exactly one character and close with '''.
59:296: Char literals must contain exactly one character and close with '''.
59:320: Char literals must contain exactly one character and close with '''.
59:385: Char literals must contain exactly one character and close with '''.
59:546: Unexpected character '→'.
61:55: Unexpected character '≤'.
61:76: Unexpected character '≤'.
61:253: Unexpected character '≈'.
65:95: Unexpected character '—'.
65:101: Char literals must contain exactly one character and close with '''.
65:440: Char literals must contain exactly one character and close with '''.
79:105: Unexpected character '→'.
85:62: Unexpected character '→'.
85:66: Unexpected character '→'.
85:175: Unexpected character '✓'.
85:238: Unexpected character '→'.
85:240: Unexpected character '→'.
85:269: Unexpected character '✓'.
85:347: Unexpected character '✓'.
85:357: Unexpected character '✓'.
87:39: Unexpected character '→'.
87:57: Unexpected character '≠'.
87:72: Unexpected character '→'.
87:107: Unexpected character '→'.
87:163: Unexpected character '→'.
87:206: Unexpected character '→'.
87:221: Unexpected character '→'.
87:224: Unexpected character '⇒'.
87:382: Unexpected character '✗'.
87:472: Unexpected character '→'.
87:527: Char literals must contain exactly one character and close with '''.
87:1346: Char literals must contain exactly one character and close with '''.
87:1377: Char literals must contain exactly one character and close with '''.
87:1490: Char literals must contain exactly one character and close with '''.
87:1678: Unexpected character 'Σ'.
89:76: Unexpected character 'Σ'.
89:217: Unexpected character '→'.
89:242: Unexpected character '→'.
89:308: Unexpected character '→'.
89:382: Unexpected character '→'.
89:403: Unexpected character '✓'.
89:430: Unexpected character '→'.
89:446: Unexpected character '✓'.
89:459: Unexpected character '→'.
89:475: Unexpected character '✓'.
89:488: Unexpected character '✓'.
97:22: Unexpected character '#'.
111:8: Unexpected character '≥'.
117:224: Unexpected character '→'.
117:316: Unexpected character '→'.
117:405: Unexpected character '✓'.
159:183: Unexpected character '✓'.
279:5: Char literals must contain exactly one character and close with '''.
317:20: Unexpected character '`'.
317:31: Unexpected character '`'.
319:6: Unexpected character '`'.
319:15: Unexpected character '`'.
349:13: Unexpected character '`'.
349:28: Unexpected character '`'.
351:11: Unexpected character '`'.
351:34: Unexpected character '`'.
359:6: Unexpected character '`'.
359:22: Unexpected character '`'.
381:87: Char literals must contain exactly one character and close with '''.
389:96: Unexpected character '→'.
413:7: Char literals must contain exactly one character and close with '''.
475:5: Char literals must contain exactly one character and close with '''.
479:44: Unexpected character '→'.
479:148: Unexpected character '✓'.
481:212: Unexpected character '→'.
481:274: Unexpected character '→'.
481:324: Char literals must contain exactly one character and close with '''.
483: ... (message truncated) |
Failed |
IN9 364
OUT9 319 |
| ✓ |
LcbVisibleBuildingsLiveCodeBench / atcoder / medium: abc372_d Buildings (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN32 586
OUT3 634 |
Code Generation - LiveCodeBench v4 AtCoder (set 7)
1 passed5 failed1 skipped |
16.7% |
IN80 493
OUT9 530 |
| × |
LcbAnnoyingStringLiveCodeBench / atcoder / hard: arc181_b Annoying String Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AnnoyingString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:86: Char literals must contain exactly one character and close with '''.
29:155: Unexpected character '—'.
37:2: Char literals must contain exactly one character and close with '''.
37:284: Char literals must contain exactly one character and close with '''.
37:300: Char literals must contain exactly one character and close with '''.
41:90: Char literals must contain exactly one character and close with '''.
45:41: Char literals must contain exactly one character and close with '''.
49:2: Char literals must contain exactly one character and close with '''.
63:4: Char literals must contain exactly one character and close with '''.
65:16: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 263
OUT2 846 |
| × |
LcbIncDecAgainLiveCodeBench / atcoder / hard: arc182_d Increment Decrement Again (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: IncDecAgain case0: expected [3] got [2] |
Failed |
IN9 308
OUT496 |
| ✓ |
LcbMaxAiBjLiveCodeBench / atcoder / medium: abc373_c Max Ai+Bj (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 174
OUT179 |
| × |
LcbPrefixBubbleSortLiveCodeBench / atcoder / hard: arc181_d Prefix Bubble Sort (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PrefixBubbleSort case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 294
OUT626 |
| – |
LcbRowColumnOrderLiveCodeBench / atcoder / hard: arc181_c Row and Column Order (Pattern B, stdin) TRIAGE: non-unique answer (any valid grid accepted) — exact string compare may fail correct solutions.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbSortLeftRightLiveCodeBench / atcoder / medium: arc181_a Sort Left and Right (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SortLeftRight case0: expected [1
0
2] got [2
0
3] |
Failed |
IN22 740
OUT3 910 |
| × |
LcbWinElectionLiveCodeBench / atcoder / hard: abc373_e How to Win the Election (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WinElection case0: expected [2 -1 1 -1 0] got [-1 0 -1 0 -1] |
Failed |
IN20 714
OUT1 473 |
Code Generation - LiveCodeBench v4 AtCoder (set 8)
1 passed5 failed1 skipped |
16.7% |
IN68 833
OUT17 635 |
| × |
LcbChmaxRushLiveCodeBench / atcoder / hard: arc182_a Chmax Rush! (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: ChmaxRush case0 runtime error: List index 0 is out of range. The list has 0 element(s).
Values:
run.Success = false |
Failed |
IN9 360
OUT3 162 |
| × |
LcbDivisorsOfProductLiveCodeBench / atcoder / hard: arc182_c Sum of Number of Divisors of Product (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DivisorsOfProduct case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:27: Expected '>' to close the List type, found '<'.
Values:
run.Compiled = false |
Failed |
IN9 205
OUT5 723 |
| – |
LcbKeepMatchedLiveCodeBench / atcoder / hard: arc183_d Keep Perfectly Matched (Pattern B, stdin) TRIAGE: non-unique answer (any maximizing procedure accepted) — exact string compare may fail correct solutions.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbMedianGoodSeqLiveCodeBench / atcoder / medium: arc183_a Median of Good Sequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: MedianGoodSeq case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 177
OUT1 174 |
| × |
LcbNearAssignmentLiveCodeBench / atcoder / hard: arc183_b Near Assignment (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NearAssignment case0: expected [Yes
Yes
No
Yes] got [Yes
Yes
Yes
Yes] |
Failed |
IN9 242
OUT1 365 |
| × |
LcbNotArgmaxLiveCodeBench / atcoder / hard: arc183_c Not Argmax (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: NotArgmax case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:33: Expected '(' after the List type. Lists are created with 'new List<T>()' or 'new List<T>(items)', found '{'.
Values:
run.Compiled = false |
Failed |
IN9 227
OUT3 032 |
| ✓ |
LcbSumMinModLinearLiveCodeBench / atcoder / hard: arc182_e Sum of Min of Mod of Linear (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN22 622
OUT3 179 |
Code Generation - LiveCodeBench v4 LeetCode (set 1)
2 passed4 failed |
33.3% |
IN55 193
OUT3 440 |
| × |
LcbAlmostEqualPairsLiveCodeBench / leetcode / medium: count-almost-equal-pairs-i (id 3534)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: almostequal case0: expected 2 got 1 |
Failed |
IN9 200
OUT442 |
| × |
LcbBeautifulArraySumLiveCodeBench / leetcode / medium: find-the-minimum-possible-sum-of-a-beautiful-array (id 3026)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: beautiful case0: expected 4 got 3 |
Failed |
IN9 189
OUT239 |
| ✓ |
LcbFinalArrayStateLiveCodeBench / leetcode / easy: final-array-state-after-k-multiplication-operations-i (id 3555)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 212
OUT149 |
| × |
LcbKConstraintSubstringsLiveCodeBench / leetcode / easy: count-substrings-that-satisfy-k-constraint-i (id 3543)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: kconstraint case0: expected 12 got 11 |
Failed |
IN9 193
OUT263 |
| × |
LcbLargestKPalindromeLiveCodeBench / leetcode / hard: find-the-largest-palindrome-divisible-by-k (id 3552)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: kpalindrome case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
185:41: Expected ';' after the for condition, found end of file.
Values:
run.Compiled = false |
Failed |
IN9 164
OUT2 138 |
| ✓ |
LcbMaxEnergyBoostLiveCodeBench / leetcode / medium: maximum-energy-boost-from-two-drinks (id 3525)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 235
OUT209 |
Code Generation - LiveCodeBench v4 LeetCode (set 2)
2 passed4 failed |
33.3% |
IN55 280
OUT1 475 |
| × |
LcbConvertDateBinaryLiveCodeBench / leetcode / easy: convert-date-to-binary (id 3567)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: datebinary case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:7: Type 'StringBuilder' has no supported method named 'Reverse' with 0 argument(s). Supported methods: Append, AppendLine, Clear, ToString.
Values:
run.Compiled = false |
Failed |
IN9 183
OUT173 |
| ✓ |
LcbKthNearestObstacleLiveCodeBench / leetcode / medium: k-th-nearest-obstacle-queries (id 3495)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 252
OUT143 |
| × |
LcbMaxCellScoreLiveCodeBench / leetcode / hard: select-cells-in-grid-with-maximum-score (id 3563)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxcell case0: expected 8 got 4 |
Failed |
IN9 180
OUT358 |
| × |
LcbMaxRangeScoreLiveCodeBench / leetcode / medium: maximize-score-of-numbers-in-ranges (id 3485)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: rangescore case0: expected 4 got 0 |
Failed |
IN9 175
OUT392 |
| × |
LcbMaxXorSubarrayLiveCodeBench / leetcode / hard: maximum-xor-score-subarray-queries (id 3551)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: maxxor case0: expected [12, 60, 60] got [14,60,63] |
Failed |
IN9 315
OUT229 |
| ✓ |
LcbSameChessboardColorLiveCodeBench / leetcode / easy: check-if-two-chessboard-squares-have-the-same-color (id 3553)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT180 |
Code Generation - LiveCodeBench v4 LeetCode (set 3)
6 failed |
0.0% |
IN55 521
OUT6 459 |
| × |
LcbKnightKillPawnsLiveCodeBench / leetcode / hard: maximum-number-of-moves-to-kill-all-pawns (LCB 3560)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: knightPawns case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:5: Unexpected character '×'.
8:42: Char literals must contain exactly one character and close with '''.
16:18: Unexpected character '×'.
20:119: Char literals must contain exactly one character and close with '''.
28:97: Unexpected character '→'.
28:149: Unexpected character '→'.
38:20: Unexpected character 'σ'.
38:32: Unexpected character 'σ'.
38:40: Unexpected character 'σ'.
38:52: Unexpected character 'σ'.
38:60: Unexpected character 'σ'.
40:75: Unexpected character '≤'.
42:26: Unexpected character '×'.
42:87: Unexpected character '≤'.
42:146: Unexpected character '×'.
42:196: Char literals must contain exactly one character and close with '''.
46:95: Char literals must contain exactly one character and close with '''.
48:122: Unexpected character '≤'.
48:133: Unexpected character '×'.
48:138: Unexpected character '≈'.
52:26: Char literals must contain exactly one character and close with '''.
52:112: Char literals must contain exactly one character and close with '''.
54:17: Unexpected character '→'.
56:43: Char literals must contain exactly one character and close with '''.
60:39: Unexpected character '×'.
60:83: Unexpected character '×'.
60:167: Unexpected character '≤'.
60:197: Unexpected character '→'.
67:55: Char literals must contain exactly one character and close with '''.
69:220: Char literals must contain exactly one character and close with '''.
71:2: Char literals must contain exactly one character and close with '''.
71:124: Char literals must contain exactly one character and close with '''.
71:257: Unexpected character '≈'.
75:42: Char literals must contain exactly one character and close with '''.
75:231: Char literals must contain exactly one character and close with '''.
77:34: Unexpected character '×'.
79:94: Char literals must contain exactly one character and close with '''.
87:8: Char literals must contain exactly one character and close with '''.
89:2: Char literals must contain exactly one character and close with '''.
89:123: Unexpected character '≈'.
120:3: Char literals must contain exactly one character and close with '''.
130:56: Unexpected character '±'.
130:59: Unexpected character '±'.
130:64: Unexpected character '±'.
130:67: Unexpected character '±'.
130:73: Char literals must contain exactly one character and close with '''.
134:11: Unexpected character '→'.
134:22: Unexpected character '→'.
140:65: Unexpected character '#'.
140:98: Char literals must contain exactly one character and close with '''.
140:114: Unexpected character '→'.
140:143: Unexpected character '→'.
140:194: Char literals must contain exactly one character and close with '''.
142:138: Unexpected character '≈'.
142:162: Char literals must contain exactly one character and close with '''.
152:65: Unexpected character '≥'.
158:62: Unexpected character '≤'.
166:9: Unexpected character '#'.
261:60: Unexpected character '→'.
267:77: Unexpected character '#'.
267:173: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 345
OUT4 906 |
| × |
LcbMaxMultiplicationScoreLiveCodeBench / leetcode / medium: maximum-multiplication-score (LCB 3518)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxMul case0 runtime error: List index 7 is out of range. The list has 7 element(s).
Values:
run.Success = false |
Failed |
IN9 290
OUT339 |
| × |
LcbMinValidStringsLiveCodeBench / leetcode / medium: minimum-number-of-valid-strings-to-form-target-i (LCB 3559)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minValid case0: expected 3 got -1 |
Failed |
IN9 187
OUT232 |
| × |
LcbPowerKSizeSubarraysLiveCodeBench / leetcode / medium: find-the-power-of-k-size-subarrays-i (LCB 3522)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: powerK case0: expected [3, 4, -1, -1, -1] got [-1,-1,-1,-1,-1] |
Failed |
IN9 255
OUT205 |
| × |
LcbReachEndMaxScoreLiveCodeBench / leetcode / medium: reach-end-of-array-with-max-score (LCB 3528)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: reachEnd case0: expected 7 got 3 |
Failed |
IN9 211
OUT166 |
| × |
LcbThreeRooksMaxSumLiveCodeBench / leetcode / hard: maximum-value-sum-by-placing-three-rooks-i (LCB 3550)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: threeRooks case0: expected 4 got 8 |
Failed |
IN9 233
OUT611 |
Code Generation - LiveCodeBench v4 LeetCode (set 4)
1 passed5 failed |
16.7% |
IN55 672
OUT7 323 |
| × |
LcbCountGoodIntegersLiveCodeBench / leetcode / hard: find-the-count-of-good-integers (LCB 3548)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: goodInt case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:17: Char literals must contain exactly one character and close with '''.
7:56: Char literals must contain exactly one character and close with '''.
10:21: Char literals must contain exactly one character and close with '''.
12:156: Unexpected character '→'.
12:184: Unexpected character 'Σ'.
12:221: Unexpected character '#'.
15:51: Unexpected character 'Σ'.
15:65: Unexpected character '≤'.
15:90: Char literals must contain exactly one character and close with '''.
19:19: Unexpected character '#'.
19:71: Unexpected character '∏'.
19:77: Unexpected character '−'.
19:81: Unexpected character '−'.
19:91: Unexpected character '−'.
19:95: Unexpected character '·'.
19:96: Unexpected character '∏'.
19:100: Unexpected character '≥'.
19:116: Unexpected character '≥'.
19:128: Unexpected character '∏'.
23:18: Char literals must contain exactly one character and close with '''.
23:374: Unexpected character '—'.
25:137: Unexpected character '⌈'.
25:141: Unexpected character '⌉'.
29:157: Unexpected character '—'.
29:233: Char literals must contain exactly one character and close with '''.
31:71: Unexpected character '⇒'.
35:80: Unexpected character '∃'.
35:158: Unexpected character '∏'.
35:165: Unexpected character '×'.
37:228: Unexpected character '≡'.
41:193: Unexpected character '≤'.
41:237: Unexpected character '≤'.
41:272: Unexpected character '→'.
43:2: Char literals must contain exactly one character and close with '''.
49:67: Unexpected character '—'.
49:83: Unexpected character '≤'.
55:228: Unexpected character '—'.
55:324: Char literals must contain exactly one character and close with '''.
57:104: Char literals must contain exactly one character and close with '''.
59:41: Unexpected character '≠'.
59:84: Unexpected character '≤'.
59:86: Unexpected character '−'.
59:166: Unexpected character '→'.
71:35: Unexpected character 'Π'.
71:95: Unexpected character 'Π'.
71:99: Unexpected character '≥'.
71:135: Unexpected character '−'.
75:101: Unexpected character '≥'.
81:67: Unexpected character '→'.
125:64: Unexpected character '→'.
125:189: Unexpected character '≤'.
129:25: Unexpected character '→'.
155:83: Unexpected character '→'.
165:57: Unexpected character '≠'.
181:4: Char literals must contain exactly one character and close with '''.
195:179: Unexpected character '→'.
199:42: Char literals must contain exactly one character and close with '''.
218:25: Unexpected character '→'.
237:4: Char literals must contain exactly one character and close with '''.
243:64: Unexpected character '≤'.
337:64: Unexpected character '≥'.
337:78: Unexpected character '→'.
337:145: Unexpected character '→'.
337:193: Unexpected character '→'.
337:247: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 272
OUT5 696 |
| × |
LcbGenerateKeyNumbersLiveCodeBench / leetcode / easy: find-the-key-of-the-numbers (LCB 3568)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: genKey case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:32: Type 'string' has no supported method named 'PadLeft'.
4:32: Type 'string' has no supported method named 'PadLeft'.
5:32: Type 'string' has no supported method named 'PadLeft'.
12:42: 'Min' expects numeric arguments, but got 'char' and 'char'.
15:31: Type 'string' has no supported method named 'TrimStart'.
Values:
run.Compiled = false |
Failed |
IN9 296
OUT179 |
| ✓ |
LcbHashDividedStringLiveCodeBench / leetcode / medium: hash-divided-string (LCB 3540)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 273
OUT131 |
| × |
LcbMaxSequenceValueLiveCodeBench / leetcode / hard: find-the-maximum-sequence-value-of-array (LCB 3575)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSeq case0: expected 5 got 0 |
Failed |
IN9 236
OUT459 |
| × |
LcbMinDamageToBobLiveCodeBench / leetcode / hard: minimum-amount-of-damage-dealt-to-bob (LCB 3531)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minDmg case1: expected 20 got 30 |
Failed |
IN9 271
OUT267 |
| × |
LcbSafeWalkGridLiveCodeBench / leetcode / medium: find-a-safe-walk-through-a-grid (LCB 3558)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: safeWalk case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:17: Expected ';' after the variable declaration, found '='.
Values:
run.Compiled = false |
Failed |
IN9 324
OUT591 |
Code Generation - LiveCodeBench v4 LeetCode (set 5)
3 passed3 failed |
50.0% |
IN55 287
OUT3 977 |
| ✓ |
LcbKthCharacterGameLiveCodeBench 3600 / leetcode / easy: find the k-th character in string game I (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 208
OUT2 531 |
| × |
LcbLongestIncreasingPathLiveCodeBench 3571 / leetcode / hard: length of the longest increasing path (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestIncreasingPath case0: expected 3 got 2 |
Failed |
IN9 281
OUT153 |
| ✓ |
LcbMountainHeightZeroLiveCodeBench 3496 / leetcode / medium: minimum number of seconds to make mountain height zero (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 281
OUT352 |
| ✓ |
LcbReportSpamLiveCodeBench 3541 / leetcode / medium: report spam message (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 164
OUT100 |
| × |
LcbSubstringsVowelsKConsonantsLiveCodeBench 3570 / leetcode / medium: count of substrings containing every vowel and k consonants I (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: substringsVowelsKConsonants case1: expected 1 got 0 |
Failed |
IN9 168
OUT542 |
| × |
LcbValidSubstringCountLiveCodeBench 3573 / leetcode / medium: count substrings that can be rearranged to contain a string I (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: validSubstringCount case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 185
OUT299 |
Code Generation - LiveCodeBench v4 LeetCode (set 6)
3 passed3 failed |
50.0% |
IN55 312
OUT4 227 |
| ✓ |
LcbFirstAlmostEqualSubstringLiveCodeBench 3580 / leetcode / hard: find the occurrence of first almost equal substring (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 200
OUT161 |
| × |
LcbLexSmallestValidSequenceLiveCodeBench 3584 / leetcode / medium: find the lexicographically smallest valid sequence (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: lexSmallestValidSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:9: Char literals must contain exactly one character and close with '''.
12:14: Char literals must contain exactly one character and close with '''.
22:36: Char literals must contain exactly one character and close with '''.
26:183: Char literals must contain exactly one character and close with '''.
28:108: Char literals must contain exactly one character and close with '''.
30:144: Unexpected character '≤'.
30:197: Unexpected character '≤'.
30:284: Unexpected character '≤'.
32:66: Char literals must contain exactly one character and close with '''.
34:5: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 281
OUT1 055 |
| × |
LcbMaxBinaryConcatenationLiveCodeBench 3579 / leetcode / medium: maximum possible number by binary concatenation (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxBinaryConcatenation case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:17: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed |
IN9 168
OUT144 |
| × |
LcbMaxUniqueTowerHeightsLiveCodeBench 3510 / leetcode / medium: maximize the total height of unique towers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxUniqueTowerHeights case1: expected 25 got 3 |
Failed |
IN9 205
OUT175 |
| ✓ |
LcbMinAfterDigitSumLiveCodeBench 3606 / leetcode / easy: minimum element after replacement with digit sum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 161
OUT103 |
| ✓ |
LcbSortedGcdPairQueriesLiveCodeBench 3583 / leetcode / hard: sorted GCD pair queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 297
OUT2 589 |
Code Generation - LiveCodeBench v5 AtCoder (set 1)
6 passed4 failed2 skipped |
60.0% |
IN92 223
OUT16 915 |
| ✓ |
LcbAbaPalindromeLiveCodeBench / atcoder / medium: abc375_d ABA (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 191
OUT909 |
| – |
LcbLaserMarkingLiveCodeBench / atcoder / medium: abc374_d Laser Marking (Pattern B, stdin) [float output, tolerance 1e-6]0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped |
IN0
OUT0 |
| × |
LcbOnlyOneProductNameLiveCodeBench / atcoder / hard: abc374_g Only One Product Name (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: OnlyOneProductName case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
44:58: Expected ')' to close the argument list, found 'idx'.
Values:
run.Compiled = false |
Failed |
IN9 229
OUT967 |
| ✓ |
LcbSeatsLiveCodeBench / atcoder / easy: abc375_a Seats (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 184
OUT109 |
| ✓ |
LcbSensorOptimizationLiveCodeBench / atcoder / hard: abc374_e Sensor Optimization Dilemma 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 280
OUT3 737 |
| ✓ |
LcbSeparatedLunchLiveCodeBench / atcoder / medium: abc374_c Separated Lunch (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 201
OUT1 703 |
| × |
LcbShippingLiveCodeBench / atcoder / hard: abc374_f Shipping (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Shipping case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:42: Unexpected character '≤'.
3:48: Unexpected character '≤'.
3:54: Unexpected character '≤'.
3:142: Char literals must contain exactly one character and close with '''.
5:77: Char literals must contain exactly one character and close with '''.
5:246: Unexpected character '≤'.
7:45: Unexpected character '≤'.
7:73: Unexpected character '≥'.
9:70: Unexpected character '≥'.
11:91: Unexpected character '≤'.
11:168: Char literals must contain exactly one character and close with '''.
21:44: Unexpected character '≤'.
23:24: Char literals must contain exactly one character and close with '''.
25:214: Unexpected character '≥'.
31:272: Unexpected character '→'.
33:89: Char literals must contain exactly one character and close with '''.
35:43: Unexpected character '≥'.
35:57: Unexpected character '≥'.
35:70: Unexpected character '≥'.
35:96: Unexpected character '≥'.
39:165: Char literals must contain exactly one character and close with '''.
43:68: Char literals must contain exactly one character and close with '''.
45:16: Char literals must contain exactly one character and close with '''.
49:8: Unexpected character 'Σ'.
49:29: Unexpected character '·'.
49:34: Unexpected character '−'.
49:50: Unexpected character '−'.
53:32: Unexpected character '≥'.
53:52: Unexpected character '≥'.
53:78: Unexpected character '≥'.
53:90: Unexpected character '≤'.
53:112: Unexpected character '·'.
53:136: Unexpected character '≤'.
53:150: Unexpected character '−'.
53:169: Unexpected character '−'.
53:170: Unexpected character '∞'.
53:198: Char literals must contain exactly one character and close with '''.
55:16: Unexpected character '≤'.
55:33: Unexpected character '−'.
55:36: Unexpected character '·'.
57:41: Unexpected character '—'.
59:189: Unexpected character '≤'.
63:40: Unexpected character '’'.
63:91: Unexpected character '−'.
63:93: Unexpected character '≤'.
63:109: Unexpected character '−'.
63:125: Unexpected character '−'.
63:135: Unexpected character '−'.
65:5: Unexpected character '—'.
65:325: Char literals must contain exactly one character and close with '''.
69:168: Unexpected character '−'.
71:65: Char literals must contain exactly one character and close with '''.
73:84: Unexpected character '≥'.
75:259: Char literals must contain exactly one character and close with '''.
77:114: Unexpected character '→'.
79:169: Unexpected character '−'.
79:175: Unexpected character '−'.
79:184: Unexpected character '−'.
79:484: Char literals must contain exactly one character and close with '''.
81:357: Unexpected character '≤'.
81:367: Unexpected character '≤'.
85:376: Char literals must contain exactly one character and close with '''.
93:25: Unexpected character '≤'.
93:41: Unexpected character '≤'.
93:44: Unexpected character '≤'.
93:70: Unexpected character '≤'.
93:76: Unexpected character '→'.
95:9: Unexpected character '#'.
111:72: Unexpected character '≈'.
121:11: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 283
OUT3 256 |
| × |
LcbSpiralRotationLiveCodeBench / atcoder / medium: abc375_c Spiral Rotation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SpiralRotation case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:13: Unexpected character '×'.
4:29: Unexpected character '≤'.
4:35: Unexpected character '≤'.
4:40: Unexpected character '−'.
4:77: Unexpected character '−'.
6:63: Unexpected character '−'.
6:69: Unexpected character '−'.
6:146: Unexpected character '−'.
8:13: Unexpected character '→'.
8:22: Unexpected character '−'.
8:30: Char literals must contain exactly one character and close with '''.
10:106: Char literals must contain exactly one character and close with '''.
12:24: Unexpected character '−'.
12:67: Char literals must contain exactly one character and close with '''.
12:252: Unexpected character '−'.
12:261: Char literals must contain exactly one character and close with '''.
14:80: Unexpected character '−'.
14:108: Unexpected character '−'.
14:167: Unexpected character '−'.
14:195: Unexpected character '−'.
14:208: Unexpected character '−'.
16:9: Char literals must contain exactly one character and close with '''.
18:151: Char literals must contain exactly one character and close with '''.
22:52: Unexpected character '−'.
22:68: Unexpected character '−'.
22:73: Unexpected character '−'.
22:115: Unexpected character '≈'.
22:144: Unexpected character '…'.
24:35: Char literals must contain exactly one character and close with '''.
26:27: Unexpected character '°'.
26:65: Unexpected character '−'.
26:143: Unexpected character '−'.
26:216: Unexpected character '−'.
26:227: Unexpected character '−'.
26:259: Unexpected character '−'.
26:271: Char literals must contain exactly one character and close with '''.
26:465: Unexpected character '−'.
26:483: Unexpected character '−'.
26:529: Unexpected character '−'.
26:538: Unexpected character '−'.
26:554: Unexpected character '−'.
26:580: Char literals must contain exactly one character and close with '''.
26:596: Unexpected character '−'.
26:598: Char literals must contain exactly one character and close with '''.
26:627: Char literals must contain exactly one character and close with '''.
26:650: Char literals must contain exactly one character and close with '''.
26:667: Unexpected character '−'.
26:670: Unexpected character '−'.
26:681: Unexpected character '−'.
26:684: Unexpected character '−'.
26:700: Unexpected character '−'.
26:707: Unexpected character '−'.
26:715: Unexpected character '−'.
26:717: Unexpected character '−'.
26:737: Char literals must contain exactly one character and close with '''.
28:57: Unexpected character '°'.
28:96: Char literals must contain exactly one character and close with '''.
30:37: Unexpected character '°'.
30:224: Unexpected character '−'.
30:232: Unexpected character '−'.
30:320: Unexpected character '≤'.
30:331: Unexpected character '≤'.
30:335: Unexpected character '−'.
30:359: Unexpected character '≤'.
30:374: Char literals must contain exactly one character and close with '''.
30:913: Unexpected character '°'.
30:960: Unexpected character '°'.
30:1043: Char literals must contain exactly one character and close with '''.
30:1348: Unexpected character 'θ'.
30:1356: Unexpected character 'θ'.
30:1373: Unexpected character 'θ'.
30:1376: Unexpected character 'θ'.
32:179: Char literals must contain exactly one character and close with '''.
34:106: Unexpected character '→'.
34:110: Unexpected character '°'.
34:159: Unexpected character '→'.
34:164: Unexpected character '°'.
34:170: Char literals must contain exactly one character and close with '''.
38:15: Unexpected character '°'.
38:54: Unexpected character '−'.
38:65: Unexpected character '°'.
38:79: Unexpected character '−'.
38:86: Unexpected character '−'.
38:97: Unexpected character '°'.
38:121: Unexpected character '−'.
40:24: Unexpected character '−'.
40:31: Unexpected character '−'.
43:4: Unexpected character '°'.
44:5: Unexpected character '°'.
44:18: Unexpected character '−'.
45:6: Unexpected character '°'.
45:13: Unexpected character '−'.
45:20: Unexpected character '−'.
46:6: Unexpected character '°'.
46: ... (message truncated) |
Failed |
IN9 251
OUT3 336 |
| ✓ |
LcbTakahashiSanLiveCodeBench / atcoder / easy: abc374_a Takahashi san 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 153
OUT42 |
| × |
LcbThreeTeamDivisionLiveCodeBench / atcoder / hard: abc375_e 3 Team Division (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ThreeTeamDivision case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:44: Unexpected character '∈'.
11:83: Unexpected character '≠'.
13:30: Unexpected character '≤'.
13:45: Unexpected character '≤'.
17:136: Char literals must contain exactly one character and close with '''.
21:126: Unexpected character '→'.
21:162: Unexpected character '≠'.
21:182: Unexpected character '→'.
25:12: Unexpected character '≠'.
29:70: Unexpected character '±'.
29:130: Char literals must contain exactly one character and close with '''.
31:43: Char literals must contain exactly one character and close with '''.
35:4: Unexpected character '’'.
35:74: Unexpected character '⊆'.
35:525: Unexpected character '✓'.
37:53: Unexpected character '⊆'.
37:62: Unexpected character '⊆'.
37:71: Unexpected character '⊆'.
41:20: Unexpected character '≤'.
43:146: Char literals must contain exactly one character and close with '''.
49:4: Char literals must contain exactly one character and close with '''.
51:21: Unexpected character '≤'.
51:27: Unexpected character '≤'.
51:45: Unexpected character '≤'.
51:92: Unexpected character '≤'.
51:100: Unexpected character '≈'.
51:165: Unexpected character 'Σ'.
53:111: Unexpected character '≤'.
59:9: Unexpected character '≤'.
59:15: Unexpected character '≤'.
59:25: Unexpected character '≤'.
59:47: Unexpected character '≤'.
59:79: Unexpected character '≤'.
59:141: Char literals must contain exactly one character and close with '''.
61:33: Unexpected character '≤'.
71:4: Char literals must contain exactly one character and close with '''.
77:133: Unexpected character '≤'.
81:115: Char literals must contain exactly one character and close with '''.
89:4: Char literals must contain exactly one character and close with '''.
123:185: Unexpected character '#'.
123:321: Char literals must contain exactly one character and close with '''.
123:354: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 270
OUT2 740 |
| – |
LcbTravelingTakahashiLiveCodeBench / atcoder / easy: abc375_b Traveling Takahashi Problem (Pattern B, stdin) [float output, tolerance 1e-6]0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBenchSkipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped |
IN0
OUT0 |
| ✓ |
LcbUnvarnishedReportLiveCodeBench / atcoder / easy: abc374_b Unvarnished Report (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 181
OUT116 |
Code Generation - LiveCodeBench v5 AtCoder (set 2)
3 passed8 failed |
27.3% |
IN102 102
OUT19 881 |
| ✓ |
LcbAvoidKnightAttackLiveCodeBench / atcoder / medium: abc377_c Avoid Knight Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 307
OUT558 |
| ✓ |
LcbAvoidRookAttackLiveCodeBench / atcoder / easy: abc377_b Avoid Rook Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 255
OUT281 |
| ✓ |
LcbCandyButtonLiveCodeBench / atcoder / easy: abc376_a Candy Button (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 223
OUT145 |
| × |
LcbCycleMinEdgesLiveCodeBench / atcoder / medium: abc376_d Cycle (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CycleMinEdges case0: expected [3] got [2] |
Failed |
IN9 245
OUT473 |
| × |
LcbHandsOnRingEasyLiveCodeBench / atcoder / medium: abc376_b Hands on Ring (Easy) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: HandsOnRingEasy case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:279: Unexpected character '—'.
11:104: Char literals must contain exactly one character and close with '''.
11:233: Char literals must contain exactly one character and close with '''.
13:23: Char literals must contain exactly one character and close with '''.
13:297: Char literals must contain exactly one character and close with '''.
15:155: Char literals must contain exactly one character and close with '''.
17:121: Char literals must contain exactly one character and close with '''.
17:163: Char literals must contain exactly one character and close with '''.
19:231: Char literals must contain exactly one character and close with '''.
19:367: Char literals must contain exactly one character and close with '''.
21:368: Unexpected character '≠'.
21:445: Char literals must contain exactly one character and close with '''.
23:155: Unexpected character '≤'.
23:246: Unexpected character '≠'.
23:281: Unexpected character '≤'.
23:290: Unexpected character '≈'.
23:318: Unexpected character '±'.
23:351: Unexpected character '±'.
25:383: Char literals must contain exactly one character and close with '''.
27:125: Char literals must contain exactly one character and close with '''.
27:531: Unexpected character '—'.
29:303: Char literals must contain exactly one character and close with '''.
29:414: Char literals must contain exactly one character and close with '''.
31:40: Unexpected character '—'.
31:48: Unexpected character '#'.
31:171: Unexpected character '→'.
31:606: Unexpected character '—'.
31:733: Unexpected character '≈'.
31:867: Char literals must contain exactly one character and close with '''.
33:13: Char literals must contain exactly one character and close with '''.
35:41: Unexpected character '≠'.
37:298: Unexpected character '—'.
67:2: Char literals must contain exactly one character and close with '''.
73:60: Unexpected character '×'.
73:67: Unexpected character '→'.
73:87: Unexpected character '≈'.
79:19: Unexpected character '≥'.
86:31: Unexpected character '→'.
92:66: Char literals must contain exactly one character and close with '''.
98:27: Char literals must contain exactly one character and close with '''.
102:4: Char literals must contain exactly one character and close with '''.
178:50: Char literals must contain exactly one character and close with '''.
180:2: Char literals must contain exactly one character and close with '''.
218:49: Char literals must contain exactly one character and close with '''.
289:112: Unexpected character '—'.
295:98: Unexpected character '#'.
295:213: Char literals must contain exactly one character and close with '''.
316:40: Unexpected character '≤'.
316:108: Unexpected character '≤'.
316:137: Unexpected character '≤'.
316:151: Unexpected character '«'.
449:29: Unexpected character '#'.
449:66: Unexpected character '—'.
Values:
run.Compiled = false |
Failed |
IN9 289
OUT5 587 |
| × |
LcbHandsOnRingHardLiveCodeBench / atcoder / hard: abc376_f Hands on Ring (Hard) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: HandsOnRingHard case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'Solve' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson.
Values:
run.Compiled = false |
Failed |
IN9 294
OUT1 |
| × |
LcbMaxTimesSumLiveCodeBench / atcoder / hard: abc376_e Max x Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaxTimesSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
9:232: Unexpected character '≥'.
9:284: Unexpected character '≥'.
11:230: Char literals must contain exactly one character and close with '''.
17:315: Unexpected character '≤'.
21:174: Unexpected character '≤'.
28:91: Unexpected character '≥'.
28:206: Unexpected character '≥'.
76:4: Char literals must contain exactly one character and close with '''.
88:840: Unexpected character '≤'.
88:890: Char literals must contain exactly one character and close with '''.
96:4: Char literals must contain exactly one character and close with '''.
207:94: Char literals must contain exactly one character and close with '''.
213:18: Unexpected character '≥'.
213:25: Unexpected character '≥'.
233:10: Unexpected character '#'.
235:14: Unexpected character '`'.
235:36: Unexpected character '`'.
243:33: Unexpected character '\'.
244:2: Unexpected character '\'.
245:37: Unexpected character '\'.
246:30: Unexpected character '\'.
247:6: Unexpected character '\'.
248:32: Unexpected character '\'.
249:31: Unexpected character '\'.
250:10: Unexpected character '\'.
251:46: Unexpected character '\'.
252:10: Unexpected character '\'.
253:6: Unexpected character '\'.
254:16: Unexpected character '\'.
255:41: Unexpected character '\'.
256:37: Unexpected character '\'.
257:35: Unexpected character '\'.
258:6: Unexpected character '\'.
259:45: Unexpected character '\'.
260:45: Unexpected character '\'.
261:36: Unexpected character '\'.
262:36: Unexpected character '\'.
263:10: Unexpected character '\'.
264:49: Unexpected character '\'.
265:38: Unexpected character '\'.
266:10: Unexpected character '\'.
267:36: Unexpected character '\'.
268:10: Unexpected character '\'.
269:49: Unexpected character '\'.
270:47: Unexpected character '\'.
271:10: Unexpected character '\'.
272:20: Unexpected character '\'.
273:37: Unexpected character '\'.
274:37: Unexpected character '\'.
275:36: Unexpected character '\'.
276:10: Unexpected character '\'.
277:36: Unexpected character '\'.
278:44: Unexpected character '\'.
279:26: Unexpected character '\'.
280:26: Unexpected character '\'.
281:10: Unexpected character '\'.
282:44: Unexpected character '\'.
283:24: Unexpected character '\'.
284:37: Unexpected character '\'.
285:36: Unexpected character '\'.
286:10: Unexpected character '\'.
287:82: Unexpected character '\'.
288:10: Unexpected character '\'.
289:28: Unexpected character '\'.
290:47: Unexpected character '\'.
291:62: Unexpected character '\'.
292:39: Unexpected character '\'.
293:39: Unexpected character '\'.
294:71: Unexpected character '\'.
295:35: Unexpected character '\'.
296:23: Unexpected character '\'.
297:20: Unexpected character '\'.
298:41: Unexpected character '\'.
299:36: Unexpected character '\'.
300:10: Unexpected character '\'.
301:24: Unexpected character '\'.
302:14: Unexpected character '\'.
303:35: Unexpected character '\'.
304:27: Unexpected character '\'.
305:18: Unexpected character '\'.
306:33: Unexpected character '\'.
307:38: Unexpected character '\'.
308:35: Unexpected character '\'.
309:40: Unexpected character '\'.
310:22: Unexpected character '\'.
311:49: Unexpected character '\'.
312:60: Unexpected character '\'.
313:26: Unexpected character '\'.
314:39: Unexpected character '\'.
315:49: Unexpected character '\'.
316:26: Unexpected character '\'.
317:39: Unexpected character '\'.
318:22: Unexpected character '\'.
319:39: Unexpected character '\'.
320:38: Unexpected character '\'.
321:38: Unexpected character '\'.
322:34: Unexpected character '\'.
323:35: Unexpected character '\'.
324:22: Unexpected character '\'.
325:47: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 281
OUT5 119 |
| × |
LcbPrepareAnotherBoxLiveCodeBench / atcoder / medium: abc376_c Prepare Another Box (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PrepareAnotherBox case0: expected [3] got [1] |
Failed |
IN9 278
OUT386 |
| × |
LcbRoadBlockedLiveCodeBench / atcoder / hard: abc375_f Road Blocked (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RoadBlocked case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
66:29: Type 'List<long>' has no supported method named 'IndexOf' with 1 argument(s).
68:29: Type 'List<long>' has no supported method named 'IndexOf' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 304
OUT996 |
| × |
LcbRoadBlocked2LiveCodeBench / atcoder / hard: abc375_g Road Blocked 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RoadBlocked2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
28:23: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '('.
Values:
run.Compiled = false |
Failed |
IN9 293
OUT4 231 |
| × |
LcbTreasureHuntingLiveCodeBench / atcoder / hard: abc376_g Treasure Hunting (Pattern B, stdin) [answer modulo 998244353]1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TreasureHunting case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:172: Char literals must contain exactly one character and close with '''.
9:70: Char literals must contain exactly one character and close with '''.
11:72: Char literals must contain exactly one character and close with '''.
21:71: Unexpected character '×'.
25:278: Unexpected character '→'.
25:300: Unexpected character '≥'.
29:33: Unexpected character 'Σ'.
29:37: Unexpected character '≥'.
29:64: Unexpected character '−'.
29:92: Unexpected character 'Σ'.
29:104: Unexpected character 'Σ'.
31:2: Char literals must contain exactly one character and close with '''.
33:46: Unexpected character 'Σ'.
33:132: Unexpected character '∪'.
33:168: Char literals must contain exactly one character and close with '''.
35:26: Char literals must contain exactly one character and close with '''.
39:306: Unexpected character 'Σ'.
39:351: Unexpected character '×'.
41:139: Char literals must contain exactly one character and close with '''.
41:206: Unexpected character '—'.
44:8: Unexpected character 'Σ'.
44:48: Unexpected character '·'.
44:161: Unexpected character '—'.
44:231: Char literals must contain exactly one character and close with '''.
46:136: Char literals must contain exactly one character and close with '''.
48:12: Unexpected character 'Σ'.
48:17: Unexpected character '∈'.
48:40: Unexpected character '·'.
49:153: Unexpected character '—'.
49:192: Char literals must contain exactly one character and close with '''.
51:10: Char literals must contain exactly one character and close with '''.
53:87: Unexpected character 'Σ'.
53:91: Unexpected character '∈'.
53:112: Unexpected character '·'.
55:16: Unexpected character 'Σ'.
59:26: Unexpected character '−'.
59:217: Unexpected character 'Σ'.
59:221: Unexpected character '∈'.
63:20: Unexpected character 'Σ'.
65:141: Unexpected character 'Σ'.
65:197: Unexpected character 'Σ'.
65:229: Char literals must contain exactly one character and close with '''.
67:8: Unexpected character 'Σ'.
67:31: Unexpected character '·'.
75:54: Unexpected character '≤'.
83:123: Unexpected character '—'.
85:17: Unexpected character 'Σ'.
87:80: Unexpected character '—'.
93:25: Unexpected character '≥'.
93:43: Unexpected character '≥'.
101:70: Unexpected character '≤'.
101:245: Unexpected character '→'.
113:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 333
OUT2 104 |
Code Generation - LiveCodeBench v5 AtCoder (set 3)
5 passed9 failed |
35.7% |
IN166 669
OUT11 205 |
| × |
LcbAddOneEdge2LiveCodeBench / atcoder abc378_f: Add One Edge 2 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AddOneEdge2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
76:25: The if condition must be 'bool', but it has type 'void'.
Values:
run.Compiled = false |
Failed |
IN9 176
OUT3 788 |
| × |
LcbAvoidQueenAttackLiveCodeBench / atcoder abc377_f: Avoid Queen Attack (hard)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: AvoidQueenAttack case1 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN20 204
OUT1 424 |
| ✓ |
LcbCountSimplePathsLiveCodeBench / atcoder abc378_d: Count Simple Paths (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 195
OUT608 |
| × |
LcbEditToMatchLiveCodeBench / atcoder abc377_g: Edit to Match (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: EditToMatch case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:28: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
run.Compiled = false |
Failed |
IN9 250
OUT395 |
| × |
LcbEverlastingLidsLiveCodeBench / atcoder abc378_g: Everlasting LIDS (hard)1 AI query
2 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: EverlastingLids case0: expected [10] got [25] |
Failed |
IN36 137
OUT3 013 |
| ✓ |
LcbGarbageCollectionLiveCodeBench / atcoder abc378_b: Garbage Collection (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 217
OUT331 |
| × |
LcbManySegments2LiveCodeBench / atcoder abc377_d: Many Segments 2 (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ManySegments2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:8: Expected a name, found '.'.
Values:
run.Compiled = false |
Failed |
IN9 207
OUT26 |
| × |
LcbModSigmaProblemLiveCodeBench / atcoder abc378_e: Mod Sigma Problem (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ModSigmaProblem case0: expected [10] got [13] |
Failed |
IN9 183
OUT431 |
| ✓ |
LcbPairingLiveCodeBench / atcoder abc378_a: Pairing (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 168
OUT184 |
| × |
LcbPermuteKTimesLiveCodeBench / atcoder abc377_e: Permute K times 2 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PermuteKTimes case2: expected [18 23 16 24 21 10 2 27 19 7 12 8 13 5 15 26 17 4 3 9 1 22 25 14 28 11 29 6 20] got [21 7 11 18 14 28 10 20 16 6 27 29 15 24 13 12 22 1 26 3 5 17 2 4 23 8 9 25 19] |
Failed |
IN9 203
OUT451 |
| ✓ |
LcbRearrangingAbcLiveCodeBench / atcoder abc377_a: Rearranging ABC (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 164
OUT146 |
| × |
LcbRepeatingLiveCodeBench / atcoder abc378_c: Repeating (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Repeating case0: expected [-1 -1 1 3 -1] got [-1 -1 0 2 -1] |
Failed |
IN9 185
OUT140 |
| ✓ |
LcbStrawberriesLiveCodeBench / atcoder abc379_b: Strawberries (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 188
OUT162 |
| × |
LcbSumOfAllSubstringsLiveCodeBench / atcoder abc379_e: Sum of All Substrings (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SumOfAllSubstrings case0: expected [514] got [419] |
Failed |
IN9 192
OUT106 |
Code Generation - LiveCodeBench v5 AtCoder (set 4)
6 passed8 failed |
42.9% |
IN268 334
OUT35 714 |
| ✓ |
Lcb1122StringLiveCodeBench / atcoder abc381_a: 11/22 String (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 273
OUT161 |
| × |
Lcb1122SubsequenceLiveCodeBench / atcoder abc381_e: 11/22 Subsequence (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Subsequence1122 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:65: Unexpected character '`'.
3:67: Unexpected character '`'.
3:76: Unexpected character '`'.
3:78: Unexpected character '`'.
3:101: Unexpected character '`'.
3:103: Unexpected character '`'.
3:117: Unexpected character '`'.
3:119: Unexpected character '`'.
3:122: Unexpected character '`'.
3:126: Unexpected character '`'.
3:129: Unexpected character '`'.
3:135: Unexpected character '`'.
3:138: Unexpected character '`'.
3:146: Unexpected character '`'.
3:173: Unexpected character '`'.
3:175: Unexpected character '`'.
3:182: Unexpected character '`'.
3:184: Unexpected character '`'.
3:190: Char literals must contain exactly one character and close with '''.
3:357: Unexpected character '`'.
3:367: Unexpected character '`'.
3:422: Unexpected character '—'.
5:45: Unexpected character '`'.
5:55: Unexpected character '`'.
5:68: Unexpected character '≥'.
5:313: Unexpected character '→'.
5:360: Unexpected character '`'.
5:368: Unexpected character '`'.
7:182: Unexpected character '`'.
7:190: Unexpected character '`'.
11:88: Unexpected character '≤'.
11:311: Unexpected character '≥'.
11:342: Unexpected character '≥'.
13:215: Unexpected character '≤'.
15:20: Unexpected character '∈'.
17:16: Unexpected character '#'.
17:41: Unexpected character '#'.
21:48: Char literals must contain exactly one character and close with '''.
23:296: Unexpected character '∈'.
23:302: Unexpected character '∩'.
25:178: Unexpected character '≈'.
27:115: Unexpected character '→'.
27:124: Unexpected character '≥'.
27:197: Unexpected character '≤'.
27:220: Unexpected character '≥'.
31:166: Unexpected character '→'.
31:291: Unexpected character '⇒'.
31:338: Unexpected character '±'.
31:490: Unexpected character '→'.
31:740: Unexpected character '≤'.
31:754: Unexpected character '≥'.
35:91: Unexpected character '≤'.
35:137: Unexpected character '≥'.
37:35: Unexpected character '≥'.
37:83: Unexpected character '≤'.
53:72: Unexpected character '→'.
53:124: Unexpected character '≤'.
132:61: Unexpected character '\'.
132:124: Unexpected character '\'.
134:64: Unexpected character '≥'.
136:219: Unexpected character '≤'.
136:227: Unexpected character '≥'.
Values:
run.Compiled = false |
Failed |
IN9 268
OUT3 863 |
| × |
LcbAnotherShuffleWindowLiveCodeBench / atcoder abc380_g: Another Shuffle Window (hard)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AnotherShuffleWindow case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN20 262
OUT11 197 |
| × |
LcbBucketToolLiveCodeBench / atcoder abc380_e: 1D Bucket Tool (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BucketTool case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
8:167: Char literals must contain exactly one character and close with '''.
18:24: Unexpected character '`'.
18:34: Unexpected character '`'.
18:131: Unexpected character '`'.
18:142: Unexpected character '`'.
18:148: Unexpected character '`'.
18:158: Unexpected character '`'.
20:73: Unexpected character '`'.
20:78: Unexpected character '`'.
20:84: Unexpected character '`'.
20:89: Unexpected character '`'.
20:94: Char literals must contain exactly one character and close with '''.
22:123: Unexpected character '`'.
22:131: Unexpected character '`'.
24:35: Unexpected character '→'.
26:55: Unexpected character '→'.
30:181: Char literals must contain exactly one character and close with '''.
32:93: Char literals must contain exactly one character and close with '''.
36:218: Unexpected character '→'.
38:2: Char literals must contain exactly one character and close with '''.
38:245: Unexpected character '—'.
40:21: Char literals must contain exactly one character and close with '''.
40:302: Unexpected character '→'.
40:398: Unexpected character '≤'.
40:478: Char literals must contain exactly one character and close with '''.
42:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 211
OUT1 953 |
| × |
LcbBuildings2LiveCodeBench / atcoder abc379_f: Buildings 2 (hard)1 AI query
5 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Buildings2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN126 653
OUT10 922 |
| × |
LcbCountGrid3LiveCodeBench / atcoder abc379_g: Count Grid 3 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountGrid3 case0: expected [6] got [0] |
Failed |
IN9 203
OUT474 |
| ✓ |
LcbCyclicLiveCodeBench / atcoder abc379_a: Cyclic (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 177
OUT116 |
| ✓ |
LcbDigitPattern123233LiveCodeBench / atcoder abc380_a: 123233 (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 164
OUT188 |
| × |
LcbExchangeGameLiveCodeBench / atcoder abc380_f: Exchange Game (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ExchangeGame case1: expected [Takahashi] got [Aoki] |
Failed |
IN9 255
OUT2 658 |
| ✓ |
LcbHomeGardenLiveCodeBench / atcoder abc379_d: Home Garden (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 191
OUT363 |
| × |
LcbHurdleParsingLiveCodeBench / atcoder abc380_b: Hurdle Parsing (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: HurdleParsing case0: expected [3 1 4 1 5] got [3 4 5] |
Failed |
IN9 175
OUT192 |
| ✓ |
LcbMoveSegmentLiveCodeBench / atcoder abc380_c: Move Segment (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 186
OUT2 201 |
| ✓ |
LcbSowingStonesLiveCodeBench / atcoder abc379_c: Sowing Stones (medium)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN20 122
OUT1 111 |
| × |
LcbStrangeMirroringLiveCodeBench / atcoder abc380_d: Strange Mirroring (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: StrangeMirroring case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
36:31: Expected ')' to close the parenthesized expression, found 'c'.
Values:
run.Compiled = false |
Failed |
IN9 194
OUT315 |
Code Generation - LiveCodeBench v5 AtCoder (set 5)
7 passed6 failed |
53.8% |
IN137 472
OUT16 908 |
| ✓ |
Lcb1122StringLiveCodeBench / atcoder / easy: 1122 String (abc381_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 256
OUT203 |
| ✓ |
Lcb1122SubarrayLiveCodeBench / atcoder / hard: 1122 Substring (abc381_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 296
OUT4 241 |
| ✓ |
LcbDailyCookieLiveCodeBench / atcoder / easy: Daily Cookie (abc382_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 262
OUT123 |
| ✓ |
LcbDailyCookie2LiveCodeBench / atcoder / easy: Daily Cookie 2 (abc382_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 284
OUT189 |
| × |
LcbFallingBarsLiveCodeBench / atcoder / hard: Falling Bars (abc382_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: FallingBars case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
38:28: Type 'List<int>' has no supported method named 'BinarySearch' with 1 argument(s).
48:34: Type 'List<int>' has no supported method named 'BinarySearch' with 1 argument(s).
51:38: Type 'List<int>' has no supported method named 'BinarySearch' with 1 argument(s).
64:25: Type 'string' has no supported method named 'TrimEnd'.
Values:
run.Compiled = false |
Failed |
IN9 405
OUT729 |
| ✓ |
LcbHumidifier1LiveCodeBench / atcoder / easy: Humidifier 1 (abc383_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 255
OUT149 |
| ✓ |
LcbHumidifier2LiveCodeBench / atcoder / medium: Humidifier 2 (abc383_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 319
OUT505 |
| ✓ |
LcbHumidifier3LiveCodeBench / atcoder / medium: Humidifier 3 (abc383_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 309
OUT743 |
| × |
LcbKaitenSushiLiveCodeBench / atcoder / medium: Kaiten Sushi (abc382_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: KaitenSushi case0: expected [1
3
-1] got [-1
2
1] |
Failed |
IN9 315
OUT301 |
| × |
LcbKeepDistanceLiveCodeBench / atcoder / hard: Keep Distance (abc382_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: KeepDistance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
20:25: Type 'string' has no supported method named 'TrimEnd'.
Values:
run.Compiled = false |
Failed |
IN9 278
OUT337 |
| × |
LcbMax1122SubstringLiveCodeBench / atcoder / medium: 11/22 Substring (abc381_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Max1122Substring case0: expected [5] got [1] |
Failed |
IN9 381
OUT433 |
| × |
LcbSumOfMaxMatchingLiveCodeBench / atcoder / hard: Sum of Max Matching (abc383_e)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SumOfMaxMatching case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
46:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN25 716
OUT8 237 |
| × |
LcbTileDistance3LiveCodeBench / atcoder / hard: Tile Distance 3 (abc382_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TileDistance3 case0: expected [4
4
800000000000] got [2
4
799999999998] |
Failed |
IN9 396
OUT718 |
Code Generation - LiveCodeBench v5 AtCoder (set 6)
7 passed6 failed1 skipped |
53.8% |
IN132 486
OUT19 505 |
| × |
LcbAbsSumLiveCodeBench / atcoder / hard: Abs Sum (abc384_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AbsSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:36: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:47: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:58: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:59: Unexpected character '\'.
3:75: Unexpected character '\'.
3:103: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:20: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:22: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:28: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:30: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:41: Char literals must contain exactly one character and close with '''.
9:12: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:18: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:21: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:33: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:48: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:50: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:63: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:65: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:122: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:126: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:140: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:142: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:154: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:158: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:172: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:174: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:60: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:73: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:115: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:119: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:9: Char literals must contain exactly one character and close with '''.
17:28: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:30: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:38: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:43: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:54: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:56: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:64: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:69: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:87: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:94: Unexpected character '\'.
17:104: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:38: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:42: Unexpected character '$'. Did you mean a string lit ... (message truncated) |
Failed |
IN9 316
OUT6 179 |
| ✓ |
LcbArcDivisionLiveCodeBench / atcoder / easy: ARC Division (abc384_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 363
OUT211 |
| ✓ |
LcbDoubleSum2LiveCodeBench / atcoder / hard: Double Sum 2 (abc384_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 253
OUT288 |
| ✓ |
LcbEquallyLiveCodeBench / atcoder / easy: Equally (abc385_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 186
OUT140 |
| ✓ |
LcbIlluminateBuildingsLiveCodeBench / atcoder / medium: Illuminate Buildings (abc385_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 226
OUT233 |
| × |
LcbMaximizeXorLiveCodeBench / atcoder / hard: Maximize XOR (abc386_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MaximizeXor case0: expected [7] got [1152921504606846974] |
Failed |
IN9 242
OUT662 |
| ✓ |
LcbPerfectStandingsLiveCodeBench / atcoder / medium: Perfect Standings (abc384_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 304
OUT464 |
| × |
LcbRepeatedSequenceLiveCodeBench / atcoder / medium: Repeated Sequence (abc384_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RepeatedSequence case1: expected [No] got [Yes] |
Failed |
IN9 248
OUT474 |
| ✓ |
LcbReplaceExceptCharLiveCodeBench / atcoder / easy: aaaadaa (abc384_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 204
OUT135 |
| ✓ |
LcbSantaClaus1LiveCodeBench / atcoder / easy: Santa Claus 1 (abc385_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 469
OUT1 719 |
| × |
LcbSantaClaus2LiveCodeBench / atcoder / hard: Santa Claus 2 (abc385_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: SantaClaus2 case1 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 444
OUT610 |
| × |
LcbSlime2LiveCodeBench / atcoder / hard: Takahashi is Slime 2 (abc384_e)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Slime2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:7: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN20 937
OUT6 664 |
| × |
LcbSnowflakeTreeLiveCodeBench / atcoder / hard: Snowflake Tree (abc385_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SnowflakeTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
8:37: Unexpected character '→'.
8:57: Unexpected character '→'.
12:140: Char literals must contain exactly one character and close with '''.
16:30: Char literals must contain exactly one character and close with '''.
16:384: Char literals must contain exactly one character and close with '''.
18:5: Unexpected character '—'.
23:9: Char literals must contain exactly one character and close with '''.
25:16: Char literals must contain exactly one character and close with '''.
27:34: Unexpected character '≥'.
27:39: Unexpected character '≥'.
27:101: Unexpected character '→'.
29:85: Unexpected character '#'.
29:230: Unexpected character '≥'.
29:383: Unexpected character '≥'.
29:506: Unexpected character '≥'.
31:82: Unexpected character '≥'.
31:161: Unexpected character '≥'.
31:193: Unexpected character '—'.
31:515: Unexpected character '∪'.
31:519: Unexpected character '∪'.
31:573: Char literals must contain exactly one character and close with '''.
31:741: Unexpected character '—'.
33:53: Unexpected character '#'.
33:69: Unexpected character '≥'.
33:85: Unexpected character '#'.
33:95: Unexpected character '≥'.
37:148: Unexpected character '#'.
37:171: Unexpected character '≥'.
37:228: Unexpected character '≥'.
37:245: Unexpected character '≥'.
39:227: Unexpected character '≤'.
39:327: Unexpected character '→'.
39:425: Unexpected character '→'.
41:89: Unexpected character '≥'.
41:191: Unexpected character '≤'.
41:224: Unexpected character '→'.
41:353: Unexpected character 'Σ'.
41:368: Unexpected character 'Σ'.
41:395: Unexpected character '≈'.
41:406: Char literals must contain exactly one character and close with '''.
43:80: Unexpected character '→'.
43:107: Unexpected character '≤'.
43:124: Unexpected character '→'.
43:223: Char literals must contain exactly one character and close with '''.
45:269: Unexpected character '∈'.
45:304: Unexpected character '≤'.
45:413: Char literals must contain exactly one character and close with '''.
52:89: Unexpected character '≥'.
55:45: Unexpected character '≥'.
55:102: Unexpected character '≥'.
Values:
run.Compiled = false |
Failed |
IN9 294
OUT1 726 |
| – |
LcbVisibleBuildingsLiveCodeBench / atcoder / hard: Visible Buildings (abc385_f)0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped |
IN0
OUT0 |
Code Generation - LiveCodeBench v5 AtCoder (set 7)
3 passed9 failed1 skipped |
25.0% |
IN121 916
OUT50 757 |
| × |
LcbAccumulatingManyTimesLiveCodeBench / atcoder / hard: arc184_e Accumulating Many Times (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AccumulatingManyTimes case0: expected [8] got [5] |
Failed |
IN9 291
OUT920 |
| × |
LcbAdjacentGcdLiveCodeBench / atcoder / hard: arc185_e Adjacent GCD (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AdjacentGcd case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'Solve' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson.
Values:
run.Compiled = false |
Failed |
IN9 267
OUT27 |
| ✓ |
LcbCalculatorMinPressesLiveCodeBench / atcoder / easy: abc386_b Calculator (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 208
OUT150 |
| × |
LcbDiagonalSeparationLiveCodeBench / atcoder / hard: abc386_d Diagonal Separation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DiagonalSeparation case0: expected [Yes] got [No] |
Failed |
IN9 287
OUT2 192 |
| × |
LcbEraseBalls2DLiveCodeBench / atcoder / hard: arc184_d Erase Balls 2D (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: EraseBalls2D case1: expected [3] got [7] |
Failed |
IN9 253
OUT2 235 |
| × |
LcbFullHouse2LiveCodeBench / atcoder / easy: abc386_a Full House 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FullHouse2 case3: expected [No] got [Yes] |
Failed |
IN9 202
OUT238 |
| × |
LcbMountainValleyFoldsLiveCodeBench / atcoder / hard: arc184_c Mountain and Valley Folds (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MountainValleyFolds case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
35:5: Expected an expression, found 'void'.
Values:
run.Compiled = false |
Failed |
IN9 311
OUT5 682 |
| ✓ |
LcbOperate1LiveCodeBench / atcoder / medium: abc386_c Operate 1 (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN20 118
OUT1 959 |
| ✓ |
LcbOperateKLiveCodeBench / atcoder / hard: abc386_f Operate K (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 215
OUT5 562 |
| × |
LcbPlusOneAndLiveCodeBench / atcoder / hard: arc185_b +1 and (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PlusOneAnd case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:76: Char literals must contain exactly one character and close with '''.
7:156: Char literals must contain exactly one character and close with '''.
9:84: Unexpected character '≤'.
9:92: Unexpected character '≥'.
9:100: Unexpected character '≤'.
9:201: Unexpected character '≤'.
9:301: Unexpected character '≥'.
9:343: Unexpected character '→'.
11:46: Unexpected character '≥'.
13:74: Char literals must contain exactly one character and close with '''.
15:100: Unexpected character '—'.
17:159: Unexpected character '→'.
17:380: Unexpected character '→'.
19:61: Unexpected character '≥'.
19:180: Unexpected character '≥'.
23:15: Unexpected character 'Σ'.
23:19: Unexpected character '≤'.
23:26: Unexpected character '≥'.
23:35: Unexpected character '∀'.
26:28: Unexpected character '≥'.
26:36: Unexpected character '−'.
26:37: Unexpected character '∞'.
28:153: Unexpected character 'Σ'.
28:157: Unexpected character '≤'.
28:209: Unexpected character '≥'.
28:228: Unexpected character '⇒'.
28:235: Unexpected character '≥'.
28:298: Unexpected character '—'.
28:371: Unexpected character '≥'.
30:125: Char literals must contain exactly one character and close with '''.
32:91: Char literals must contain exactly one character and close with '''.
34:117: Unexpected character '≤'.
36:120: Unexpected character '≈'.
42:217: Unexpected character '—'.
44:165: Unexpected character '≥'.
46:21: Char literals must contain exactly one character and close with '''.
73:4: Char literals must contain exactly one character and close with '''.
77:39: Char literals must contain exactly one character and close with '''.
107:4: Char literals must contain exactly one character and close with '''.
119:4: Char literals must contain exactly one character and close with '''.
127:145: Unexpected character '≥'.
131:71: Char literals must contain exactly one character and close with '''.
137:116: Unexpected character '≥'.
137:241: Unexpected character '≥'.
137:476: Unexpected character '≤'.
137:483: Unexpected character '≤'.
139:229: Char literals must contain exactly one character and close with '''.
139:374: Unexpected character '—'.
151:4: Char literals must contain exactly one character and close with '''.
177:4: Char literals must contain exactly one character and close with '''.
219:4: Char literals must contain exactly one character and close with '''.
231:4: Char literals must contain exactly one character and close with '''.
249:4: Char literals must contain exactly one character and close with '''.
305:3: Unexpected character '녕'.
307:3: Unexpected character 'á'.
311:5: Unexpected character 'å'.
313:11: Unexpected character 'ä'.
313:18: Unexpected character 'ä'.
317:2: Unexpected character 'ä'.
321:2: Unexpected character 'ä'.
321:3: Unexpected character 'ä'.
323:5: Unexpected character 'ä'.
331:5: Unexpected character 'ä'.
333:4: Unexpected character 'ä'.
335:4: Unexpected character 'ä'.
363:2: Unexpected character 'ä'.
377:11: Unexpected character 'ä'.
381:8: Unexpected character 'ä'.
381:9: Unexpected character 'ä'.
383:6: Unexpected character 'ä'.
387:5: Unexpected character 'ä'.
397:5: Unexpected character 'ä'.
2399:5: Char literals must contain exactly one character and close with '''.
4277:7: Unexpected character 'í'.
4307:13: Char literals must contain exactly one character and close with '''.
4651:10: Char literals must contain exactly one character and close with '''.
4711:6: Unexpected character 'ñ'.
Values:
run.Compiled = false |
Failed |
IN9 232
OUT25 032 |
| × |
LcbRandomWalkOnTreeLiveCodeBench / atcoder / hard: arc185_d Random Walk on Tree (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RandomWalkOnTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:19: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 282
OUT4 636 |
| – |
LcbSumOfThreeIntegersLiveCodeBench / atcoder / hard: arc185_c Sum of Three Integers (Pattern B, stdin) NOTE: multiple valid triples may exist; exact-match oracle may reject a different valid answer.0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbUndercluedLiveCodeBench / atcoder / hard: arc186_a Underclued (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Underclued case1: expected [No
Yes
No
Yes
No
Yes] got [No
No
No
No
No
No] |
Failed |
IN9 250
OUT2 124 |
Code Generation - LiveCodeBench v5 AtCoder (set 8)
1 passed11 failed1 skipped |
8.3% |
IN213 600
OUT53 010 |
| × |
LcbAbcSymmetryLiveCodeBench / atcoder / hard: arc188_a ABC Symmetry (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AbcSymmetry case0: expected [1] got [3] |
Failed |
IN21 269
OUT1 738 |
| × |
LcbBallAndBoxLiveCodeBench / atcoder / hard: arc186_c Ball and Box (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BallAndBox case0: expected [2
0
28] got [4
999999999
279] |
Failed |
IN9 288
OUT1 527 |
| × |
LcbBallsAndBoxesLiveCodeBench / atcoder / hard: arc189_c Balls and Boxes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BallsAndBoxes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:68: Char literals must contain exactly one character and close with '''.
11:219: Char literals must contain exactly one character and close with '''.
15:116: Char literals must contain exactly one character and close with '''.
17:73: Char literals must contain exactly one character and close with '''.
17:382: Char literals must contain exactly one character and close with '''.
19:134: Char literals must contain exactly one character and close with '''.
23:40: Char literals must contain exactly one character and close with '''.
27:184: Char literals must contain exactly one character and close with '''.
29:239: Unexpected character '≠'.
31:36: Unexpected character '#'.
31:81: Char literals must contain exactly one character and close with '''.
33:4: Char literals must contain exactly one character and close with '''.
35:676: Unexpected character '→'.
35:722: Unexpected character '≠'.
35:760: Unexpected character '→'.
35:886: Char literals must contain exactly one character and close with '''.
37:26: Unexpected character '#'.
37:40: Unexpected character '≠'.
37:63: Unexpected character '#'.
37:74: Unexpected character '≠'.
37:199: Char literals must contain exactly one character and close with '''.
39:242: Unexpected character '∪'.
39:266: Unexpected character '≠'.
39:448: Char literals must contain exactly one character and close with '''.
47:42: Unexpected character '→'.
55:51: Unexpected character '≠'.
59:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 253
OUT2 019 |
| – |
LcbHonestLiarConfusedLiveCodeBench / atcoder / hard: arc188_c Honest or Liar or Confused (Pattern B, stdin) NOTE: multiple valid confused sets may exist; exact-match oracle may reject a different valid answer.0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbMinimizeSumLiveCodeBench / atcoder / hard: arc189_b Minimize Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimizeSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:82: Unexpected character '≤'.
3:86: Unexpected character '≤'.
3:89: Unexpected character '−'.
5:126: Char literals must contain exactly one character and close with '''.
5:205: Unexpected character '−'.
5:240: Char literals must contain exactly one character and close with '''.
7:44: Char literals must contain exactly one character and close with '''.
9:14: Unexpected character '—'.
9:88: Char literals must contain exactly one character and close with '''.
15:83: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 237
OUT541 |
| × |
LcbMirrorAndOrderLiveCodeBench / atcoder / hard: arc188_d Mirror and Order (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MirrorAndOrder case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
53:18: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 322
OUT2 140 |
| × |
LcbMissingSubsequenceLiveCodeBench / atcoder / hard: arc186_e Missing Subsequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MissingSubsequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
102:18: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 245
OUT3 828 |
| ✓ |
LcbPolishManiaLiveCodeBench / atcoder / hard: arc186_d Polish Mania (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN88 757
OUT30 233 |
| × |
LcbReversi2LiveCodeBench / atcoder / medium: arc189_a Reversi 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Reversi2 case0: expected [3] got [24] |
Failed |
IN9 279
OUT940 |
| × |
LcbSumOfCCLiveCodeBench / atcoder / hard: arc187_b Sum of CC (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SumOfCC case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:26: Unexpected character '₁'.
5:35: Unexpected character 'ₙ'.
5:108: Unexpected character 'ᵢ'.
5:110: Unexpected character '≤'.
5:113: Unexpected character 'ⱼ'.
7:69: Char literals must contain exactly one character and close with '''.
11:35: Char literals must contain exactly one character and close with '''.
13:53: Unexpected character 'ᵢ'.
13:58: Unexpected character 'ⱼ'.
15:40: Char literals must contain exactly one character and close with '''.
19:82: Unexpected character 'ᵢ'.
19:84: Unexpected character '≤'.
19:87: Unexpected character 'ⱼ'.
21:15: Unexpected character 'ᵢ'.
21:17: Unexpected character '≤'.
21:20: Unexpected character 'ⱼ'.
21:27: Unexpected character 'ⱼ'.
21:29: Unexpected character '≤'.
21:140: Unexpected character 'ᵢ'.
21:141: Unexpected character '≤'.
25:24: Unexpected character '≤'.
25:44: Unexpected character '≤'.
25:47: Unexpected character '→'.
25:94: Unexpected character '≤'.
27:34: Unexpected character '≤'.
27:48: Unexpected character '≤'.
27:62: Unexpected character '≤'.
29:22: Char literals must contain exactly one character and close with '''.
30:135: Unexpected character '→'.
31:13: Unexpected character '≤'.
31:92: Unexpected character '≤'.
31:189: Unexpected character '→'.
37:26: Char literals must contain exactly one character and close with '''.
39:35: Unexpected character 'ᵢ'.
39:37: Unexpected character '≤'.
39:40: Unexpected character 'ⱼ'.
41:137: Unexpected character '≤'.
41:144: Unexpected character 'ᵢ'.
41:149: Unexpected character 'ⱼ'.
41:172: Unexpected character '≤'.
41:177: Unexpected character 'ᵢ'.
41:193: Unexpected character 'ⱼ'.
41:238: Unexpected character 'ᵢ'.
41:241: Unexpected character 'ⱼ'.
41:254: Unexpected character '≤'.
41:264: Char literals must contain exactly one character and close with '''.
41:371: Unexpected character '≤'.
41:376: Unexpected character 'ᵢ'.
41:378: Unexpected character '≥'.
41:461: Unexpected character 'ᵢ'.
41:466: Unexpected character 'ⱼ'.
41:477: Unexpected character '≤'.
41:485: Unexpected character '⟺'.
41:493: Unexpected character '≤'.
41:498: Unexpected character 'ᵢ'.
41:513: Unexpected character 'ⱼ'.
41:525: Unexpected character '—'.
41:776: Unexpected character '⟺'.
41:784: Unexpected character '≤'.
41:788: Unexpected character 'ᵢ'.
41:802: Unexpected character 'ⱼ'.
41:964: Unexpected character '≤'.
41:968: Unexpected character 'ᵢ'.
41:982: Unexpected character 'ⱼ'.
41:1040: Unexpected character '⟺'.
43:27: Char literals must contain exactly one character and close with '''.
45:142: Unexpected character 'ᵢ'.
45:162: Unexpected character 'ⱼ'.
47:38: Unexpected character 'ᵢ'.
47:47: Unexpected character '₁'.
49:62: Unexpected character '→'.
49:112: Unexpected character '≤'.
49:156: Unexpected character '✓'.
49:165: Unexpected character '≤'.
49:170: Unexpected character '≥'.
49:192: Unexpected character '✓'.
52:96: Unexpected character '✓'.
53:56: Unexpected character '→'.
53:71: Unexpected character '✓'.
54:54: Unexpected character '→'.
54:75: Unexpected character '≤'.
54:79: Unexpected character '≤'.
54:93: Unexpected character '≤'.
54:97: Unexpected character '≤'.
54:109: Unexpected character '≤'.
54:113: Unexpected character '≤'.
54:120: Unexpected character '→'.
54:158: Unexpected character '✓'.
55:53: Unexpected character '→'.
55:80: Unexpected character '≤'.
55:93: Unexpected character '≤'.
55:107: Unexpected character '≤'.
55:114: Unexpected character '→'.
55:134: Unexpected character '→'.
55:163: Unexpected character '→'.
55:170: Unexpected character '✗'.
57:97: Unexpected character '≤'.
57:103: Unexpected character '≤'.
57:117: Unexpected character '≤'.
57:132: Unexpected character '≤'.
57:176: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 284
OUT6 733 |
| × |
LcbSymmetricPaintingLiveCodeBench / atcoder / hard: arc188_b Symmetric Painting (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SymmetricPainting case0: expected [Yes
No
Yes
No] got [No
No
Yes
No] |
Failed |
IN9 241
OUT1 058 |
| × |
LcbTakahashiIsSlimeLiveCodeBench / atcoder / hard: arc189_d Takahashi is Slime (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TakahashiIsSlime case0: expected [4 30 2 13 2 13] got [4 30 13 13 13 13] |
Failed |
IN20 180
OUT567 |
| × |
LcbTypicalPermutationDescriptorLiveCodeBench / atcoder / hard: arc186_b Typical Permutation Descriptor (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TypicalPermutationDescriptor case0: expected [3] got [1] |
Failed |
IN9 245
OUT1 686 |
Code Generation - LiveCodeBench v5 LeetCode (set 1)
4 passed7 failed |
36.4% |
IN101 058
OUT8 975 |
| × |
LcbCountWinningSequencesLiveCodeBench / leetcode / hard: count-the-number-of-winning-sequences1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countWinning case0: expected 3 got 0 |
Failed |
IN9 255
OUT790 |
| × |
LcbDfsPalindromesLiveCodeBench / leetcode / hard: check-if-dfs-strings-are-palindromes1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<bool>>(run.ResultJson) == FromJson<List<bool>>(expected[i])
Context: dfsPalindromes case0: expected [true, true, false, true, true, true] got [false,false,false,true,true,true] |
Failed |
IN9 259
OUT620 |
| ✓ |
LcbFindXSumLiveCodeBench / leetcode / easy: find-x-sum-of-all-k-long-subarrays-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 231
OUT4 588 |
| ✓ |
LcbIsBalancedStringLiveCodeBench / leetcode / easy: check-balanced-string1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 121
OUT115 |
| × |
LcbLengthAfterTransformationsLiveCodeBench / leetcode / medium: total-characters-in-string-after-transformations-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: lengthAfter case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:36: Unexpected character '`'.
3:43: Unexpected character '`'.
13:52: Unexpected character '—'.
16:80: Unexpected character '→'.
32:5: Char literals must contain exactly one character and close with '''.
34:35: Char literals must contain exactly one character and close with '''.
46:17: Unexpected character '→'.
50:2: Char literals must contain exactly one character and close with '''.
66:21: Unexpected character '→'.
67:89: Unexpected character '→'.
69:109: Unexpected character '✓'.
71:119: Unexpected character '✓'.
Values:
run.Compiled = false |
Failed |
IN9 173
OUT1 255 |
| ✓ |
LcbMaxFactorScoreLiveCodeBench / leetcode / medium: find-the-maximum-factor-score-of-array1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 166
OUT333 |
| × |
LcbMinDivisionOperationsLiveCodeBench / leetcode / medium: minimum-division-operations-to-make-array-non-decreasing1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOperations case0: expected 1 got -1 |
Failed |
IN9 155
OUT190 |
| × |
LcbMinTimeToReachRoomLiveCodeBench / leetcode / medium: find-minimum-time-to-reach-last-room-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minTimeToReach case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
23:29: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
24:29: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
Values:
run.Compiled = false |
Failed |
IN9 225
OUT389 |
| ✓ |
LcbNumberOfSubstringsKFreqLiveCodeBench / leetcode / medium: count-substrings-with-k-frequency-characters-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 111
OUT214 |
| × |
LcbStringSequenceLiveCodeBench / leetcode / medium: find-the-sequence-of-strings-appeared-on-the-screen1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: stringSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:13: Cannot assign a value of type 'int' to variable 'cur' of type 'char'.
Values:
run.Compiled = false |
Failed |
IN9 174
OUT155 |
| × |
LcbSubsequencePairCountGcdLiveCodeBench / leetcode / hard: find-the-number-of-subsequences-with-equal-gcd1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: subseqPair case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
12:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 188
OUT326 |
Code Generation - LiveCodeBench v5 LeetCode (set 2)
4 passed6 failed |
40.0% |
IN92 112
OUT23 690 |
| × |
LcbCountBalancedPermutationsLiveCodeBench / leetcode / hard: count-number-of-balanced-permutations1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: countBalPerm case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 158
OUT6 637 |
| × |
LcbCountKReducibleNumbersLiveCodeBench / leetcode / hard: count-k-reducible-numbers-less-than-n1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: countKReducible case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 185
OUT590 |
| ✓ |
LcbCountValidSelectionsLiveCodeBench / leetcode / easy: make-array-elements-equal-to-zero1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 241
OUT380 |
| ✓ |
LcbHasIncreasingSubarraysLiveCodeBench / leetcode / easy: adjacent-increasing-subarrays-detection-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 221
OUT194 |
| ✓ |
LcbIsZeroArrayLiveCodeBench / leetcode / medium: zero-array-transformation-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 213
OUT176 |
| × |
LcbMaximizeSumOfWeightsLiveCodeBench / leetcode / hard: maximize-sum-of-weights-after-edge-removals1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maximizeSumWeights case0: expected 22 got 24 |
Failed |
IN9 268
OUT440 |
| × |
LcbMinAdjacentDifferenceLiveCodeBench / leetcode / hard: minimize-the-maximum-adjacent-element-difference1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minDifference case2: expected 1 got 0 |
Failed |
IN9 171
OUT1 096 |
| × |
LcbMinArraySumLiveCodeBench / leetcode / medium: minimum-array-sum1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minArraySum case0: expected 23 got 28 |
Failed |
IN9 282
OUT435 |
| ✓ |
LcbMinimumPositiveSumSubarrayLiveCodeBench / leetcode / easy: minimum-positive-sum-subarray1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 215
OUT169 |
| × |
LcbSumOfGoodSubsequencesLiveCodeBench / leetcode / hard: sum-of-good-subsequences1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: sumGood case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:20: Unexpected character '`'.
3:25: Unexpected character '`'.
5:23: Unexpected character '`'.
5:29: Unexpected character '`'.
5:84: Unexpected character '`'.
5:86: Unexpected character '`'.
8:3: Unexpected character '`'.
8:10: Unexpected character '`'.
9:3: Unexpected character '`'.
9:10: Unexpected character '`'.
11:72: Unexpected character '`'.
11:76: Unexpected character '`'.
12:27: Unexpected character '`'.
12:33: Unexpected character '`'.
13:38: Unexpected character '`'.
13:44: Unexpected character '`'.
13:49: Unexpected character '`'.
13:55: Unexpected character '`'.
15:185: Unexpected character '±'.
15:217: Unexpected character '—'.
15:237: Unexpected character '±'.
15:375: Char literals must contain exactly one character and close with '''.
17:56: Unexpected character '→'.
25:26: Unexpected character '#'.
37:94: Unexpected character '→'.
37:107: Unexpected character '→'.
37:121: Unexpected character '✓'.
42:70: Unexpected character '✓'.
44:27: Unexpected character '#'.
54:21: Unexpected character '#'.
60:12: Unexpected character '#'.
60:89: Char literals must contain exactly one character and close with '''.
66:4: Char literals must contain exactly one character and close with '''.
76:4: Char literals must contain exactly one character and close with '''.
126:124: Unexpected character '→'.
136:65: Unexpected character '#'.
136:112: Unexpected character '#'.
140:144: Unexpected character '→'.
154:2: Char literals must contain exactly one character and close with '''.
156:8: Unexpected character '#'.
156:11: Unexpected character '`'.
156:40: Unexpected character '`'.
158:10: Unexpected character '`'.
158:14: Unexpected character '`'.
168:204: Unexpected character '#'.
182:4: Char literals must contain exactly one character and close with '''.
184:20: Unexpected character '`'.
184:34: Unexpected character '`'.
211:34: Unexpected character '#'.
215:26: Unexpected character '#'.
215:34: Unexpected character '`'.
215:36: Unexpected character '`'.
235:4: Char literals must contain exactly one character and close with '''.
237:454: Unexpected character '±'.
253:48: Unexpected character '`'.
253:90: Unexpected character '`'.
259:4: Char literals must contain exactly one character and close with '''.
265:22: Unexpected character '#'.
265:25: Unexpected character '`'.
265:49: Unexpected character '`'.
267:1: Unexpected character '`'.
267:9: Unexpected character '`'.
267:33: Unexpected character '→'.
269:1: Unexpected character '`'.
269:10: Unexpected character '`'.
271:1: Unexpected character '`'.
271:24: Unexpected character '`'.
287:2: Char literals must contain exactly one character and close with '''.
297:4: Char literals must contain exactly one character and close with '''.
348:5: Char literals must contain exactly one character and close with '''.
362:259: Char literals must contain exactly one character and close with '''.
382:112: Unexpected character '`'.
382:118: Unexpected character '`'.
382:346: Char literals must contain exactly one character and close with '''.
398:4: Char literals must contain exactly one character and close with '''.
400:2: Char literals must contain exactly one character and close with '''.
404:12: Unexpected character '#'.
404:62: Unexpected character '`'.
404:70: Unexpected character '`'.
404:142: Unexpected character '`'.
404:177: Unexpected character '`'.
415:6: Unexpected character '`'.
415:18: Unexpected character '`'.
415:29: Unexpected character '→'.
429:4: Char literals must contain exactly one character and close with '''.
485:2: Char literals must contain exactly one character and close with '''.
491:4: Char literals must contain exactly one character and close with '''.
503:12: Char literals must contain exactly one character and close with '''.
545:67: Char literals must contain exactly one character and close with '''.
557:2: Char literals must contain exactly one character and close with '''.
599:11: Char liter ... (message truncated) |
Failed |
IN9 158
OUT13 573 |
Code Generation - LiveCodeBench v5 LeetCode (set 3)
6 passed5 failed |
54.5% |
IN101 333
OUT8 968 |
| × |
LcbMaxCollectedFruitsLiveCodeBench / leetcode / hard: find-the-maximum-number-of-fruits-collected (id 3648)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxCollectedFruits case0 runtime error: List index -9 is out of range. The list has 81 element(s).
Values:
run.Success = false |
Failed |
IN9 358
OUT1 258 |
| ✓ |
LcbMaxFrequencyOpsLiveCodeBench / leetcode / medium: maximum-frequency-of-an-element-after-performing-operations-i (id 3622)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 177
OUT172 |
| × |
LcbMaxRemovalsSourceLiveCodeBench / leetcode / medium: find-maximum-removals-from-source-string (id 3487)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxRemovals case0: expected 1 got 2 |
Failed |
IN9 275
OUT2 301 |
| × |
LcbMaxScoreTouristLiveCodeBench / leetcode / medium: maximum-points-tourist-can-earn (id 3587)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxScore case0 runtime error: List index 1 is out of range. The list has 1 element(s).
Values:
run.Success = false |
Failed |
IN9 267
OUT288 |
| ✓ |
LcbMinBitwiseArrayLiveCodeBench / leetcode / easy: construct-the-minimum-bitwise-array-i (id 3605)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 189
OUT3 415 |
| × |
LcbNumberOfWaysEventLiveCodeBench / leetcode / hard: find-the-number-of-possible-ways-for-an-event (id 3604)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfWays case0: expected 6 got 0 |
Failed |
IN9 216
OUT379 |
| ✓ |
LcbPossibleStringCountLiveCodeBench / leetcode / easy: find-the-original-typed-string-i (id 3617)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 135
OUT126 |
| ✓ |
LcbShiftDistanceLiveCodeBench / leetcode / medium: shift-distance-between-two-strings (id 3591)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 203
OUT355 |
| ✓ |
LcbSmallestDivisibleDigitProductLiveCodeBench / leetcode / easy: smallest-divisible-digit-product-i (id 3626)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 125
OUT179 |
| ✓ |
LcbStoneRemovalGameLiveCodeBench / leetcode / easy: stone-removal-game (id 3625)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 134
OUT81 |
| × |
LcbZeroArrayTransformIIILiveCodeBench / leetcode / medium: zero-array-transformation-iii (id 3647)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxRemoval case0: expected 1 got -1 |
Failed |
IN9 254
OUT414 |
Code Generation - LiveCodeBench v5 LeetCode (set 4)
3 passed7 failed |
30.0% |
IN91 869
OUT18 439 |
| × |
LcbButtonLongestTimeLiveCodeBench / leetcode / easy: button-with-longest-push-time (id 3632)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: buttonWithLongestTime case1: expected 10 got 1 |
Failed |
IN9 204
OUT234 |
| ✓ |
LcbCountComponentsLcmLiveCodeBench / leetcode / hard: count-connected-components-in-lcm-graph (id 3680)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 173
OUT1 167 |
| × |
LcbDigitOpsEqualLiveCodeBench / leetcode / medium: digit-operations-to-make-two-integers-equal (id 3655)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: digitOpsEqual case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
5:6: Unexpected character '—'.
5:178: Unexpected character '±'.
5:205: Unexpected character '±'.
5:400: Unexpected character '—'.
7:103: Unexpected character '±'.
7:201: Char literals must contain exactly one character and close with '''.
7:401: Unexpected character '→'.
7:417: Char literals must contain exactly one character and close with '''.
9:22: Unexpected character '→'.
9:46: Unexpected character '±'.
9:54: Char literals must contain exactly one character and close with '''.
11:199: Unexpected character '→'.
11:230: Unexpected character '—'.
13:33: Unexpected character '→'.
13:41: Unexpected character '→'.
13:105: Unexpected character '→'.
13:135: Unexpected character '⇒'.
13:190: Unexpected character '⇒'.
13:195: Char literals must contain exactly one character and close with '''.
15:204: Unexpected character '≤'.
15:234: Unexpected character '≤'.
15:271: Unexpected character '≤'.
15:312: Unexpected character '—'.
15:418: Unexpected character '≤'.
15:433: Char literals must contain exactly one character and close with '''.
17:2: Char literals must contain exactly one character and close with '''.
19:103: Unexpected character '≤'.
21:139: Char literals must contain exactly one character and close with '''.
37:11: Unexpected character '≥'.
37:20: Unexpected character '≤'.
75:41: Unexpected character '…'.
85:4: Char literals must contain exactly one character and close with '''.
95:53: Unexpected character '≥'.
97:182: Unexpected character '≥'.
105:4: Char literals must contain exactly one character and close with '''.
197:4: Char literals must contain exactly one character and close with '''.
221:4: Char literals must contain exactly one character and close with '''.
237:4: Char literals must contain exactly one character and close with '''.
247:179: Char literals must contain exactly one character and close with '''.
249:14: Char literals must contain exactly one character and close with '''.
265:15: Unexpected character '→'.
265:96: Unexpected character '→'.
265:130: Unexpected character '→'.
265:135: Unexpected character '✓'.
265:148: Unexpected character '✓'.
269:7: Unexpected character '→'.
269:94: Unexpected character '→'.
269:97: Unexpected character '→'.
269:100: Unexpected character '→'.
269:144: Unexpected character '→'.
269:147: Unexpected character '→'.
269:150: Unexpected character '→'.
269:153: Unexpected character '→'.
269:156: Unexpected character '→'.
275:4: Char literals must contain exactly one character and close with '''.
277:38: Unexpected character '→'.
279:227: Unexpected character '→'.
279:230: Unexpected character '→'.
279:233: Unexpected character '→'.
279:259: Unexpected character '→'.
281:33: Unexpected character '⇒'.
281:106: Unexpected character '⇒'.
281:206: Unexpected character '→'.
281:209: Unexpected character '→'.
281:212: Unexpected character '→'.
281:215: Unexpected character '→'.
281:234: Unexpected character '→'.
285:135: Unexpected character '±'.
291:4: Char literals must contain exactly one character and close with '''.
299:52: Char literals must contain exactly one character and close with '''.
301:111: Char literals must contain exactly one character and close with '''.
303:12: Unexpected character '→'.
305:2: Char literals must contain exactly one character and close with '''.
402:36: Unexpected character '≥'.
402:48: Unexpected character '≥'.
402:83: Unexpected character '≥'.
404:46: Unexpected character '→'.
410:46: Unexpected character '≤'.
414:69: Unexpected character '≤'.
438:2: Char literals must contain exactly one character and close with '''.
440:16: Unexpected character '#'.
466:21: Unexpected character '→'.
474:66: Unexpected character '—'.
488:2: Char literals must contain exactly one character and close with '''.
494:19: Char literals must contain exactly one character and close with '''.
538:44: Unexpected character ... (message truncated) |
Failed |
IN9 199
OUT12 971 |
| × |
LcbLargestOutlierLiveCodeBench / leetcode / medium: identify-the-largest-outlier-in-an-array (id 3594)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: getLargestOutlier case0: expected 10 got 5 |
Failed |
IN9 198
OUT189 |
| × |
LcbMaxRectangleAreaLiveCodeBench / leetcode / medium: maximum-area-rectangle-with-point-constraints-i (id 3681)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxRectangleArea case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:46: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 212
OUT611 |
| × |
LcbMaxSubarraySumDivKLiveCodeBench / leetcode / medium: maximum-subarray-sum-with-length-divisible-by-k (id 3653)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxSubarraySum case0: expected 3 got 2 |
Failed |
IN9 148
OUT151 |
| × |
LcbMinOpsEqualKLiveCodeBench / leetcode / easy: minimum-operations-to-make-array-values-equal-to-k (id 3621)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsEqualK case2: expected 4 got 5 |
Failed |
IN9 187
OUT246 |
| × |
LcbMinTimeBreakLocksLiveCodeBench / leetcode / medium: minimum-time-to-break-locks-i (id 3649)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: findMinimumTime case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:43: Unexpected character '→'.
25:152: Char literals must contain exactly one character and close with '''.
27:133: Char literals must contain exactly one character and close with '''.
29:114: Unexpected character '≥'.
29:117: Unexpected character '→'.
29:159: Unexpected character '≥'.
29:162: Unexpected character '→'.
29:174: Unexpected character '≥'.
29:212: Unexpected character '→'.
29:242: Unexpected character '≥'.
29:245: Unexpected character '→'.
33:36: Unexpected character '→'.
33:52: Unexpected character '→'.
33:68: Unexpected character '→'.
35:36: Char literals must contain exactly one character and close with '''.
43:362: Unexpected character '→'.
43:734: Unexpected character '→'.
45:74: Char literals must contain exactly one character and close with '''.
49:16: Unexpected character '#'.
53:24: Unexpected character '→'.
71:19: Unexpected character '#'.
85:4: Char literals must contain exactly one character and close with '''.
103:5: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 182
OUT1 685 |
| ✓ |
LcbSmallestAllSetBitsLiveCodeBench / leetcode / easy: smallest-number-with-all-set-bits (id 3676)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 121
OUT60 |
| ✓ |
LcbTransformedArrayLiveCodeBench / leetcode / easy: transformed-array (id 3651)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 245
OUT1 125 |
Code Generation - LiveCodeBench v5 LeetCode (set 5)
4 passed6 failed |
40.0% |
IN91 765
OUT4 997 |
| ✓ |
LcbBeautifulSplitsLiveCodeBench / leetcode / medium: count beautiful splits in an array (LCB id 3686)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 146
OUT309 |
| × |
LcbCountGoodArraysLiveCodeBench / leetcode / hard: count arrays with k matching adjacent elements (LCB id 3682)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: countGoodArrays case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 183
OUT356 |
| ✓ |
LcbCountLen3SubarraysLiveCodeBench / leetcode / easy: count length-3 subarrays with a condition (LCB id 3685)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 115
OUT114 |
| ✓ |
LcbCountSpecialSubsequencesLiveCodeBench / leetcode / medium: count special subsequences (LCB id 3699)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 197
OUT184 |
| × |
LcbLargestBoxStringLiveCodeBench / leetcode / medium: lexicographically largest string from the box I (LCB id 3683)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: answerString case1: expected "g" got "" |
Failed |
IN9 147
OUT158 |
| × |
LcbMakeStringGoodLiveCodeBench / leetcode / hard: minimum operations to make character frequencies equal (LCB id 3638)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: makeStringGood case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:18: Unexpected character '`'.
3:20: Unexpected character '`'.
3:49: Unexpected character '—'.
6:66: Unexpected character '→'.
8:6: Unexpected character '—'.
8:174: Char literals must contain exactly one character and close with '''.
8:187: Char literals must contain exactly one character and close with '''.
11:10: Unexpected character '→'.
11:28: Unexpected character '×'.
11:33: Unexpected character '×'.
11:38: Unexpected character '×'.
11:128: Unexpected character '→'.
12:10: Unexpected character '→'.
12:16: Unexpected character '×'.
12:21: Unexpected character '×'.
13:11: Unexpected character '→'.
13:17: Unexpected character '×'.
13:22: Unexpected character '×'.
13:27: Unexpected character '×'.
13:57: Unexpected character '→'.
13:71: Char literals must contain exactly one character and close with '''.
15:86: Char literals must contain exactly one character and close with '''.
21:212: Char literals must contain exactly one character and close with '''.
23:23: Unexpected character '×'.
23:27: Unexpected character '×'.
23:31: Unexpected character '×'.
23:59: Char literals must contain exactly one character and close with '''.
23:130: Unexpected character '→'.
23:132: Unexpected character '→'.
31:303: Unexpected character '—'.
31:485: Unexpected character '—'.
37:181: Unexpected character '≥'.
39:5: Char literals must contain exactly one character and close with '''.
41:66: Unexpected character '≥'.
41:252: Unexpected character '≤'.
43:19: Char literals must contain exactly one character and close with '''.
49:21: Char literals must contain exactly one character and close with '''.
51:26: Unexpected character '≤'.
53:31: Char literals must contain exactly one character and close with '''.
55:2: Char literals must contain exactly one character and close with '''.
59:16: Char literals must contain exactly one character and close with '''.
63:2: Char literals must contain exactly one character and close with '''.
67:2: Char literals must contain exactly one character and close with '''.
71:1103: Unexpected character '—'.
73:54: Char literals must contain exactly one character and close with '''.
73:248: Unexpected character '—'.
Values:
run.Compiled = false |
Failed |
IN9 151
OUT2 151 |
| ✓ |
LcbMaxCurrencyAmountLiveCodeBench / leetcode / medium: maximize amount after two days of conversions (LCB id 3613)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 327
OUT645 |
| × |
LcbMaxDistinctElementsLiveCodeBench / leetcode / medium: maximum number of distinct elements after operations (LCB id 3620)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxDistinctElements case0: expected 6 got 4 |
Failed |
IN9 154
OUT456 |
| × |
LcbMinIdenticalSubstringLiveCodeBench / leetcode / hard: smallest substring with identical characters I (LCB id 3690)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: minLength case0 runtime error: String index 6 is out of range. The string has 6 character(s).
Values:
run.Success = false |
Failed |
IN9 176
OUT486 |
| × |
LcbMinOpsDistinctLiveCodeBench / leetcode / easy: minimum operations to make elements distinct (LCB id 3656)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minimumOperations case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:18: Expected a name, found '('.
Values:
run.Compiled = false |
Failed |
IN9 169
OUT138 |
Code Generation - LiveCodeBench v5 LeetCode (set 6)
2 passed8 failed |
20.0% |
IN92 060
OUT53 101 |
| × |
LcbCheckValidCutsLiveCodeBench / leetcode / medium: check if grid can be cut into sections (LCB id 3657)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: checkValidCuts case0: expected true got false |
Failed |
IN9 265
OUT1 139 |
| × |
LcbCountXorPathsLiveCodeBench / leetcode / medium: count paths with the given xor value (LCB id 3659)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countPathsWithXorValue case0: expected 3 got 6 |
Failed |
IN9 250
OUT308 |
| × |
LcbLongestDecreasingDiffLiveCodeBench / leetcode / medium: longest subsequence with decreasing adjacent difference (LCB id 3716)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: longestSubsequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:9: Unknown variable 'dp'.
10:13: Unknown variable 'dp'.
10:43: Unknown variable 'dp'.
16:28: Unknown variable 'dp'.
17:28: Unknown variable 'dp'.
17:40: Unknown variable 'dp'.
22:17: Unknown variable 'dp'.
22:43: Unknown variable 'dp'.
Values:
run.Compiled = false |
Failed |
IN9 206
OUT272 |
| × |
LcbMaxCoinsKBagsLiveCodeBench / leetcode / medium: maximum coins from k consecutive bags (LCB id 3715)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maximumCoins case0 runtime error: List index 6 is out of range. The list has 6 element(s).
Values:
run.Success = false |
Failed |
IN9 195
OUT549 |
| × |
LcbMaxIntervalScoreLiveCodeBench / leetcode / hard: maximum score of non-overlapping intervals (LCB id 3562)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maximumWeight case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
17:66: Unexpected character '`'.
17:82: Unexpected character '`'.
25:2: Char literals must contain exactly one character and close with '''.
29:26: Unexpected character '`'.
29:45: Unexpected character '`'.
37:301: Char literals must contain exactly one character and close with '''.
45:16: Unexpected character '`'.
45:22: Unexpected character '`'.
51:2: Char literals must contain exactly one character and close with '''.
53:48: Unexpected character '`'.
53:50: Unexpected character '`'.
199:5: Char literals must contain exactly one character and close with '''.
217:27: Unexpected character '`'.
217:33: Unexpected character '`'.
225:2: Char literals must contain exactly one character and close with '''.
227:359: Char literals must contain exactly one character and close with '''.
247:2: Char literals must contain exactly one character and close with '''.
257:114: Unexpected character '—'.
273:2: Char literals must contain exactly one character and close with '''.
275:34: Char literals must contain exactly one character and close with '''.
285:490: Char literals must contain exactly one character and close with '''.
293:18: Unexpected character '`'.
293:33: Unexpected character '`'.
313:2: Char literals must contain exactly one character and close with '''.
475:5: Char literals must contain exactly one character and close with '''.
481:43: Unexpected character '`'.
481:75: Unexpected character '`'.
481:89: Unexpected character '`'.
481:105: Unexpected character '`'.
481:107: Unexpected character '–'.
497:2: Char literals must contain exactly one character and close with '''.
505:2: Char literals must contain exactly one character and close with '''.
527:2: Char literals must contain exactly one character and close with '''.
529:25: Unexpected character '`'.
529:34: Unexpected character '`'.
539:36: Unexpected character '`'.
539:38: Unexpected character '`'.
539:50: Unexpected character '`'.
539:55: Unexpected character '`'.
551:108: Unexpected character '–'.
587:2: Char literals must contain exactly one character and close with '''.
589:40: Unexpected character '#'.
597:2: Char literals must contain exactly one character and close with '''.
731:5: Char literals must contain exactly one character and close with '''.
733:6: Char literals must contain exactly one character and close with '''.
745:34: Unexpected character '`'.
745:40: Unexpected character '`'.
745:74: Unexpected character '`'.
745:87: Unexpected character '`'.
751:2: Char literals must contain exactly one character and close with '''.
769:44: Unexpected character '#'.
769:56: Unexpected character '`'.
769:62: Unexpected character '`'.
769:73: Unexpected character '–'.
775:2: Char literals must contain exactly one character and close with '''.
785:43: Unexpected character '`'.
785:62: Unexpected character '`'.
799:2: Char literals must contain exactly one character and close with '''.
813:72: Unexpected character '`'.
813:86: Unexpected character '`'.
815:11: Unexpected character '`'.
815:21: Unexpected character '`'.
821:2: Char literals must contain exactly one character and close with '''.
839:2: Char literals must contain exactly one character and close with '''.
847:21: Unexpected character '`'.
847:39: Unexpected character '`'.
897:2: Char literals must contain exactly one character and close with '''.
905:34: Char literals must contain exactly one character and close with '''.
919:2: Char literals must contain exactly one character and close with '''.
923:43: Unexpected character '`'.
923:52: Unexpected character '`'.
923:111: Unexpected character '`'.
923:126: Unexpected character '`'.
931:2: Char literals must contain exactly one character and close with '''.
949:63: Char literals must contain exactly one character and close with '''.
965:137: Char literals must contain exactly one character and close with '''.
973:2: Char literals must contain exactly one character and close with '''. ... (message truncated) |
Failed |
IN9 259
OUT43 153 |
| × |
LcbMaxProductEquivalentLiveCodeBench / leetcode / easy: maximum subarray with equal products (LCB id 3702)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxLength case0: expected 5 got 4 |
Failed |
IN9 183
OUT433 |
| ✓ |
LcbMaxSubarrayAfterRemovalLiveCodeBench / leetcode / hard: maximize subarray sum after removing all occurrences of one element (LCB id 3688)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 154
OUT344 |
| × |
LcbMiddleModeSubsequencesLiveCodeBench / leetcode / hard: subsequences with a unique middle mode I (LCB id 3700)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: subsequencesWithMiddleMode case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:105: Unexpected character '≠'.
7:114: Unexpected character '≤'.
7:173: Unexpected character '≥'.
7:379: Char literals must contain exactly one character and close with '''.
10:80: Unexpected character '✓'.
11:35: Unexpected character '→'.
12:24: Unexpected character '→'.
13:28: Unexpected character '→'.
14:40: Unexpected character '→'.
14:59: Unexpected character '✗'.
15:31: Unexpected character '→'.
16:31: Unexpected character '→'.
16:49: Unexpected character '✗'.
26:122: Char literals must contain exactly one character and close with '''.
30:21: Unexpected character '≠'.
30:33: Unexpected character '≤'.
30:211: Unexpected character '≠'.
30:219: Char literals must contain exactly one character and close with '''.
36:62: Unexpected character '≤'.
36:65: Unexpected character '→'.
36:153: Unexpected character '≥'.
38:46: Unexpected character '≤'.
40:46: Unexpected character '≤'.
40:82: Unexpected character '→'.
40:101: Unexpected character '≤'.
40:120: Unexpected character '→'.
40:187: Unexpected character '≤'.
42:55: Unexpected character '≤'.
46:33: Unexpected character '∈'.
49:7: Unexpected character '≥'.
57:61: Unexpected character '∈'.
61:51: Char literals must contain exactly one character and close with '''.
64:11: Unexpected character '→'.
65:7: Unexpected character '→'.
66:4: Unexpected character '≥'.
66:7: Unexpected character '→'.
68:56: Char literals must contain exactly one character and close with '''.
70:53: Unexpected character '≥'.
76:94: Char literals must contain exactly one character and close with '''.
96:51: Char literals must contain exactly one character and close with '''.
101:55: Unexpected character '#'.
104:14: Unexpected character 'Σ'.
104:34: Unexpected character '×'.
106:41: Unexpected character '#'.
106:43: Unexpected character '∈'.
121:180: Char literals must contain exactly one character and close with '''.
125:133: Char literals must contain exactly one character and close with '''.
137:51: Unexpected character '≠'.
138:22: Char literals must contain exactly one character and close with '''.
144:50: Unexpected character 'Σ'.
145:38: Unexpected character 'Σ'.
145:49: Unexpected character '≠'.
149:21: Char literals must contain exactly one character and close with '''.
151:76: Unexpected character '—'.
153:130: Char literals must contain exactly one character and close with '''.
167:182: Char literals must contain exactly one character and close with '''.
169:91: Char literals must contain exactly one character and close with '''.
177:19: Unexpected character 'Σ'.
178:29: Unexpected character 'Σ'.
180:315: Unexpected character 'Σ'.
180:393: Unexpected character '≠'.
180:649: Unexpected character 'Σ'.
184:28: Unexpected character '#'.
184:44: Unexpected character '#'.
200:89: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 211
OUT4 591 |
| ✓ |
LcbMirrorScoreLiveCodeBench / leetcode / medium: find mirror score of a string (LCB id 3634)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 184
OUT2 059 |
| × |
LcbSubstringPatternLiveCodeBench / leetcode / easy: substring matching pattern (LCB id 3684)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: hasMatch case0: expected true got false |
Failed |
IN9 153
OUT253 |
Code Generation - LiveCodeBench v6 AtCoder (set 1)
7 passed9 failed |
43.8% |
IN160 630
OUT11 855 |
| × |
LcbComingOfAgeLiveCodeBench / atcoder abc388_d / medium: Coming of Age Celebration1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ComingOfAge case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:76: Unexpected character '≥'.
5:126: Char literals must contain exactly one character and close with '''.
6:58: Unexpected character '≥'.
13:61: Unexpected character '—'.
13:77: Char literals must contain exactly one character and close with '''.
15:77: Unexpected character '≥'.
25:133: Char literals must contain exactly one character and close with '''.
29:76: Char literals must contain exactly one character and close with '''.
35:11: Char literals must contain exactly one character and close with '''.
45:356: Unexpected character '≤'.
45:459: Char literals must contain exactly one character and close with '''.
51:9: Char literals must contain exactly one character and close with '''.
65:277: Unexpected character '→'.
65:323: Unexpected character '≥'.
65:333: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 208
OUT2 178 |
| × |
LcbCountArraysLiveCodeBench / atcoder abc387_f / hard: Count Arrays1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountArrays case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:13: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 230
OUT422 |
| × |
LcbDangerousSugorokuLiveCodeBench / atcoder abc388_f / hard: Dangerous Sugoroku1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DangerousSugoroku case2: expected [Yes] got [No] |
Failed |
IN22 636
OUT5 005 |
| ✓ |
LcbDigitProductLiveCodeBench / atcoder abc389_a / easy: 9x91 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 148
OUT62 |
| ✓ |
LcbFactorialInverseLiveCodeBench / atcoder abc389_b / easy: tcaF1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 154
OUT71 |
| ✓ |
LcbHeavySnakeLiveCodeBench / atcoder abc388_b / easy: Heavy Snake1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 193
OUT256 |
| × |
LcbKagamimochi2LiveCodeBench / atcoder abc388_g / hard: Simultaneous Kagamimochi 21 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Kagamimochi2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:208: Unexpected character '≤'.
3:258: Unexpected character '≤'.
5:74: Char literals must contain exactly one character and close with '''.
5:460: Char literals must contain exactly one character and close with '''.
7:9: Char literals must contain exactly one character and close with '''.
9:73: Char literals must contain exactly one character and close with '''.
17:211: Char literals must contain exactly one character and close with '''.
23:21: Char literals must contain exactly one character and close with '''.
23:240: Char literals must contain exactly one character and close with '''.
25:29: Unexpected character '—'.
25:41: Char literals must contain exactly one character and close with '''.
27:93: Char literals must contain exactly one character and close with '''.
29:34: Char literals must contain exactly one character and close with '''.
33:2: Char literals must contain exactly one character and close with '''.
41:162: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 257
OUT1 261 |
| ✓ |
LcbNotXSumLiveCodeBench / atcoder abc387_b / easy: 9x9 Sum1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 209
OUT129 |
| × |
LcbOddEvenGraphLiveCodeBench / atcoder abc389_g / hard: Odd Even Graph1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OddEvenGraph case0: expected [12 9 3 0] got [36 216 1296 7776] |
Failed |
IN9 244
OUT783 |
| ✓ |
LcbQuestionUpcLiveCodeBench / atcoder abc388_a / easy: ?UPC1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 154
OUT37 |
| ✓ |
LcbRatedRangeLiveCodeBench / atcoder abc389_f / hard: Rated Range1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 203
OUT294 |
| × |
LcbSimultaneousKagamimochiLiveCodeBench / atcoder abc388_e / hard: Simultaneous Kagamimochi1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SimultaneousKagamimochi case0: expected [3] got [2] |
Failed |
IN9 196
OUT176 |
| × |
LcbSnakeNumbersLiveCodeBench / atcoder abc387_c / medium: Snake Numbers1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SnakeNumbers case0: expected [6] got [-69] |
Failed |
IN9 203
OUT386 |
| ✓ |
LcbSquareOfSumLiveCodeBench / atcoder abc387_a / easy: Happy New Year 20251 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 135
OUT65 |
| × |
LcbSquaresInCircleLiveCodeBench / atcoder abc389_d / medium: Squares in Circle1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SquaresInCircle case0: expected [5] got [25] |
Failed |
IN9 234
OUT552 |
| × |
LcbVariousKagamimochiLiveCodeBench / atcoder abc388_c / medium: Various Kagamimochi1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: VariousKagamimochi case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 226
OUT178 |
Code Generation - LiveCodeBench v6 AtCoder (set 2)
8 passed7 failed |
53.3% |
IN175 355
OUT33 776 |
| ✓ |
LcbDoubleSum3LiveCodeBench / atcoder abc390_f / hard: Double Sum 31 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN35 311
OUT3 063 |
| ✓ |
LcbGeometricSequenceLiveCodeBench / atcoder abc390_b / easy: Geometric Sequence1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 164
OUT1 220 |
| × |
LcbGravityLiveCodeBench / atcoder abc391_d / medium: Gravity1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Gravity case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:62: Char literals must contain exactly one character and close with '''.
13:56: Char literals must contain exactly one character and close with '''.
17:43: Char literals must contain exactly one character and close with '''.
21:103: Char literals must contain exactly one character and close with '''.
35:162: Char literals must contain exactly one character and close with '''.
41:112: Unexpected character '≥'.
43:101: Unexpected character '≥'.
45:28: Char literals must contain exactly one character and close with '''.
71:50: Char literals must contain exactly one character and close with '''.
77:153: Unexpected character '≥'.
81:32: Unexpected character '≥'.
83:80: Char literals must contain exactly one character and close with '''.
85:106: Unexpected character '≥'.
85:172: Char literals must contain exactly one character and close with '''.
87:90: Unexpected character '→'.
87:292: Unexpected character '≥'.
87:351: Unexpected character '≥'.
91:119: Unexpected character '→'.
91:211: Unexpected character '→'.
93:71: Unexpected character '≥'.
93:100: Unexpected character '≥'.
93:114: Unexpected character '→'.
93:142: Unexpected character '→'.
93:171: Unexpected character '→'.
95:40: Unexpected character '≥'.
97:94: Unexpected character '≥'.
99:110: Unexpected character '→'.
108:85: Unexpected character '≤'.
114:87: Char literals must contain exactly one character and close with '''.
128:14: Char literals must contain exactly one character and close with '''.
132:153: Unexpected character '≤'.
132:184: Char literals must contain exactly one character and close with '''.
134:33: Unexpected character 'τ'.
134:81: Unexpected character 'τ'.
136:1: Unexpected character 'τ'.
138:77: Unexpected character '’'.
140:2: Char literals must contain exactly one character and close with '''.
144:29: Char literals must contain exactly one character and close with '''.
150:2: Char literals must contain exactly one character and close with '''.
160:248: Char literals must contain exactly one character and close with '''.
162:57: Char literals must contain exactly one character and close with '''.
166:134: Char literals must contain exactly one character and close with '''.
170:2: Char literals must contain exactly one character and close with '''.
172:252: Unexpected character '→'.
172:484: Char literals must contain exactly one character and close with '''.
174:54: Char literals must contain exactly one character and close with '''.
195:93: Char literals must contain exactly one character and close with '''.
197:52: Unexpected character '≤'.
199:64: Char literals must contain exactly one character and close with '''.
203:35: Unexpected character '→'.
205:4: Char literals must contain exactly one character and close with '''.
209:94: Unexpected character '–'.
Values:
run.Compiled = false |
Failed |
IN9 295
OUT7 574 |
| × |
LcbKthLargestLiveCodeBench / atcoder abc391_f / hard: K1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: KthLargest case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:19: Assignment target must be a variable or an element like 'items[0]'. Assigning to struct fields is not supported.
Values:
run.Compiled = false |
Failed |
IN9 236
OUT7 684 |
| × |
LcbMajorityVoteLiveCodeBench / atcoder abc391_e / hard: Hierarchical Majority Vote1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MajorityVote case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:23: Expected ';' after the variable declaration, found ','.
Values:
run.Compiled = false |
Failed |
IN9 222
OUT6 782 |
| × |
LcbManyLcsLiveCodeBench / atcoder abc391_g / hard: Many LCS1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ManyLcs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'Solve' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson.
Values:
run.Compiled = false |
Failed |
IN9 213
OUT5 |
| ✓ |
LcbOneSwapSortLiveCodeBench / atcoder abc390_a / easy: 124351 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 197
OUT153 |
| ✓ |
LcbOppositeDirectionLiveCodeBench / atcoder abc391_a / easy: Lucky Direction1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 174
OUT119 |
| ✓ |
LcbPaintRectangleLiveCodeBench / atcoder abc390_c / medium: Paint to make a rectangle1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 220
OUT1 294 |
| × |
LcbPermutationConcatLiveCodeBench / atcoder abc390_g / hard: Permutation Concatenation1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PermutationConcat case0: expected [1332] got [666] |
Failed |
IN9 245
OUT2 115 |
| × |
LcbSeekGridLiveCodeBench / atcoder abc391_b / easy: Seek Grid1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SeekGrid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:10: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed |
IN9 212
OUT361 |
| ✓ |
LcbShuffledEquationLiveCodeBench / atcoder abc392_a / easy: Shuffled Equation1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 189
OUT123 |
| ✓ |
LcbSquarePriceLiveCodeBench / atcoder abc389_e / hard: Square Price1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN20 259
OUT757 |
| ✓ |
LcbStoneXorLiveCodeBench / atcoder abc390_d / medium: Stone XOR1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 208
OUT1 665 |
| × |
LcbVitaminBalanceLiveCodeBench / atcoder abc390_e / hard: Vitamin Balance1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: VitaminBalance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
45:23: Function 'CanAchieve' must return 'string', but this return statement has type 'bool'.
47:23: Function 'CanAchieve' must return 'string', but this return statement has type 'bool'.
49:23: Function 'CanAchieve' must return 'string', but this return statement has type 'bool'.
50:5: Function 'CanAchieve' must return 'string', but this return statement has type 'bool'.
81:13: The if condition must be 'bool', but it has type 'string'.
Values:
run.Compiled = false |
Failed |
IN9 210
OUT861 |
Code Generation - LiveCodeBench v6 AtCoder (set 3)
5 passed5 failed |
50.0% |
IN92 287
OUT8 605 |
| ✓ |
LcbABCIntervalsLiveCodeBench / atcoder / abc393_b A..B..C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT153 |
| × |
LcbBibStaringLiveCodeBench / atcoder / abc392_c Bib (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: BibStaring case0 runtime error: List index 4 is out of range. The list has 4 element(s).
Values:
run.Success = false |
Failed |
IN9 235
OUT259 |
| × |
LcbDoublesProbabilityLiveCodeBench / atcoder / abc392_d Doubles (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DoublesProbability case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:17: Char literals must contain exactly one character and close with '''.
19:29: Char literals must contain exactly one character and close with '''.
25:2: Char literals must contain exactly one character and close with '''.
41:28: Unexpected character '#'.
115:56: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 245
OUT2 289 |
| ✓ |
LcbFineTripletsLiveCodeBench / atcoder / abc392_g Fine Triplets (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 215
OUT207 |
| × |
LcbGcdOfSubsetLiveCodeBench / atcoder / abc393_e GCD of Subset (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GcdOfSubset case0: expected [3
4
6
1
6] got [-1
-1
-1
-1
-1] |
Failed |
IN9 214
OUT649 |
| × |
LcbInsertSequenceLiveCodeBench / atcoder / abc392_f Insert (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: InsertSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '.'.
Values:
run.Compiled = false |
Failed |
IN9 230
OUT2 |
| ✓ |
LcbPoisonousOysterLiveCodeBench / atcoder / abc393_a Poisonous Oyster (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 250
OUT122 |
| ✓ |
LcbPrefixLisQueryLiveCodeBench / atcoder / abc393_f Prefix LIS Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 260
OUT4 725 |
| ✓ |
LcbSwapToGatherLiveCodeBench / atcoder / abc393_d Swap to Gather (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 213
OUT197 |
| × |
LcbWhoIsMissingLiveCodeBench / atcoder / abc392_b Who is Missing? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WhoIsMissing case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '.'.
Values:
run.Compiled = false |
Failed |
IN9 233
OUT2 |
Code Generation - LiveCodeBench v6 AtCoder (set 4)
7 passed3 failed |
70.0% |
IN117 439
OUT19 545 |
| × |
LcbAlkaneSubtreeLiveCodeBench / atcoder / abc394_f Alkane (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AlkaneSubtree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
48:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 248
OUT11 131 |
| ✓ |
LcbCatConcatLiveCodeBench / atcoder / abc394_b cat (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT334 |
| ✓ |
LcbColorfulBracketsLiveCodeBench / atcoder / abc394_d Colorful Bracket Sequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 188
OUT238 |
| ✓ |
LcbDebugWaAcLiveCodeBench / atcoder / abc394_c Debug (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 159
OUT83 |
| × |
LcbDenseBuildingsLiveCodeBench / atcoder / abc394_g Dense Buildings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DenseBuildings case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:20: Unexpected character '×'.
3:256: Unexpected character '≥'.
5:34: Char literals must contain exactly one character and close with '''.
9:348: Unexpected character '≥'.
9:399: Unexpected character '—'.
9:460: Unexpected character '≥'.
9:681: Char literals must contain exactly one character and close with '''.
9:812: Unexpected character '—'.
9:1110: Unexpected character '≥'.
9:1269: Char literals must contain exactly one character and close with '''.
11:187: Unexpected character '→'.
11:303: Unexpected character '≥'.
11:650: Unexpected character '≥'.
11:723: Unexpected character '→'.
11:848: Unexpected character '≤'.
13:422: Char literals must contain exactly one character and close with '''.
14:59: Unexpected character '→'.
14:108: Unexpected character '∩'.
14:141: Unexpected character '→'.
14:175: Unexpected character '→'.
14:186: Unexpected character '→'.
14:218: Unexpected character '≥'.
14:254: Unexpected character '≥'.
14:291: Unexpected character '≥'.
16:48: Char literals must contain exactly one character and close with '''.
18:98: Unexpected character '≥'.
18:275: Unexpected character '≤'.
18:286: Char literals must contain exactly one character and close with '''.
20:148: Char literals must contain exactly one character and close with '''.
22:50: Unexpected character '≥'.
24:270: Unexpected character '≥'.
24:287: Unexpected character '≥'.
24:313: Unexpected character '⇒'.
28:20: Unexpected character '≥'.
Values:
run.Compiled = false |
Failed |
IN9 380
OUT2 595 |
| ✓ |
LcbFilterTwosLiveCodeBench / atcoder / abc394_a 22222 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 151
OUT79 |
| ✓ |
LcbMakeTargetLiveCodeBench / atcoder / abc395_b Make Target (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 263
OUT1 174 |
| × |
LcbPalindromicShortestPathLiveCodeBench / atcoder / abc394_e Palindromic Shortest Path (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PalindromicShortestPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:169: Char literals must contain exactly one character and close with '''.
7:107: Char literals must contain exactly one character and close with '''.
15:98: Unexpected character '→'.
15:99: Unexpected character '…'.
15:100: Unexpected character '→'.
17:197: Char literals must contain exactly one character and close with '''.
19:195: Char literals must contain exactly one character and close with '''.
21:43: Unexpected character '→'.
21:77: Unexpected character '→'.
21:124: Unexpected character '→'.
21:138: Unexpected character '→'.
21:154: Unexpected character '→'.
23:57: Unexpected character '≤'.
23:73: Unexpected character '·'.
23:90: Char literals must contain exactly one character and close with '''.
25:419: Unexpected character '≤'.
27:145: Char literals must contain exactly one character and close with '''.
31:5: Unexpected character '—'.
31:464: Unexpected character '→'.
31:535: Unexpected character '∧'.
31:549: Unexpected character '∧'.
31:577: Unexpected character '·'.
31:581: Unexpected character '·'.
31:714: Unexpected character '≤'.
33:21: Char literals must contain exactly one character and close with '''.
35:94: Char literals must contain exactly one character and close with '''.
35:195: Char literals must contain exactly one character and close with '''.
37:22: Char literals must contain exactly one character and close with '''.
37:215: Char literals must contain exactly one character and close with '''.
39:35: Unexpected character '×'.
45:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 314
OUT1 389 |
| ✓ |
LcbShortestDuplicateSubarrayLiveCodeBench / atcoder / abc395_c Shortest Duplicate Subarray (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 216
OUT163 |
| ✓ |
LcbSmoothOcclusionLiveCodeBench / atcoder / abc395_f Smooth Occlusion (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN34 345
OUT2 359 |
Code Generation - LiveCodeBench v6 AtCoder (set 5)
4 passed11 failed1 skipped |
26.7% |
IN159 981
OUT46 620 |
| × |
LcbBuyBallsLiveCodeBench / atcoder / medium: Buy Balls (abc396_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BuyBalls case0: expected [19] got [16] |
Failed |
IN9 231
OUT371 |
| × |
LcbCardPileLiveCodeBench / atcoder / easy: Card Pile (abc396_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CardPile case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:25: Type 'string' has no supported method named 'TrimEnd'.
Values:
run.Compiled = false |
Failed |
IN9 210
OUT160 |
| – |
LcbCubesLiveCodeBench / atcoder / hard: Cubes (abc397_d)0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbFlipEdgeLiveCodeBench / atcoder / hard: Flip Edge (abc395_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 224
OUT775 |
| × |
LcbFlipRowOrColLiveCodeBench / atcoder / hard: Flip Row or Col (abc396_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: FlipRowOrCol case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:13: Unexpected character '×'.
10:54: Unexpected character '∈'.
10:68: Unexpected character '∈'.
12:13: Unexpected character 'Σ'.
16:26: Unexpected character '×'.
16:96: Char literals must contain exactly one character and close with '''.
20:49: Unexpected character 'Σ'.
27:134: Unexpected character '≤'.
27:139: Unexpected character '→'.
27:146: Unexpected character '≈'.
29:14: Unexpected character '—'.
29:20: Char literals must contain exactly one character and close with '''.
29:233: Char literals must contain exactly one character and close with '''.
29:393: Char literals must contain exactly one character and close with '''.
31:131: Char literals must contain exactly one character and close with '''.
31:251: Unexpected character '≤'.
33:174: Unexpected character '≤'.
35:31: Unexpected character '→'.
37:20: Char literals must contain exactly one character and close with '''.
39:1: Unexpected character 'Σ'.
39:27: Unexpected character 'Σ'.
39:146: Unexpected character '#'.
41:152: Unexpected character '≈'.
41:171: Unexpected character '→'.
43:136: Char literals must contain exactly one character and close with '''.
45:132: Unexpected character '#'.
45:158: Unexpected character '#'.
47:117: Unexpected character 'Σ'.
51:29: Char literals must contain exactly one character and close with '''.
63:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 220
OUT2 095 |
| × |
LcbMaximizeDistanceLiveCodeBench / atcoder / hard: Maximize Distance (abc397_g)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MaximizeDistance case0: expected [1] got [0] |
Failed |
IN20 104
OUT5 199 |
| × |
LcbMinRestrictedSumLiveCodeBench / atcoder / hard: Min of Restricted Sum (abc396_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinRestrictedSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:114: Char literals must contain exactly one character and close with '''.
10:24: Unexpected character '≥'.
12:115: Char literals must contain exactly one character and close with '''.
18:97: Unexpected character '→'.
24:73: Char literals must contain exactly one character and close with '''.
26:2: Char literals must contain exactly one character and close with '''.
32:121: Unexpected character '→'.
34:145: Char literals must contain exactly one character and close with '''.
44:13: Unexpected character '#'.
50:64: Char literals must contain exactly one character and close with '''.
62:4: Char literals must contain exactly one character and close with '''.
74:4: Char literals must contain exactly one character and close with '''.
95:38: Unexpected character '≥'.
97:4: Char literals must contain exactly one character and close with '''.
115:4: Char literals must contain exactly one character and close with '''.
147:16: Unexpected character '#'.
147:124: Char literals must contain exactly one character and close with '''.
149:2: Char literals must contain exactly one character and close with '''.
181:4: Char literals must contain exactly one character and close with '''.
225:4: Char literals must contain exactly one character and close with '''.
237:125: Char literals must contain exactly one character and close with '''.
247:4: Char literals must contain exactly one character and close with '''.
275:4: Char literals must contain exactly one character and close with '''.
291:4: Char literals must contain exactly one character and close with '''.
293:43: Unexpected character '#'.
307:2: Char literals must contain exactly one character and close with '''.
371:4: Char literals must contain exactly one character and close with '''.
399:4: Char literals must contain exactly one character and close with '''.
401:2: Char literals must contain exactly one character and close with '''.
516:5: Char literals must contain exactly one character and close with '''.
518:2: Char literals must contain exactly one character and close with '''.
534:4: Char literals must contain exactly one character and close with '''.
546:2: Char literals must contain exactly one character and close with '''.
552:125: Char literals must contain exactly one character and close with '''.
560:17: Unexpected character '#'.
562:1: Unexpected character '`'.
562:36: Unexpected character '`'.
574:2: Char literals must contain exactly one character and close with '''.
604:2: Char literals must contain exactly one character and close with '''.
612:4: Char literals must contain exactly one character and close with '''.
616:2: Char literals must contain exactly one character and close with '''.
731:5: Char literals must contain exactly one character and close with '''.
735:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 245
OUT6 422 |
| × |
LcbMinXorPathLiveCodeBench / atcoder / medium: Minimum XOR Path (abc396_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinXorPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:34: Expected ')' to close the argument list, found 'best'.
Values:
run.Compiled = false |
Failed |
IN9 227
OUT570 |
| × |
LcbPathDecompositionLiveCodeBench / atcoder / hard: Path Decomposition of a Tree (abc397_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PathDecomposition case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
101:1: Unexpected character '`'.
101:11: Unexpected character '`'.
111:6: Unexpected character '`'.
111:15: Unexpected character '`'.
117:20: Unexpected character '`'.
117:29: Unexpected character '`'.
121:2: Char literals must contain exactly one character and close with '''.
123:46: Unexpected character '#'.
123:60: Unexpected character '–'.
Values:
run.Compiled = false |
Failed |
IN9 197
OUT16 304 |
| × |
LcbRotatedInversionsLiveCodeBench / atcoder / hard: Rotated Inversions (abc396_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RotatedInversions case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
28:25: Type 'string' has no supported method named 'TrimEnd'.
Values:
run.Compiled = false |
Failed |
IN9 233
OUT2 101 |
| ✓ |
LcbStrictlyIncreasingLiveCodeBench / atcoder / easy: Strictly Increasing? (abc395_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT125 |
| ✓ |
LcbThermometerLiveCodeBench / atcoder / easy: Thermometer (abc397_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 187
OUT87 |
| ✓ |
LcbTicketGateLogLiveCodeBench / atcoder / medium: Ticket Gate Log (abc397_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 190
OUT1 492 |
| × |
LcbTripleFourLiveCodeBench / atcoder / easy: Triple Four (abc396_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TripleFour case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:42: Type 'List<string>' has no supported method named 'ToList' with 0 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 195
OUT110 |
| × |
LcbVarietySplitEasyLiveCodeBench / atcoder / medium: Variety Split Easy (abc397_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: VarietySplitEasy case0: expected [5] got [7] |
Failed |
IN9 203
OUT310 |
| × |
LcbVarietySplitHardLiveCodeBench / atcoder / hard: Variety Split Hard (abc397_f)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: VarietySplitHard case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:5: Unexpected character '`'.
9:14: Unexpected character '`'.
9:75: Unexpected character '✓'.
10:5: Unexpected character '`'.
10:14: Unexpected character '`'.
10:77: Unexpected character '✓'.
11:5: Unexpected character '`'.
11:14: Unexpected character '`'.
11:77: Unexpected character '✓'.
12:5: Unexpected character '`'.
12:14: Unexpected character '`'.
12:79: Unexpected character '✓'.
13:5: Unexpected character '`'.
13:14: Unexpected character '`'.
13:79: Unexpected character '✓'.
14:5: Unexpected character '`'.
14:14: Unexpected character '`'.
14:81: Unexpected character '✓'.
15:5: Unexpected character '`'.
15:14: Unexpected character '`'.
15:81: Unexpected character '✓'.
16:5: Unexpected character '`'.
16:14: Unexpected character '`'.
16:83: Unexpected character '✓'.
17:5: Unexpected character '`'.
17:14: Unexpected character '`'.
17:83: Unexpected character '✓'.
18:5: Unexpected character '`'.
18:14: Unexpected character '`'.
18:85: Unexpected character '✓'.
19:5: Unexpected character '`'.
19:14: Unexpected character '`'.
19:85: Unexpected character '✓'.
20:5: Unexpected character '`'.
20:14: Unexpected character '`'.
20:87: Unexpected character '✓'.
21:5: Unexpected character '`'.
21:14: Unexpected character '`'.
21:87: Unexpected character '✓'.
22:5: Unexpected character '`'.
22:14: Unexpected character '`'.
22:91: Unexpected character '✓'.
23:5: Unexpected character '`'.
23:14: Unexpected character '`'.
23:83: Unexpected character '✓'.
24:5: Unexpected character '`'.
24:15: Unexpected character '`'.
24:87: Unexpected character '✓'.
25:5: Unexpected character '`'.
25:15: Unexpected character '`'.
25:85: Unexpected character '✓'.
26:5: Unexpected character '`'.
26:15: Unexpected character '`'.
26:88: Unexpected character '✓'.
27:5: Unexpected character '`'.
27:16: Unexpected character '`'.
27:88: Unexpected character '✓'.
28:5: Unexpected character '`'.
28:16: Unexpected character '`'.
28:91: Unexpected character '✓'.
29:5: Unexpected character '`'.
29:16: Unexpected character '`'.
29:88: Unexpected character '✓'.
30:5: Unexpected character '`'.
30:16: Unexpected character '`'.
30:91: Unexpected character '✓'.
31:5: Unexpected character '`'.
31:16: Unexpected character '`'.
31:88: Unexpected character '✓'.
32:5: Unexpected character '`'.
32:16: Unexpected character '`'.
32:91: Unexpected character '✓'.
33:5: Unexpected character '`'.
33:16: Unexpected character '`'.
33:88: Unexpected character '✓'.
34:5: Unexpected character '`'.
34:16: Unexpected character '`'.
34:91: Unexpected character '✓'.
35:5: Unexpected character '`'.
35:16: Unexpected character '`'.
35:88: Unexpected character '✓'.
36:5: Unexpected character '`'.
36:16: Unexpected character '`'.
36:91: Unexpected character '✓'.
37:5: Unexpected character '`'.
37:16: Unexpected character '`'.
37:88: Unexpected character '✓'.
38:5: Unexpected character '`'.
38:16: Unexpected character '`'.
38:91: Unexpected character '✓'.
39:5: Unexpected character '`'.
39:16: Unexpected character '`'.
39:88: Unexpected character '✓'.
40:5: Unexpected character '`'.
40:16: Unexpected character '`'.
40:91: Unexpected character '✓'.
41:5: Unexpected character '`'.
41:16: Unexpected character '`'.
41:88: Unexpected character '✓'.
42:5: Unexpected character '`'.
42:16: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN20 123
OUT10 499 |
Code Generation - LiveCodeBench v6 AtCoder (set 6)
6 passed9 failed |
40.0% |
IN165 848
OUT19 119 |
| ✓ |
LcbAbc400PartyLiveCodeBench / atcoder / easy: ABC400 Party (abc400_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 180
OUT61 |
| × |
LcbAbcbaLiveCodeBench / atcoder / hard: ABCBA (abc398_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Abcba case0: expected [ABCBA] got [ABCCB] |
Failed |
IN9 145
OUT208 |
| × |
LcbBonfireLiveCodeBench / atcoder / hard: Bonfire (abc398_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Bonfire case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:234: Char literals must contain exactly one character and close with '''.
9:78: Unexpected character '—'.
11:111: Char literals must contain exactly one character and close with '''.
13:61: Char literals must contain exactly one character and close with '''.
17:96: Unexpected character '≤'.
19:37: Unexpected character '—'.
25:26: Unexpected character '±'.
25:130: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 329
OUT647 |
| ✓ |
LcbDoorsInCenterLiveCodeBench / atcoder / easy: Doors in the Center (abc398_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 160
OUT137 |
| ✓ |
LcbFullHouse3LiveCodeBench / atcoder / medium: Full House 3 (abc398_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 202
OUT226 |
| × |
LcbGoodIntegersLiveCodeBench / atcoder / medium: 2^a b^2 good integers (abc400_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GoodIntegers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:18: Unexpected character '#'.
5:26: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 200
OUT2 566 |
| ✓ |
LcbHammingDistanceLiveCodeBench / atcoder / easy: Hamming Distance (abc399_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 184
OUT92 |
| ✓ |
LcbMakeItForestLiveCodeBench / atcoder / medium: Make it Forest (abc399_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 182
OUT400 |
| × |
LcbNotOnlyTreeGameLiveCodeBench / atcoder / hard: Not Only Tree Game (abc398_g)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NotOnlyTreeGame case0: expected [Aoki] got [Takahashi] |
Failed |
IN36 893
OUT4 161 |
| × |
LcbRangePowerSumLiveCodeBench / atcoder / hard: Range Power Sum (abc399_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: RangePowerSum case0 runtime error: List index 4 is out of range. The list has 4 element(s).
Values:
run.Success = false |
Failed |
IN9 217
OUT806 |
| ✓ |
LcbRankingWithTiesLiveCodeBench / atcoder / easy: Ranking with Ties (abc399_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 211
OUT1 444 |
| × |
LcbReplaceLiveCodeBench / atcoder / hard: Replace (abc399_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Replace case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:46: Char literals must contain exactly one character and close with '''.
9:210: Char literals must contain exactly one character and close with '''.
11:165: Char literals must contain exactly one character and close with '''.
11:667: Char literals must contain exactly one character and close with '''.
15:144: Unexpected character '#'.
17:50: Char literals must contain exactly one character and close with '''.
23:272: Char literals must contain exactly one character and close with '''.
25:120: Char literals must contain exactly one character and close with '''.
27:4: Char literals must contain exactly one character and close with '''.
39:149: Unexpected character '#'.
62:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 213
OUT2 411 |
| × |
LcbSwitchSeatsLiveCodeBench / atcoder / medium: Switch Seats (abc399_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SwitchSeats case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:232: Char literals must contain exactly one character and close with '''.
5:2: Char literals must contain exactly one character and close with '''.
5:256: Char literals must contain exactly one character and close with '''.
5:324: Char literals must contain exactly one character and close with '''.
5:457: Char literals must contain exactly one character and close with '''.
7:554: Unexpected character '↔'.
7:563: Char literals must contain exactly one character and close with '''.
9:133: Unexpected character '—'.
9:145: Char literals must contain exactly one character and close with '''.
11:256: Unexpected character '—'.
13:544: Char literals must contain exactly one character and close with '''.
15:103: Unexpected character '→'.
17:51: Unexpected character '✓'.
Values:
run.Compiled = false |
Failed |
IN9 258
OUT1 701 |
| × |
LcbUniquenessLiveCodeBench / atcoder / medium: Uniqueness (abc398_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Uniqueness case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 186
OUT908 |
| × |
LcbWallBreakerLiveCodeBench / atcoder / medium: Takahashi the Wall Breaker (abc400_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WallBreaker case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
31:9: Variable 'drs' has type 'int' but its initializer has type 'List<int>'.
32:9: Variable 'dcs' has type 'int' but its initializer has type 'List<int>'.
57:30: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
58:31: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
74:34: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
75:35: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
Values:
run.Compiled = false |
Failed |
IN9 288
OUT3 351 |
Code Generation - LiveCodeBench v6 AtCoder (set 7)
1 passed12 failed2 skipped |
7.7% |
IN158 210
OUT54 184 |
| – |
LcbAPowNLiveCodeBench / atcoder / hard: A^n (arc191_c) NOTE: special judge - any valid (A,M) pair is accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbArcArcLiveCodeBench / atcoder / medium: ARC Arc (arc192_a)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ArcArc case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN22 494
OUT5 461 |
| × |
LcbBasicGridUpdatesLiveCodeBench / atcoder / hard: Basic Grid Problem with Updates (arc190_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BasicGridUpdates case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
17:80: Char literals must contain exactly one character and close with '''.
19:184: Char literals must contain exactly one character and close with '''.
23:64: Char literals must contain exactly one character and close with '''.
25:732: Unexpected character '→'.
33:236: Char literals must contain exactly one character and close with '''.
35:158: Char literals must contain exactly one character and close with '''.
37:964: Unexpected character '∘'.
37:1090: Unexpected character '×'.
39:112: Unexpected character '→'.
39:117: Unexpected character '≈'.
39:123: Unexpected character '×'.
39:128: Unexpected character '≈'.
39:131: Unexpected character '→'.
39:173: Unexpected character '→'.
39:280: Unexpected character '×'.
39:394: Unexpected character '→'.
39:419: Unexpected character '≤'.
39:425: Unexpected character '≤'.
39:429: Unexpected character '→'.
39:451: Unexpected character '≈'.
39:464: Unexpected character '≈'.
41:53: Char literals must contain exactly one character and close with '''.
41:195: Unexpected character '×'.
41:582: Unexpected character '≈'.
41:590: Unexpected character '≈'.
41:595: Unexpected character '→'.
41:609: Unexpected character '×'.
49:2: Char literals must contain exactly one character and close with '''.
53:2: Char literals must contain exactly one character and close with '''.
61:346: Char literals must contain exactly one character and close with '''.
63:328: Unexpected character '→'.
63:604: Unexpected character '—'.
65:22: Unexpected character '≈'.
65:48: Unexpected character '≈'.
65:51: Unexpected character '→'.
67:14: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 420
OUT3 108 |
| × |
LcbBrokenWheelLiveCodeBench / atcoder / hard: Broken Wheel (arc193_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BrokenWheel case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 337
OUT8 052 |
| × |
LcbComplementIntervalGraphLiveCodeBench / atcoder / hard: Complement Interval Graph (arc193_a)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ComplementIntervalGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:9: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN33 352
OUT13 787 |
| × |
LcbFennecVsSnuke2LiveCodeBench / atcoder / hard: Fennec VS. Snuke 2 (arc192_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FennecVsSnuke2 case1: expected [Snuke] got [Fennec] |
Failed |
IN9 277
OUT117 |
| × |
LcbFractionLineLiveCodeBench / atcoder / hard: Fraction Line (arc192_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: FractionLine case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 320
OUT3 552 |
| – |
LcbInsideOrOutsideLiveCodeBench / atcoder / hard: Inside or Outside (arc190_a) NOTE: special judge - multiple minimum-cost operation sequences are accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbMatrixPowSumLiveCodeBench / atcoder / hard: Matrix Pow Sum (arc190_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MatrixPowSum case0: expected [0 2
1 2] got [0 1
0 2] |
Failed |
IN9 305
OUT528 |
| × |
LcbMovingPiecesOnGraphLiveCodeBench / atcoder / hard: Moving Pieces on Graph (arc191_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MovingPiecesOnGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:39: Unexpected character '≠'.
7:51: Char literals must contain exactly one character and close with '''.
9:39: Char literals must contain exactly one character and close with '''.
13:14: Char literals must contain exactly one character and close with '''.
20:34: Unexpected character '≠'.
20:87: Unexpected character '²'.
22:54: Char literals must contain exactly one character and close with '''.
24:40: Unexpected character '≥'.
24:126: Char literals must contain exactly one character and close with '''.
24:217: Unexpected character '—'.
24:322: Char literals must contain exactly one character and close with '''.
26:136: Char literals must contain exactly one character and close with '''.
28:143: Unexpected character '⇒'.
30:13: Char literals must contain exactly one character and close with '''.
36:42: Unexpected character '→'.
36:57: Unexpected character '→'.
54:13: Char literals must contain exactly one character and close with '''.
60:15: Char literals must contain exactly one character and close with '''.
68:85: Char literals must contain exactly one character and close with '''.
73:118: Char literals must contain exactly one character and close with '''.
77:21: Char literals must contain exactly one character and close with '''.
79:66: Unexpected character '⟺'.
81:2: Char literals must contain exactly one character and close with '''.
83:12: Char literals must contain exactly one character and close with '''.
85:99: Unexpected character '≤'.
87:12: Char literals must contain exactly one character and close with '''.
87:129: Char literals must contain exactly one character and close with '''.
87:165: Char literals must contain exactly one character and close with '''.
91:49: Unexpected character '—'.
91:65: Char literals must contain exactly one character and close with '''.
93:2: Char literals must contain exactly one character and close with '''.
96:39: Unexpected character '–'.
100:20: Char literals must contain exactly one character and close with '''.
102:2: Char literals must contain exactly one character and close with '''.
107:101: Char literals must contain exactly one character and close with '''.
113:72: Char literals must contain exactly one character and close with '''.
115:25: Char literals must contain exactly one character and close with '''.
117:2: Char literals must contain exactly one character and close with '''.
121:51: Unexpected character '≤'.
121:64: Unexpected character '≈'.
121:76: Unexpected character '→'.
125:53: Unexpected character '→'.
125:95: Unexpected character '→'.
143:2: Char literals must contain exactly one character and close with '''.
145:85: Unexpected character '→'.
145:276: Unexpected character '→'.
147:2: Char literals must contain exactly one character and close with '''.
162:60: Char literals must contain exactly one character and close with '''.
163:26: Unexpected character '→'.
165:54: Char literals must contain exactly one character and close with '''.
193:58: Unexpected character '—'.
199:89: Unexpected character '→'.
201:31: Char literals must contain exactly one character and close with '''.
209:2: Char literals must contain exactly one character and close with '''.
259:5: Char literals must contain exactly one character and close with '''.
261:2: Char literals must contain exactly one character and close with '''.
261:99: Char literals must contain exactly one character and close with '''.
283:2: Char literals must contain exactly one character and close with '''.
293:2: Char literals must contain exactly one character and close with '''.
297:40: Unexpected character '—'.
301:2: Char literals must contain exactly one character and close with '''.
309:2: Char literals must contain exactly one character and close with '''.
315:2: Char literals must contain exactly one character and close with '''.
323:2: Char literals must contain exactly one character and close with '''.
341:2: Char literals ... (message truncated) |
Failed |
IN9 314
OUT5 820 |
| × |
LcbPatisserieAbc3LiveCodeBench / atcoder / hard: Patisserie ABC 3 (abc400_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PatisserieAbc3 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:77: Unexpected character '≤'.
7:39: Unexpected character '≥'.
11:32: Unexpected character '≤'.
13:10: Unexpected character '—'.
13:43: Char literals must contain exactly one character and close with '''.
25:83: Unexpected character '≤'.
25:147: Unexpected character '≤'.
25:185: Unexpected character '→'.
25:193: Unexpected character '≤'.
25:195: Unexpected character 'Σ'.
25:282: Unexpected character '≤'.
25:284: Unexpected character 'Σ'.
25:312: Unexpected character 'Σ'.
27:119: Char literals must contain exactly one character and close with '''.
29:8: Unexpected character '—'.
29:116: Char literals must contain exactly one character and close with '''.
31:141: Unexpected character '—'.
31:249: Char literals must contain exactly one character and close with '''.
34:54: Char literals must contain exactly one character and close with '''.
34:96: Unexpected character '—'.
40:154: Unexpected character '·'.
40:283: Unexpected character '·'.
42:9: Unexpected character 'Σ'.
42:29: Unexpected character '≤'.
42:77: Unexpected character 'Σ'.
42:92: Unexpected character '·'.
42:150: Unexpected character 'Σ'.
42:165: Unexpected character '·'.
42:198: Char literals must contain exactly one character and close with '''.
44:111: Unexpected character '→'.
48:80: Unexpected character '→'.
48:91: Unsupported numeric literal suffix 't'. SC# supports 'm' for decimal literals and 'L' for long integer literals.
48:95: Unexpected character '≠'.
48:193: Unsupported numeric literal suffix 't'. SC# supports 'm' for decimal literals and 'L' for long integer literals.
50:88: Unexpected character '→'.
52:25: Char literals must contain exactly one character and close with '''.
54:64: Unexpected character '≤'.
56:33: Unexpected character '∫'.
56:47: Unexpected character '≤'.
56:59: Unexpected character '∫'.
56:62: Unexpected character '∞'.
56:68: Unexpected character '≤'.
56:77: Unexpected character '≤'.
56:86: Unexpected character '≤'.
56:107: Unexpected character '∫'.
56:115: Unexpected character '≥'.
56:124: Unexpected character '∫'.
56:129: Unexpected character '≥'.
56:136: Unexpected character '≥'.
56:143: Unexpected character '≥'.
56:162: Unexpected character '∫'.
56:171: Unexpected character '≥'.
56:182: Unexpected character '≥'.
56:193: Unexpected character '≥'.
58:5: Unexpected character '‑'.
60:27: Char literals must contain exactly one character and close with '''.
64:73: Unexpected character 'Σ'.
64:78: Unexpected character '…'.
64:664: Unexpected character '→'.
66:22: Char literals must contain exactly one character and close with '''.
124:25: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 334
OUT3 039 |
| × |
LcbReplaceDigitsLiveCodeBench / atcoder / medium: Replace Digits (arc191_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ReplaceDigits case0: expected [593] got [131] |
Failed |
IN9 241
OUT240 |
| × |
LcbRingoFavoriteNumbers3LiveCodeBench / atcoder / hard: Ringo's Favorite Numbers 3 (abc400_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: RingoFavoriteNumbers3 case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 283
OUT1 141 |
| × |
LcbSnukeKyotoTripLiveCodeBench / atcoder / hard: Snuke's Kyoto Trip (arc192_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SnukeKyotoTrip case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:40: Unexpected character '≤'.
3:44: Unexpected character '≤'.
3:51: Unexpected character '≤'.
3:55: Unexpected character '≤'.
3:181: Unexpected character '×'.
5:22: Unexpected character '≤'.
5:24: Unexpected character '≤'.
5:32: Unexpected character '≤'.
5:34: Unexpected character '≤'.
7:27: Unexpected character '≤'.
7:29: Unexpected character '≤'.
7:37: Unexpected character '≤'.
7:39: Unexpected character '≤'.
7:47: Char literals must contain exactly one character and close with '''.
9:152: Unexpected character '≥'.
11:232: Unexpected character '—'.
13:48: Unexpected character '#'.
13:193: Unexpected character 'Σ'.
13:527: Unexpected character 'Σ'.
13:543: Unexpected character '∈'.
13:615: Unexpected character '→'.
13:617: Unexpected character '→'.
15:13: Unexpected character 'Σ'.
15:69: Unexpected character 'Σ'.
15:89: Unexpected character 'Σ'.
15:151: Unexpected character '≥'.
15:174: Char literals must contain exactly one character and close with '''.
19:64: Unexpected character '×'.
23:131: Char literals must contain exactly one character and close with '''.
29:93: Unexpected character '×'.
29:116: Unexpected character 'Σ'.
40:53: Unexpected character '×'.
46:4: Char literals must contain exactly one character and close with '''.
48:418: Unexpected character '≥'.
48:423: Unexpected character '≥'.
48:716: Unexpected character '—'.
48:876: Unexpected character 'Σ'.
48:902: Unexpected character '#'.
48:967: Unexpected character '#'.
48:975: Unexpected character '→'.
48:1014: Unexpected character 'Σ'.
52:476: Unexpected character '#'.
52:484: Unexpected character '→'.
52:502: Unexpected character 'Σ'.
52:515: Unexpected character '#'.
52:523: Unexpected character '→'.
52:548: Unexpected character '#'.
52:556: Unexpected character '→'.
54:134: Unexpected character 'Σ'.
62:4: Unexpected character '’'.
62:154: Unexpected character 'Σ'.
62:166: Unexpected character 'Σ'.
62:178: Unexpected character 'Σ'.
62:205: Unexpected character 'Σ'.
64:132: Unexpected character 'Σ'.
64:138: Unexpected character '#'.
64:146: Unexpected character '→'.
64:185: Unexpected character '∈'.
64:313: Unexpected character '#'.
64:321: Unexpected character '→'.
66:62: Unexpected character '∉'.
66:84: Unexpected character '∈'.
66:99: Unexpected character '∉'.
66:167: Unexpected character '∈'.
66:340: Unexpected character '∈'.
66:440: Unexpected character '#'.
66:517: Unexpected character '∈'.
66:538: Unexpected character '∉'.
66:547: Unexpected character 'Σ'.
66:559: Unexpected character '∈'.
66:616: Unexpected character '→'.
68:15: Unexpected character 'Σ'.
68:19: Unexpected character '∉'.
68:38: Unexpected character 'Σ'.
68:42: Unexpected character '∉'.
68:54: Unexpected character 'Σ'.
68:58: Unexpected character '∉'.
68:72: Unexpected character 'Σ'.
68:76: Unexpected character '∉'.
68:99: Unexpected character 'Σ'.
68:103: Unexpected character '∈'.
72:19: Unexpected character '∉'.
72:91: Unexpected character '≤'.
72:103: Unexpected character '≤'.
72:200: Unexpected character '≤'.
72:212: Unexpected character '≤'.
74:32: Unexpected character '≤'.
74:36: Unexpected character '≤'.
74:188: Unexpected character '≤'.
74:192: Unexpected character '≤'.
74:195: Unexpected character '\'.
76:184: Unexpected character '∈'.
76:263: Unexpected character '—'.
78:24: Char literals must contain exactly one character and close with '''.
82:41: Unexpected character 'Σ'.
82:55: Unexpected character 'Σ'.
82:119: Unexpected character 'Σ'.
82:130: Unexpected character 'Σ'.
82:175: Char literals must contain exactly one character and close with '''.
82:281: Char literals must contain exactly one character and close with '''.
84:7: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 300
OUT9 191 |
| ✓ |
LcbSumOfGeometricSeriesLiveCodeBench / atcoder / easy: Sum of Geometric Series (abc400_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 233
OUT148 |
Code Generation - LiveCodeBench v6 AtCoder (set 8)
1 passed13 failed1 skipped |
7.1% |
IN155 387
OUT38 291 |
| × |
LcbAdjacentDeleteLiveCodeBench / atcoder / hard: Adjacent Delete (arc196_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AdjacentDelete case0: expected [5] got [3] |
Failed |
IN9 205
OUT331 |
| × |
LcbCostToFlipLiveCodeBench / atcoder / hard: Cost to Flip (arc194_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CostToFlip case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:16: Unexpected character '→'.
7:16: Unexpected character '→'.
11:37: Unexpected character '≠'.
11:47: Char literals must contain exactly one character and close with '''.
11:160: Unexpected character '→'.
17:63: Unexpected character '→'.
17:84: Unexpected character '→'.
38:55: Unexpected character '→'.
38:82: Unexpected character '→'.
39:57: Unexpected character '→'.
39:84: Unexpected character '→'.
41:90: Unexpected character '→'.
41:260: Unexpected character '→'.
41:670: Unexpected character '→'.
47:14: Unexpected character '—'.
47:167: Char literals must contain exactly one character and close with '''.
63:2: Char literals must contain exactly one character and close with '''.
73:46: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 278
OUT1 859 |
| – |
LcbHamiltonianPiecesLiveCodeBench / atcoder / hard: Hamiltonian Pieces (arc195_c) NOTE: special judge - any valid placement is accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbMagnetsLiveCodeBench / atcoder / hard: Magnets (arc193_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Magnets case0: expected [3
-1
5] got [-1
-1
3] |
Failed |
IN9 355
OUT721 |
| × |
LcbMinimumCostSortLiveCodeBench / atcoder / hard: Minimum Cost Sort (arc194_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimumCostSort case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:53: 'Cast' expects a numeric or char argument, or a string when casting to Date, DateTime or TimeSpan, but it has type 'string'.
Values:
run.Compiled = false |
Failed |
IN9 218
OUT7 251 |
| × |
LcbOperationsOnStackLiveCodeBench / atcoder / medium: Operations on a Stack (arc194_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OperationsOnStack case0: expected [8] got [5] |
Failed |
IN9 237
OUT127 |
| × |
LcbRandomTreeDistanceLiveCodeBench / atcoder / hard: Random Tree Distance (arc195_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RandomTreeDistance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:77: Unexpected character '≤'.
3:83: Unexpected character '≤'.
7:13: Unexpected character '∈'.
7:96: Unexpected character '∏'.
9:104: Unexpected character 'Σ'.
9:204: Unexpected character '→'.
9:237: Unexpected character 'Σ'.
9:285: Unexpected character 'Σ'.
9:303: Unexpected character '·'.
9:315: Unexpected character '≡'.
9:317: Unexpected character 'Σ'.
9:327: Unexpected character '#'.
9:358: Unexpected character '·'.
11:18: Unexpected character 'Σ'.
11:22: Unexpected character '∈'.
11:48: Char literals must contain exactly one character and close with '''.
15:12: Unexpected character 'Σ'.
15:28: Unexpected character '·'.
15:53: Unexpected character '·'.
27:12: Unexpected character '⊆'.
27:21: Unexpected character '≥'.
27:85: Unexpected character '≥'.
27:233: Unexpected character '≥'.
29:102: Unexpected character '≥'.
29:124: Unexpected character '∉'.
29:228: Unexpected character '≥'.
29:269: Unexpected character '≥'.
31:132: Char literals must contain exactly one character and close with '''.
33:17: Unexpected character '⊆'.
33:19: Unexpected character '≥'.
34:22: Unexpected character '→'.
35:9: Unexpected character '≤'.
35:19: Unexpected character '∉'.
35:52: Unexpected character '∈'.
36:7: Unexpected character '≥'.
38:31: Unexpected character '≥'.
40:141: Char literals must contain exactly one character and close with '''.
42:7: Char literals must contain exactly one character and close with '''.
44:49: Char literals must contain exactly one character and close with '''.
46:51: Char literals must contain exactly one character and close with '''.
48:125: Char literals must contain exactly one character and close with '''.
50:20: Unexpected character '∈'.
52:28: Unexpected character '→'.
53:24: Unexpected character '→'.
54:24: Unexpected character '→'.
58:45: Unexpected character '→'.
58:60: Unexpected character '✓'.
59:62: Unexpected character '→'.
59:96: Unexpected character '→'.
59:101: Unexpected character '✓'.
60:19: Unexpected character '→'.
60:40: Unexpected character '→'.
60:44: Unexpected character '✓'.
62:43: Unexpected character 'Σ'.
62:179: Unexpected character '→'.
64:16: Unexpected character 'Σ'.
66:4: Char literals must contain exactly one character and close with '''.
68:29: Unexpected character '∈'.
68:37: Unexpected character '∈'.
69:38: Unexpected character '⇒'.
70:34: Unexpected character '⇒'.
73:96: Unexpected character 'Σ'.
73:177: Unexpected character '→'.
73:215: Unexpected character 'α'.
73:220: Unexpected character 'β'.
73:225: Unexpected character 'γ'.
75:250: Char literals must contain exactly one character and close with '''.
78:68: Unexpected character '↔'.
82:43: Unexpected character 'Σ'.
82:76: Unexpected character 'Σ'.
85:51: Unexpected character '∈'.
85:62: Unexpected character '≥'.
85:170: Unexpected character '—'.
87:120: Unexpected character '∈'.
87:248: Unexpected character '\'.
89:2: Char literals must contain exactly one character and close with '''.
91:99: Char literals must contain exactly one character and close with '''.
92:53: Unexpected character '∈'.
92:95: Unexpected character '→'.
92:107: Unexpected character '∈'.
92:146: Unexpected character '≠'.
92:174: Char literals must contain exactly one character and close with '''.
94:176: Unexpected character 'Σ'.
100:44: Unexpected character '#'.
101:70: Unexpected character '→'.
101:87: Unexpected character '→'.
101:89: Unexpected character '✓'.
101:113: Unexpected character '→'.
101:122: Unexpected character '→'.
101:124: Unexpected character '✓'.
101:163: Unexpected character '∈'.
101:213: Unexpected character '✓'.
102:50: Unexpected character '→'.
102:61: Unexpected character '✓'.
102:86: Unexpected character '→'.
102:94: Unexpected character '✓'.
104:44: Unexpected character '#'.
104:95: Unexpected character '⊆'.
104:97: Unexpected character '≥'.
104:123: Unexpected character '→'.
106:54: Too many lexical errors; further errors are not reported.
Va ... (message truncated) |
Failed |
IN9 369
OUT3 821 |
| × |
LcbReverseBracketsLiveCodeBench / atcoder / hard: Reverse Brackets (arc194_d)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ReverseBrackets case0: expected [2] got [16] |
Failed |
IN34 243
OUT12 885 |
| × |
LcbRoadwayLiveCodeBench / atcoder / hard: Roadway (arc196_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Roadway case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 426
OUT3 384 |
| × |
LcbStronglyConnectedLiveCodeBench / atcoder / hard: Strongly Connected (arc196_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: StronglyConnected case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 301
OUT1 105 |
| × |
LcbSwap0X1YLiveCodeBench / atcoder / hard: Swap 0^X and 1^Y (arc194_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Swap0X1Y case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:46: Unexpected character '`'.
3:57: Unexpected character '`'.
3:79: Unexpected character '`'.
3:94: Unexpected character '`'.
3:104: Unexpected character '→'.
3:114: Unexpected character '`'.
3:126: Unexpected character '`'.
3:132: Unexpected character '`'.
3:149: Unexpected character '`'.
3:266: Unexpected character '—'.
5:6: Char literals must contain exactly one character and close with '''.
7:117: Unexpected character '—'.
7:290: Char literals must contain exactly one character and close with '''.
9:259: Unexpected character '`'.
9:267: Unexpected character '`'.
9:272: Unexpected character '`'.
9:280: Unexpected character '`'.
9:282: Unexpected character '—'.
9:319: Unexpected character '#'.
9:326: Unexpected character '#'.
15:143: Unexpected character '#'.
15:152: Unexpected character '#'.
15:283: Unexpected character '→'.
15:289: Unexpected character '→'.
15:290: Unexpected character '−'.
15:328: Char literals must contain exactly one character and close with '''.
17:113: Char literals must contain exactly one character and close with '''.
24:141: Char literals must contain exactly one character and close with '''.
26:366: Unexpected character '±'.
26:387: Unexpected character '±'.
28:363: Char literals must contain exactly one character and close with '''.
32:21: Unexpected character '#'.
32:297: Char literals must contain exactly one character and close with '''.
38:4: Char literals must contain exactly one character and close with '''.
42:107: Unexpected character '—'.
Values:
run.Compiled = false |
Failed |
IN9 388
OUT1 574 |
| × |
LcbSwapAndEraseLiveCodeBench / atcoder / hard: Swap and Erase (arc195_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SwapAndErase case0: expected [3
4
8] got [6
3
10] |
Failed |
IN9 316
OUT505 |
| × |
LcbTorusLoopLiveCodeBench / atcoder / hard: Torus Loop (arc196_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TorusLoop case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
7:24: Unexpected character '→'.
8:24: Unexpected character '→'.
10:107: Char literals must contain exactly one character and close with '''.
12:18: Char literals must contain exactly one character and close with '''.
14:126: Char literals must contain exactly one character and close with '''.
16:17: Char literals must contain exactly one character and close with '''.
26:9: Unexpected character '—'.
26:54: Char literals must contain exactly one character and close with '''.
28:61: Char literals must contain exactly one character and close with '''.
32:377: Unexpected character '#'.
32:506: Unexpected character '→'.
32:583: Char literals must contain exactly one character and close with '''.
38:206: Unexpected character '—'.
40:64: Char literals must contain exactly one character and close with '''.
46:5: Char literals must contain exactly one character and close with '''.
52:123: Char literals must contain exactly one character and close with '''.
52:1045: Unexpected character '—'.
52:1124: Char literals must contain exactly one character and close with '''.
52:1221: Char literals must contain exactly one character and close with '''.
52:1816: Unexpected character '—'.
72:34: Unexpected character '⇔'.
72:50: Unexpected character '⇔'.
72:52: Unexpected character '¬'.
74:296: Unexpected character '—'.
74:370: Unexpected character '≤'.
82:142: Unexpected character '⇒'.
82:154: Unexpected character '∀'.
82:159: Unexpected character '⇒'.
82:175: Unexpected character 'α'.
82:188: Unexpected character 'α'.
82:203: Unexpected character 'α'.
82:204: Unexpected character '∈'.
83:6: Unexpected character 'α'.
83:23: Unexpected character '⇒'.
83:37: Unexpected character '⇒'.
83:53: Unexpected character '⇒'.
84:3: Unexpected character 'α'.
84:20: Unexpected character '⇒'.
84:25: Unexpected character '⇒'.
89:101: Char literals must contain exactly one character and close with '''.
91:73: Char literals must contain exactly one character and close with '''.
97:195: Char literals must contain exactly one character and close with '''.
101:14: Char literals must contain exactly one character and close with '''.
103:144: Unexpected character '≠'.
103:153: Unexpected character '≠'.
105:38: Unexpected character '⊕'.
105:45: Unexpected character '⊕'.
105:75: Unexpected character '→'.
106:12: Unexpected character '↔'.
108:10: Unexpected character '≡'.
112:2: Char literals must contain exactly one character and close with '''.
114:14: Char literals must contain exactly one character and close with '''.
116:362: Unexpected character 'α'.
116:388: Unexpected character 'α'.
116:411: Unexpected character '→'.
118:2: Char literals must contain exactly one character and close with '''.
122:20: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 493
OUT3 747 |
| ✓ |
LcbTwiceSubsequenceLiveCodeBench / atcoder / medium: Twice Subsequence (arc195_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 258
OUT608 |
| × |
LcbUniformSumLiveCodeBench / atcoder / hard: Uniform Sum (arc195_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: UniformSum case1: expected [No] got [Yes] |
Failed |
IN9 300
OUT373 |
Code Generation - LiveCodeBench v6 LeetCode (set 1)
6 passed5 failed |
54.5% |
IN101 311
OUT10 100 |
| ✓ |
LcbCountEvenDiffPartitionsLiveCodeBench 3704 / leetcode / easy: count partitions with even sum difference (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT148 |
| ✓ |
LcbCountMentionsPerUserLiveCodeBench 3721 / leetcode / medium: count mentions per user (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 295
OUT2 289 |
| ✓ |
LcbCountNonDecreasingSubarraysLiveCodeBench 3674 / leetcode / hard: count non-decreasing subarrays after k operations (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 214
OUT183 |
| × |
LcbMaxFrequencyAfterOpLiveCodeBench 3751 / leetcode / medium: maximum frequency after subarray operation (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxFrequencyAfterOp case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
26:467: Unexpected character '≠'.
26:508: Unexpected character '≠'.
26:518: Unexpected character '→'.
26:523: Unexpected character '≠'.
26:544: Char literals must contain exactly one character and close with '''.
26:663: Char literals must contain exactly one character and close with '''.
30:42: Unexpected character '→'.
34:47: Unexpected character '→'.
34:174: Char literals must contain exactly one character and close with '''.
38:336: Unexpected character '#'.
38:356: Unexpected character '#'.
38:395: Unexpected character '#'.
40:160: Unexpected character '≠'.
42:278: Unexpected character '✓'.
42:481: Unexpected character '✓'.
Values:
run.Compiled = false |
Failed |
IN9 180
OUT1 449 |
| × |
LcbMinimizeMaxEdgeWeightLiveCodeBench 3720 / leetcode / medium: minimize the maximum edge weight of graph (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minimizeMaxEdgeWeight case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:26: Unexpected character '`'.
6:36: Unexpected character '`'.
8:66: Unexpected character '—'.
10:150: Unexpected character '→'.
10:156: Unexpected character '→'.
14:14: Unexpected character '≤'.
16:9: Char literals must contain exactly one character and close with '''.
16:205: Unexpected character '≤'.
18:185: Unexpected character '≤'.
18:280: Unexpected character '≤'.
20:16: Char literals must contain exactly one character and close with '''.
20:230: Unexpected character '≤'.
20:281: Unexpected character '≠'.
20:379: Unexpected character '≥'.
22:156: Char literals must contain exactly one character and close with '''.
24:280: Unexpected character '≠'.
24:318: Unexpected character '→'.
26:69: Unexpected character '≤'.
26:102: Unexpected character '≠'.
30:41: Unexpected character '≤'.
30:341: Unexpected character '—'.
30:419: Char literals must contain exactly one character and close with '''.
32:98: Unexpected character '≥'.
32:209: Unexpected character '≥'.
32:563: Unexpected character '⟺'.
32:604: Unexpected character '≥'.
34:86: Unexpected character '≥'.
36:8: Unexpected character '—'.
36:55: Unexpected character '≤'.
36:84: Char literals must contain exactly one character and close with '''.
38:75: Unexpected character '≤'.
40:95: Unexpected character '≤'.
40:100: Unexpected character '→'.
40:105: Unexpected character '→'.
40:110: Unexpected character '→'.
40:115: Unexpected character '→'.
40:153: Unexpected character '→'.
40:155: Unexpected character '→'.
40:164: Unexpected character '→'.
40:166: Unexpected character '→'.
40:217: Unexpected character '✓'.
42:28: Char literals must contain exactly one character and close with '''.
46:180: Unexpected character '≤'.
46:207: Unexpected character '→'.
50:61: Unexpected character '≤'.
51:157: Char literals must contain exactly one character and close with '''.
52:75: Unexpected character '→'.
54:30: Unexpected character '→'.
54:62: Unexpected character '→'.
68:31: Unexpected character '≥'.
68:107: Unexpected character '→'.
68:115: Unexpected character '→'.
68:136: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 238
OUT2 843 |
| ✓ |
LcbMinMaxSubarraySumsLiveCodeBench 3725 / leetcode / hard: max and min sums of at most size k subarrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 124
OUT234 |
| × |
LcbMinMaxSubsequenceSumsLiveCodeBench 3714 / leetcode / medium: max and min sums of at most size k subsequences (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minMaxSubsequenceSums case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:65: Unexpected character '`'.
3:70: Unexpected character '`'.
6:21: Unexpected character '→'.
7:21: Unexpected character '→'.
8:21: Unexpected character '→'.
10:150: Char literals must contain exactly one character and close with '''.
14:53: Unexpected character '→'.
15:17: Unexpected character '✓'.
19:39: Unexpected character '→'.
20:11: Unexpected character '✓'.
22:6: Char literals must contain exactly one character and close with '''.
44:128: Unexpected character '—'.
44:203: Char literals must contain exactly one character and close with '''.
46:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 162
OUT1 626 |
| × |
LcbPaintHouseIVLiveCodeBench 3737 / leetcode / medium: paint house IV (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: paintHouseIV case0: expected 9 got 6 |
Failed |
IN9 265
OUT395 |
| × |
LcbRobotMaxCoinsLiveCodeBench 3677 / leetcode / medium: maximum amount of money robot can earn (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: robotMaxCoins case0 runtime error: Cannot cast value to 'int' because it is outside the target type range.
Values:
run.Success = false |
Failed |
IN9 264
OUT517 |
| ✓ |
LcbVariableLengthSubarraySumLiveCodeBench 3731 / leetcode / easy: sum of variable length subarrays (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 143
OUT114 |
| ✓ |
LcbZigzagGridSkipLiveCodeBench 3708 / leetcode / easy: zigzag grid traversal with skip (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 251
OUT302 |
Code Generation - LiveCodeBench v6 LeetCode (set 2)
4 passed6 failed |
40.0% |
IN91 925
OUT6 818 |
| ✓ |
LcbAssignElementsToGroupsLiveCodeBench 3760 / leetcode / medium: assign elements to groups with constraints (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 256
OUT262 |
| × |
LcbCountSubstringsDivLastDigitLiveCodeBench 3696 / leetcode / hard: count substrings divisible by last digit (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: countSubstringsDivLastDigit case0: expected 11 got 7 |
Failed |
IN9 125
OUT146 |
| × |
LcbEatPizzasMaxWeightLiveCodeBench 3779 / leetcode / medium: eat pizzas (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: eatPizzasMaxWeight case0: expected 14 got 10 |
Failed |
IN9 227
OUT144 |
| ✓ |
LcbMaxEvenOddFreqDiffLiveCodeBench 3753 / leetcode / easy: maximum difference between even and odd frequency I (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT227 |
| × |
LcbMaximizeMinGameScoreLiveCodeBench 3762 / leetcode / hard: maximize the minimum game score (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maximizeMinGameScore case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
53:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 233
OUT4 288 |
| ✓ |
LcbMaxManhattanAfterKLiveCodeBench 3754 / leetcode / medium: maximum manhattan distance after k changes (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 233
OUT498 |
| × |
LcbMinIncrementsTargetMultiplesLiveCodeBench 3697 / leetcode / hard: minimum increments for target multiples in an array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minIncrementsTargetMultiples case0: expected 1 got 0 |
Failed |
IN9 159
OUT231 |
| × |
LcbSelectKDisjointSpecialLiveCodeBench 3771 / leetcode / medium: select k disjoint special substrings (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: selectKDisjointSpecial case1: expected false got true |
Failed |
IN9 178
OUT392 |
| × |
LcbSortMatrixDiagonalsLiveCodeBench 3748 / leetcode / medium: sort matrix by diagonals (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<List<int>>>(run.ResultJson) == FromJson<List<List<int>>>(expected[i])
Context: sortMatrixDiagonals case0: expected [[8, 2, 3], [9, 6, 7], [4, 5, 1]] got [[8,7,3],[5,6,2],[4,9,1]] |
Failed |
IN9 184
OUT399 |
| ✓ |
LcbSpecialSubstringLengthKLiveCodeBench 3709 / leetcode / easy: find special substring of length k (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 167
OUT231 |
Code Generation - LiveCodeBench v6 LeetCode (set 3)
5 passed6 failed |
45.5% |
IN101 405
OUT3 834 |
| ✓ |
LcbChooseKMaxSumLiveCodeBench / leetcode / medium / id 3759: for each i, sum of k largest nums2[j] where nums1[j] < nums1[i]1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 259
OUT209 |
| ✓ |
LcbFruitsIntoBasketsLiveCodeBench / leetcode / medium / id 3791: count fruit types that remain unplaced into baskets1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 233
OUT183 |
| × |
LcbLargestAlmostMissingLiveCodeBench / leetcode / easy / id 3705: largest integer appearing in exactly one size-k subarray1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: largestmissing case1: expected 3 got 2 |
Failed |
IN9 182
OUT303 |
| ✓ |
LcbLexSmallestGeneratedLiveCodeBench / leetcode / hard / id 3770: lexicographically smallest string generated from str1 and str21 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 238
OUT580 |
| × |
LcbLongestPalinSubseqKOpsLiveCodeBench / leetcode / medium / id 3786: longest palindromic subsequence after at most k letter shifts1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: palinsubseq case0: expected 3 got 4 |
Failed |
IN9 204
OUT471 |
| × |
LcbLongestVDiagonalLiveCodeBench / leetcode / hard / id 3733: length of longest V-shaped diagonal segment1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: vdiag case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 281
OUT639 |
| × |
LcbMaxDistanceSquareLiveCodeBench / leetcode / hard / id 3781: maximize the minimum Manhattan distance among k boundary points1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxdist case1: expected 1 got 3 |
Failed |
IN9 276
OUT327 |
| × |
LcbMaxSubarraysConflictLiveCodeBench / leetcode / hard / id 3789: max non-empty subarrays after removing one conflicting pair1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxsub case0: expected 9 got 0 |
Failed |
IN9 220
OUT381 |
| × |
LcbMaxSumKSubarraysLiveCodeBench / leetcode / medium / id 3722: max sum of k non-overlapping subarrays each length >= m1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxsumksub case0 runtime error: List index -1 is out of range. The list has 7 element(s).
Values:
run.Success = false |
Failed |
IN9 157
OUT395 |
| ✓ |
LcbMaxSumWithLimitsLiveCodeBench / leetcode / medium / id 3764: maximum sum of at most k elements with per-row limits1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 192
OUT209 |
| ✓ |
LcbSameDigitsAfterOpsLiveCodeBench / leetcode / easy / id 3768: check if final two digits are equal after repeated pairwise sums1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT137 |
Code Generation - LiveCodeBench v6 LeetCode (set 4)
4 passed6 failed |
40.0% |
IN92 022
OUT4 474 |
| ✓ |
LcbClosestEqualQueriesLiveCodeBench / leetcode / medium / id 3750: circular min distance to an equal-valued element per query1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 205
OUT310 |
| ✓ |
LcbCountBeautifulNumbersLiveCodeBench / leetcode / hard / id 3801: count numbers in [l,r] whose digit-product is divisible by digit-sum1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 157
OUT185 |
| × |
LcbLongestPalinConcatLiveCodeBench / leetcode / medium / id 3793: longest palindrome from a substring of s concatenated with one of t1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: palinconcat case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:6: Char literals must contain exactly one character and close with '''.
7:86: Unexpected character '→'.
17:75: Char literals must contain exactly one character and close with '''.
19:58: Char literals must contain exactly one character and close with '''.
27:83: Char literals must contain exactly one character and close with '''.
31:253: Unexpected character '→'.
31:294: Unexpected character '≤'.
33:37: Char literals must contain exactly one character and close with '''.
35:95: Char literals must contain exactly one character and close with '''.
41:235: Char literals must contain exactly one character and close with '''.
43:88: Unexpected character '≤'.
43:93: Unexpected character '≤'.
47:46: Char literals must contain exactly one character and close with '''.
49:52: Char literals must contain exactly one character and close with '''.
51:91: Unexpected character '⇔'.
53:44: Unexpected character '⇒'.
55:14: Char literals must contain exactly one character and close with '''.
59:2: Char literals must contain exactly one character and close with '''.
61:34: Char literals must contain exactly one character and close with '''.
111:77: Unexpected character '—'.
111:119: Unexpected character '≥'.
111:159: Unexpected character '→'.
113:32: Unexpected character '≥'.
113:43: Unexpected character '≥'.
Values:
run.Compiled = false |
Failed |
IN9 167
OUT2 606 |
| ✓ |
LcbMaxContainersShipLiveCodeBench / leetcode / easy / id 3817: max containers of weight w on an n x n deck within maxWeight1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 173
OUT55 |
| ✓ |
LcbMaxUniqueSubarraySumLiveCodeBench / leetcode / easy / id 3788: max sum of a subarray with all-unique elements after deletions1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 176
OUT136 |
| × |
LcbMinCostReachPositionsLiveCodeBench / leetcode / easy / id 3832: min total cost to reach each position via prefix-min swaps1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: mincosts case0: expected [5, 3, 3, 1, 1, 1] got [0,0,0,0,0,0] |
Failed |
IN9 227
OUT192 |
| × |
LcbMinOpsElementsZeroLiveCodeBench / leetcode / hard / id 3744: sum over queries of min pair-divide operations to zero a range1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minops case0: expected 3 got 6 |
Failed |
IN9 180
OUT133 |
| × |
LcbMinTimeBrewPotionsLiveCodeBench / leetcode / medium / id 3794: min synchronized time for n wizards to brew m potions in order1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: mintime case0: expected 110 got 14 |
Failed |
IN9 249
OUT179 |
| × |
LcbPropertiesGraphLiveCodeBench / leetcode / medium / id 3809: connected components where edges need >= k shared distinct values1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: propgraph case2: expected 2 got 1 |
Failed |
IN9 236
OUT322 |
| × |
LcbZeroArrayTransformIVLiveCodeBench / leetcode / medium / id 3795: min k queries to reduce nums to a zero array (subset decrements)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minzero case0: expected 2 got 1 |
Failed |
IN9 252
OUT356 |
Code Generation - LiveCodeBench v6 LeetCode (set 5)
4 passed7 failed |
36.4% |
IN101 116
OUT14 121 |
| ✓ |
LcbFindValidDigitPairLiveCodeBench / leetcode / easy: find-valid-pair-of-adjacent-digits-in-string (id 3736)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT254 |
| ✓ |
LcbLongestSpecialPathLiveCodeBench / leetcode / hard: longest-special-path (id 3687)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 283
OUT3 503 |
| × |
LcbManhattanArrangementsLiveCodeBench / leetcode / hard: manhattan-distances-of-all-arrangements-of-pieces (id 3739)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: manhattanArrange case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:136: Unexpected character '×'.
3:151: Unexpected character '⁹'.
5:38: Unexpected character 'ᵢ'.
5:39: Unexpected character '−'.
5:41: Unexpected character 'ⱼ'.
5:48: Unexpected character 'ᵢ'.
5:49: Unexpected character '−'.
5:51: Unexpected character 'ⱼ'.
7:182: Unexpected character '≥'.
9:32: Char literals must contain exactly one character and close with '''.
13:36: Unexpected character '₁'.
13:37: Unexpected character '−'.
13:39: Unexpected character '₂'.
13:46: Unexpected character '₁'.
13:47: Unexpected character '−'.
13:49: Unexpected character '₂'.
13:113: Unexpected character '₁'.
13:114: Unexpected character '−'.
13:116: Unexpected character '₂'.
13:147: Unexpected character '₁'.
13:148: Unexpected character '−'.
13:150: Unexpected character '₂'.
15:61: Unexpected character '₁'.
15:68: Unexpected character '₂'.
15:83: Unexpected character '₁'.
15:84: Unexpected character '−'.
15:86: Unexpected character '₂'.
15:146: Unexpected character '₁'.
15:153: Unexpected character '₂'.
15:208: Char literals must contain exactly one character and close with '''.
18:64: Unexpected character '⊂'.
19:57: Unexpected character '×'.
19:60: Unexpected character '#'.
21:65: Unexpected character '−'.
21:106: Unexpected character '−'.
21:134: Unexpected character '−'.
21:139: Unexpected character '−'.
21:161: Unexpected character '≥'.
23:20: Unexpected character '−'.
23:25: Unexpected character '−'.
23:29: Unexpected character '×'.
23:87: Unexpected character '−'.
23:105: Unexpected character '−'.
25:41: Unexpected character '−'.
27:15: Unexpected character '−'.
28:72: Unexpected character '−'.
28:83: Unexpected character '²'.
30:31: Unexpected character '∈'.
30:40: Unexpected character '∈'.
32:29: Unexpected character '₁'.
32:32: Unexpected character '₁'.
32:37: Unexpected character '₂'.
32:40: Unexpected character '₂'.
32:49: Unexpected character '₁'.
32:50: Unexpected character '−'.
32:52: Unexpected character '₂'.
33:3: Unexpected character 'Σ'.
33:7: Unexpected character '₁'.
33:10: Unexpected character '₂'.
33:15: Unexpected character '₁'.
33:16: Unexpected character '−'.
33:18: Unexpected character '₂'.
33:21: Unexpected character '×'.
33:25: Unexpected character '×'.
34:4: Unexpected character '²'.
34:6: Unexpected character '×'.
34:8: Unexpected character 'Σ'.
34:12: Unexpected character '₁'.
34:15: Unexpected character '₂'.
34:19: Unexpected character '₂'.
34:20: Unexpected character '−'.
34:22: Unexpected character '₁'.
36:1: Unexpected character 'Σ'.
36:5: Unexpected character '₁'.
36:8: Unexpected character '₂'.
36:12: Unexpected character '₂'.
36:13: Unexpected character '−'.
36:15: Unexpected character '₁'.
36:24: Unexpected character '∈'.
36:30: Unexpected character '−'.
37:3: Unexpected character 'Σ'.
37:13: Unexpected character '−'.
37:19: Unexpected character '×'.
37:23: Unexpected character '−'.
39:25: Unexpected character '²'.
39:27: Unexpected character '×'.
39:29: Unexpected character 'Σ'.
39:33: Unexpected character '₁'.
39:36: Unexpected character '₂'.
39:40: Unexpected character '₂'.
39:41: Unexpected character '−'.
39:43: Unexpected character '₁'.
39:49: Unexpected character '²'.
39:51: Unexpected character '×'.
39:53: Unexpected character 'Σ'.
39:63: Unexpected character '−'.
39:70: Unexpected character '−'.
41:33: Unexpected character '−'.
41:38: Unexpected character '−'.
43:38: Unexpected character '→'.
43:80: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 179
OUT1 973 |
| ✓ |
LcbMaxAdjacentCircularLiveCodeBench / leetcode / easy: maximum-difference-between-adjacent-elements-in-a-circular-array (id 3747)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 106
OUT114 |
| × |
LcbMaxFreeTimeLiveCodeBench / leetcode / medium: reschedule-meetings-for-maximum-free-time-i (id 3743)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxFreeTime case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:61: Char literals must contain exactly one character and close with '''.
9:50: Char literals must contain exactly one character and close with '''.
11:20: Char literals must contain exactly one character and close with '''.
15:102: Unexpected character '≥'.
17:136: Unexpected character '≤'.
21:100: Char literals must contain exactly one character and close with '''.
23:129: Char literals must contain exactly one character and close with '''.
25:121: Unexpected character '≤'.
28:183: Unexpected character '✓'.
30:102: Unexpected character '→'.
30:112: Unexpected character '✓'.
30:316: Unexpected character '—'.
32:74: Unexpected character '≈'.
34:70: Unexpected character '≤'.
34:394: Char literals must contain exactly one character and close with '''.
36:2: Char literals must contain exactly one character and close with '''.
38:133: Unexpected character '≤'.
42:24: Unexpected character '→'.
42:95: Char literals must contain exactly one character and close with '''.
92:146: Unexpected character '→'.
92:233: Unexpected character '→'.
92:297: Unexpected character '→'.
94:85: Unexpected character '—'.
94:94: Char literals must contain exactly one character and close with '''.
100:14: Char literals must contain exactly one character and close with '''.
102:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 285
OUT1 822 |
| × |
LcbMaxProductAlternatingSumLiveCodeBench / leetcode / hard: maximum-product-of-subsequences-with-an-alternating-sum-equal-to-k (id 3777)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxProductAlt case0: expected 6 got 2 |
Failed |
IN9 211
OUT429 |
| × |
LcbMinCostGoodCaptionLiveCodeBench / leetcode / hard: minimum-cost-good-caption (id 3701)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minCostGoodCaption case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
62:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 214
OUT4 510 |
| × |
LcbMinOpsKSubarraysEqualLiveCodeBench / leetcode / hard: minimum-operations-to-make-elements-within-k-subarrays-equal (id 3717)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minOpsKSub case1: expected 3 got 0 |
Failed |
IN9 187
OUT641 |
| × |
LcbMinPairRemovalSortLiveCodeBench / leetcode / easy: minimum-pair-removal-to-sort-array-i (id 3773)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minPairRemoval case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:34: List constructor expects a List or HashSet to copy, but got 'int'.
Values:
run.Compiled = false |
Failed |
IN9 134
OUT238 |
| × |
LcbSeparateSquaresLiveCodeBench / leetcode / medium: separate-squares-i (id 3763)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: separateSquares case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
18:19: Unknown variable 'double'.
Values:
run.Compiled = false |
Failed |
IN9 198
OUT464 |
| ✓ |
LcbSumOfGoodNumbersLiveCodeBench / leetcode / easy: sum-of-good-numbers (id 3723)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 156
OUT173 |
Code Generation - LiveCodeBench v6 LeetCode (set 6)
4 passed6 failed |
40.0% |
IN91 912
OUT18 664 |
| ✓ |
LcbCountCopyArraysLiveCodeBench / leetcode / medium: find-the-number-of-copy-arrays (id 3785)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 257
OUT217 |
| × |
LcbLongestCommonPrefixRemovalLiveCodeBench / leetcode / hard: longest-common-prefix-of-k-strings-after-removal (id 3784)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: lcpRemoval case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
5:70: Unexpected character '`'.
5:72: Unexpected character '`'.
7:103: Unexpected character '—'.
7:177: Char literals must contain exactly one character and close with '''.
11:253: Char literals must contain exactly one character and close with '''.
15:236: Char literals must contain exactly one character and close with '''.
25:4: Char literals must contain exactly one character and close with '''.
31:4: Char literals must contain exactly one character and close with '''.
35:154: Unexpected character '✓'.
36:126: Unexpected character '✓'.
39:80: Unexpected character '✓'.
63:5: Char literals must contain exactly one character and close with '''.
73:2: Char literals must contain exactly one character and close with '''.
75:69: Unexpected character '—'.
99:4: Char literals must contain exactly one character and close with '''.
113:97: Unexpected character '—'.
117:332: Unexpected character '—'.
123:2: Char literals must contain exactly one character and close with '''.
137:2: Char literals must contain exactly one character and close with '''.
151:160: Unexpected character '✓'.
165:2: Char literals must contain exactly one character and close with '''.
167:44: Char literals must contain exactly one character and close with '''.
179:2: Char literals must contain exactly one character and close with '''.
193:2: Char literals must contain exactly one character and close with '''.
249:20: Unexpected character '`'.
249:28: Unexpected character '`'.
257:13: Char literals must contain exactly one character and close with '''.
287:2: Char literals must contain exactly one character and close with '''.
301:2: Char literals must contain exactly one character and close with '''.
347:2: Char literals must contain exactly one character and close with '''.
375:2: Char literals must contain exactly one character and close with '''.
3469:2: Char literals must contain exactly one character and close with '''.
3475:59: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 199
OUT9 057 |
| × |
LcbMaxActiveSectionsTradeLiveCodeBench / leetcode / medium: maximize-active-section-with-trade-i (id 3805)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxActiveTrade case3: expected 4 got 6 |
Failed |
IN9 243
OUT423 |
| × |
LcbMinCostDivideArrayLiveCodeBench / leetcode / hard: minimum-cost-to-divide-array-into-subarrays (id 3765)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minCostDivide case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:22: Unexpected character '`'.
3:27: Unexpected character '`'.
5:66: Char literals must contain exactly one character and close with '''.
7:44: Unexpected character '→'.
9:66: Unexpected character '→'.
14:28: Unexpected character '→'.
39:81: Char literals must contain exactly one character and close with '''.
55:259: Char literals must contain exactly one character and close with '''.
61:5: Char literals must contain exactly one character and close with '''.
65:4: Char literals must contain exactly one character and close with '''.
71:377: Char literals must contain exactly one character and close with '''.
73:104: Char literals must contain exactly one character and close with '''.
75:4: Char literals must contain exactly one character and close with '''.
79:27: Unexpected character '≥'.
81:51: Unexpected character '≥'.
85:17: Unexpected character '≥'.
87:4: Char literals must contain exactly one character and close with '''.
89:79: Unexpected character '≥'.
91:122: Unexpected character '≥'.
91:128: Unexpected character '→'.
91:132: Unexpected character '≥'.
91:145: Unexpected character '≤'.
91:175: Unexpected character '∈'.
91:189: Char literals must contain exactly one character and close with '''.
101:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 247
OUT3 189 |
| × |
LcbMinCostRemoveElementsLiveCodeBench / leetcode / medium: find-minimum-cost-to-remove-array-elements (id 3776)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minCostRemove case0: expected 12 got 8 |
Failed |
IN9 175
OUT386 |
| ✓ |
LcbPermutationsIVLiveCodeBench / leetcode / hard: permutations-iv (id 3783)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 164
OUT4 193 |
| ✓ |
LcbReverseDegreeLiveCodeBench / leetcode / easy: reverse-degree-of-a-string (id 3811)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 131
OUT76 |
| × |
LcbShortestMatchingSubstringLiveCodeBench / leetcode / hard: shortest-matching-substring (id 3692)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: shortestMatching case0: expected 8 got -1 |
Failed |
IN9 192
OUT624 |
| ✓ |
LcbTransformArrayParityLiveCodeBench / leetcode / easy: transform-array-by-parity (id 3778)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT127 |
| × |
LcbUnique3DigitEvenLiveCodeBench / leetcode / easy: unique-3-digit-even-numbers (id 3799)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: unique3Digit case1: expected 2 got 3 |
Failed |
IN9 155
OUT372 |