Code Generation - LiveCodeBench v1 AtCoder (set 1)
3 passed5 failed |
37.5% |
IN74 056
OUT4 210 |
| × |
LcbAntiDdosCountLiveCodeBench / atcoder abc301_f / hard: Anti (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AntiDdosCount case0: expected [676] got [380204032] |
Failed |
IN9 282
OUT697 |
| × |
LcbAtCoderCardsLiveCodeBench / atcoder abc301_c / medium: AtCoder Cards (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AtCoderCards case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:16: Variable 'words' has type 'List<char>' but its initializer has type 'List<string>'.
Values:
run.Compiled = false |
Failed |
IN9 214
OUT409 |
| ✓ |
LcbAttackCountLiveCodeBench / atcoder abc302_a / easy: Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 168
OUT69 |
| × |
LcbBitmaskLeqNLiveCodeBench / atcoder abc301_d / medium: Bitmask (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BitmaskLeqN case0: expected [1] got [0] |
Failed |
IN9 242
OUT242 |
| ✓ |
LcbFillTheGapsLiveCodeBench / atcoder abc301_b / easy: Fill the Gaps (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 277
OUT310 |
| ✓ |
LcbFindSnukeLiveCodeBench / atcoder abc302_b / medium: Find snuke (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 318
OUT664 |
| × |
LcbOverallWinnerLiveCodeBench / atcoder abc301_a / easy: Overall Winner (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OverallWinner case1: expected [T] got [A] |
Failed |
IN9 228
OUT244 |
| × |
LcbPacCandyPathLiveCodeBench / atcoder abc301_e / hard: Pac (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PacCandyPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:29: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed |
IN9 327
OUT1 575 |
Code Generation - LiveCodeBench v1 AtCoder (set 10)
4 passed2 failed2 skipped |
66.7% |
IN66 748
OUT6 398 |
| × |
LcbBlueSpringPassesLiveCodeBench / atcoder / medium: abc318_c Blue Spring (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: BlueSpringPasses case0 runtime error: List index 5 is out of range. The list has 5 element(s).
Values:
run.Success = false |
Failed |
IN9 217
OUT259 |
| – |
LcbBookPrerequisitesLiveCodeBench / atcoder / hard: abc315_e Prerequisites (Pattern B, stdin) FLAG: non-unique output (any valid reading order accepted); exact-string driver can reject a correct answer -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: non-unique output: multiple valid reading orders accepted; exact-string driver would reject correct answers |
Skipped |
IN0
OUT0 |
| – |
LcbCheckpointShortcutsLiveCodeBench / atcoder / hard: abc315_f Shortcuts (Pattern B, stdin) FLAG: float output (Euclidean distance sum); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: SC# stdin driver: floating-point distance output needs tolerance comparison; exact string compare unsafe |
Skipped |
IN0
OUT0 |
| ✓ |
LcbFullMoonDaysLiveCodeBench / atcoder / easy: abc318_a Full Moon (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 187
OUT101 |
| ✓ |
LcbIceCreamFlavorsLiveCodeBench / atcoder / medium: abc315_c Flavors (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN20 676
OUT2 166 |
| ✓ |
LcbMagicalCookiesLiveCodeBench / atcoder / medium: abc315_d Magical Cookies (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 251
OUT797 |
| ✓ |
LcbMiddleDayLiveCodeBench / atcoder / easy: abc315_b The Middle Day (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 229
OUT215 |
| × |
LcbOverlappingSheetsLiveCodeBench / atcoder / easy: abc318_b Overlapping sheets (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OverlappingSheets case0: expected [20] got [] |
Failed |
IN9 188
OUT2 860 |
Code Generation - LiveCodeBench v1 AtCoder (set 11)
4 passed3 failed1 skipped |
57.1% |
IN91 618
OUT4 844 |
| ✓ |
LcbBusStopsQueriesLiveCodeBench / atcoder / hard: abc319_e Bus Stops (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 306
OUT342 |
| – |
LcbFalseHopeProbabilityLiveCodeBench / atcoder / medium: abc319_c False Hope (Pattern B, stdin) FLAG: float output (probability); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: SC# stdin driver: probability (float) output needs tolerance comparison; exact string compare unsafe |
Skipped |
IN0
OUT0 |
| ✓ |
LcbLeylandNumberLiveCodeBench / atcoder / easy: abc320_a Leyland Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 152
OUT78 |
| ✓ |
LcbLongestPalindromeSubLiveCodeBench / atcoder / easy: abc320_b Longest Palindrome (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 161
OUT244 |
| ✓ |
LcbMeasureDivisorsLiveCodeBench / atcoder / easy: abc319_b Measure (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 214
OUT178 |
| × |
LcbMinimumWindowWidthLiveCodeBench / atcoder / medium: abc319_d Minimum Width (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimumWindowWidth case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:30: Function 'CanFit' must return 'string', but this return statement has type 'bool'.
23:34: Function 'CanFit' must return 'string', but this return statement has type 'bool'.
28:5: Function 'CanFit' must return 'string', but this return statement has type 'bool'.
52:13: The if condition must be 'bool', but it has type 'string'.
Values:
run.Compiled = false |
Failed |
IN9 233
OUT427 |
| × |
LcbSandwichTriplesLiveCodeBench / atcoder / hard: abc318_e Sandwiches (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SandwichTriples case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:16: Variable 'parts' has type 'List<long>' but its initializer has type 'List<string>'.
10:34: 'long.Parse' expects a string argument, but it has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 180
OUT229 |
| × |
LcbWeightedMaxMatchingLiveCodeBench / atcoder / medium: abc318_d General Weighted Max Matching (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: WeightedMaxMatching case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN36 372
OUT3 346 |
Code Generation - LiveCodeBench v1 AtCoder (set 12)
5 passed4 failed |
55.6% |
IN108 932
OUT6 832 |
| ✓ |
Lcb321LikeLiveCodeBench / atcoder / easy: abc321_a 321-like Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 245
OUT88 |
| × |
LcbCompleteBinaryTreeLiveCodeBench / atcoder / hard: abc321_e Complete Binary Tree (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CompleteBinaryTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
33:22: The shift count of '>>' must be 'int', but got 'long'.
37:28: The shift count of '>>' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed |
IN21 860
OUT2 327 |
| ✓ |
LcbCutoffLiveCodeBench / atcoder / easy: abc321_b Cutoff (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 263
OUT260 |
| ✓ |
LcbFirstAbc2LiveCodeBench / atcoder / easy: abc322_a First ABC 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 186
OUT113 |
| ✓ |
LcbKth321LikeLiveCodeBench / atcoder / medium: abc321_c K-th 321-like Number (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN22 400
OUT2 401 |
| × |
LcbRelativePositionLiveCodeBench / atcoder / medium: abc320_d Relative Position (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RelativePosition case1: expected [0 0
2 1
-1 -2] got [0 0
undecidable
undecidable] |
Failed |
IN9 243
OUT560 |
| × |
LcbSetMenuLiveCodeBench / atcoder / medium: abc321_d Set Menu (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SetMenu case0: expected [24] got [34] |
Failed |
IN9 221
OUT439 |
| × |
LcbSlotStrategy2LiveCodeBench / atcoder / medium: abc320_c Slot Strategy 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SlotStrategy2 case0: expected [6] got [0] |
Failed |
IN9 260
OUT306 |
| ✓ |
LcbSomenNagashiLiveCodeBench / atcoder / hard: abc320_e Somen Nagashi (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 254
OUT338 |
Code Generation - LiveCodeBench v1 AtCoder (set 13)
4 passed5 failed |
44.4% |
IN109 088
OUT8 488 |
| ✓ |
LcbFestivalLiveCodeBench / atcoder / medium: abc322_c Festival (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 220
OUT207 |
| × |
LcbMergeSlimesLiveCodeBench / atcoder / hard: abc323_d Merge Slimes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MergeSlimes case0: expected [3] got [2] |
Failed |
IN9 202
OUT348 |
| × |
LcbPlaylistProbLiveCodeBench / atcoder / hard: abc323_e Playlist (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PlaylistProb case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:42: Char literals must contain exactly one character and close with '''.
5:92: Unexpected character '≤'.
9:38: Unexpected character '∃'.
9:62: Unexpected character '≤'.
9:119: Unexpected character '≤'.
11:24: Unexpected character '≤'.
11:122: Unexpected character '≤'.
11:319: Unexpected character '≤'.
13:36: Unexpected character 'Σ'.
13:50: Unexpected character '≤'.
13:61: Unexpected character '·'.
13:77: Unexpected character '·'.
13:106: Unexpected character '≤'.
13:124: Unexpected character '·'.
15:64: Unexpected character '—'.
15:196: Char literals must contain exactly one character and close with '''.
17:9: Unexpected character '—'.
17:87: Unexpected character '≤'.
19:56: Unexpected character '≤'.
19:153: Unexpected character '≤'.
21:50: Unexpected character '·'.
23:29: Unexpected character 'Σ'.
23:43: Unexpected character '≤'.
23:48: Unexpected character '·'.
23:55: Unexpected character '·'.
23:85: Unexpected character '·'.
23:113: Unexpected character '∈'.
29:70: Unexpected character '·'.
29:73: Unexpected character '#'.
29:96: Unexpected character '·'.
29:103: Unexpected character '—'.
39:21: Unexpected character 'Σ'.
43:31: Unexpected character '·'.
43:33: Unexpected character '∫'.
45:49: Unexpected character '—'.
45:98: Char literals must contain exactly one character and close with '''.
47:129: Unexpected character 'Σ'.
47:138: Unexpected character '·'.
47:250: Unexpected character '·'.
47:251: Unexpected character 'Σ'.
49:20: Unexpected character '·'.
49:21: Unexpected character 'Σ'.
49:28: Unexpected character '≤'.
49:55: Unexpected character '·'.
49:56: Unexpected character 'Σ'.
49:60: Unexpected character '∈'.
51:18: Unexpected character '·'.
51:19: Unexpected character 'Σ'.
51:80: Unexpected character '≥'.
53:17: Unexpected character '·'.
53:18: Unexpected character 'Σ'.
55:14: Char literals must contain exactly one character and close with '''.
55:191: Unexpected character '≤'.
55:207: Unexpected character '→'.
55:211: Unexpected character '∈'.
55:251: Unexpected character '→'.
55:255: Unexpected character '∈'.
55:407: Unexpected character '≈'.
55:449: Unexpected character '→'.
57:18: Unexpected character '·'.
57:19: Unexpected character 'Σ'.
57:116: Unexpected character '—'.
57:193: Char literals must contain exactly one character and close with '''.
59:28: Unexpected character 'Σ'.
59:239: Unexpected character 'Σ'.
61:21: Char literals must contain exactly one character and close with '''.
65:62: Unexpected character '≥'.
65:110: Unexpected character '→'.
65:143: Unexpected character '≥'.
Values:
run.Compiled = false |
Failed |
IN9 297
OUT2 661 |
| × |
LcbPolyominoLiveCodeBench / atcoder / medium: abc322_d Polyomino (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Polyomino case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
65:18: Expected a loop variable name, found 'in'.
Values:
run.Compiled = false |
Failed |
IN9 274
OUT1 349 |
| ✓ |
LcbPrefixSuffixLiveCodeBench / atcoder / easy: abc322_b Prefix and Suffix (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 232
OUT152 |
| ✓ |
LcbProductDevelopmentLiveCodeBench / atcoder / hard: abc322_e Product Development (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN35 164
OUT2 937 |
| × |
LcbRoundRobinRankLiveCodeBench / atcoder / easy: abc323_b Round (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RoundRobinRank case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:22: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed |
IN9 243
OUT251 |
| × |
LcbWeakBeatsLiveCodeBench / atcoder / easy: abc323_a Weak Beats (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WeakBeats case1: expected [Yes] got [No] |
Failed |
IN9 188
OUT105 |
| ✓ |
LcbWorldTourFinalsLiveCodeBench / atcoder / medium: abc323_c World Tour Finals (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 268
OUT478 |
Code Generation - LiveCodeBench v1 AtCoder (set 14)
4 passed2 failed1 skipped |
66.7% |
IN65 940
OUT1 995 |
| ✓ |
Lcb2x3yLiveCodeBench / atcoder / easy: abc324_b 2^x 3^y (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 152
OUT109 |
| ✓ |
LcbAllValuesEqualLiveCodeBench / atcoder / easy: abc324_a Same (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT107 |
| – |
LcbBeautifulPathLiveCodeBench / atcoder / hard: abc324_f Beautiful Path (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: float stdout: stdin driver compares stdout by exact trimmed string, cannot tolerance-match a floating-point ratio like 0.7500000000000000 |
Skipped |
IN0
OUT0 |
| × |
LcbErrorCorrectionLiveCodeBench / atcoder / medium: abc324_c Error Correction (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ErrorCorrection case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:34: Function 'CanBeOriginal' must return 'string', but this return statement has type 'bool'.
16:9: Function 'CanBeOriginal' must return 'string', but this return statement has type 'bool'.
33:32: Function 'CanBeOriginal' must return 'string', but this return statement has type 'bool'.
37:9: Function 'CanBeOriginal' must return 'string', but this return statement has type 'bool'.
54:32: Function 'CanBeOriginal' must return 'string', but this return statement has type 'bool'.
58:9: Function 'CanBeOriginal' must return 'string', but this return statement has type 'bool'.
60:5: Function 'CanBeOriginal' must return 'string', but this return statement has type 'bool'.
72:13: The if condition must be 'bool', but it has type 'string'.
Values:
run.Compiled = false |
Failed |
IN9 268
OUT549 |
| ✓ |
LcbJointTwoStringsLiveCodeBench / atcoder / hard: abc324_e Joint Two Strings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 202
OUT411 |
| × |
LcbSquarePermutationLiveCodeBench / atcoder / hard: abc324_d Square Permutation (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SquarePermutation case0: expected [2] got [0] |
Failed |
IN20 025
OUT784 |
| ✓ |
LcbSurnameSanLiveCodeBench / atcoder / easy: abc325_a Takahashi san (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 144
OUT35 |
Code Generation - LiveCodeBench v1 AtCoder (set 15)
5 passed3 failed1 skipped |
62.5% |
IN74 293
OUT3 023 |
| ✓ |
Lcb326LikeNumberLiveCodeBench / atcoder / abc326_b: 3261 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 225
OUT168 |
| – |
LcbAbcPuzzleLiveCodeBench / atcoder / abc326_d: ABC Puzzle NOTE: output is NON-UNIQUE (any valid grid is accepted); exact string compare is unsafe -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: non-unique output: any valid grid is accepted, exact string compare unsafe |
Skipped |
IN0
OUT0 |
| × |
LcbCarTrainTravelLiveCodeBench / atcoder / abc325_e: Our clients, please wait a moment1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CarTrainTravel case0: expected [78] got [80] |
Failed |
IN9 357
OUT559 |
| ✓ |
LcbGiftIntervalLiveCodeBench / atcoder / abc326_c: Peak1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 249
OUT214 |
| × |
LcbPrintingMachineLiveCodeBench / atcoder / abc325_d: Printing Machine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PrintingMachine case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
36:15: Unknown variable 'Ends'.
37:15: Unknown variable 'Ends'.
Values:
run.Compiled = false |
Failed |
IN9 286
OUT337 |
| ✓ |
LcbSensorClustersLiveCodeBench / atcoder / abc325_c: Sensors1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 310
OUT573 |
| × |
LcbSensorCostDilemmaLiveCodeBench / atcoder / abc325_f: Sensor Optimization Dilemma1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SensorCostDilemma case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
33:17: Expected '=' after the variable name. 'var' local variables need an initializer so their type can be inferred, e.g. 'var di = ...;'
Values:
run.Compiled = false |
Failed |
IN9 380
OUT866 |
| ✓ |
LcbStairsOrElevatorLiveCodeBench / atcoder / abc326_a: 2UP3DOWN1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 201
OUT108 |
| ✓ |
LcbWorldMeetingLiveCodeBench / atcoder / abc325_b: World Meeting1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 285
OUT198 |
Code Generation - LiveCodeBench v1 AtCoder (set 16)
3 passed4 failed1 skipped |
42.9% |
IN64 711
OUT1 841 |
| ✓ |
LcbAdjacentABLiveCodeBench / atcoder / abc327_a: ab1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 198
OUT125 |
| × |
LcbGoodTupleLiveCodeBench / atcoder / abc327_d: Good Tuple Problem1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GoodTuple case3: expected [Yes] got [No] |
Failed |
IN9 279
OUT286 |
| – |
LcbMaximizeRatingLiveCodeBench / atcoder / abc327_e: Maximize Rating NOTE: output is a FLOAT with tolerance 1e-6; exact trimmed string compare is unsafe -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: floating-point output with 1e-6 tolerance; exact string compare unsafe |
Skipped |
IN0
OUT0 |
| ✓ |
LcbNotTooHardLiveCodeBench / atcoder / abc328_a: Not Too Hard1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 199
OUT118 |
| × |
LcbNumberPlaceLiveCodeBench / atcoder / abc327_c: Number Place1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: NumberPlace case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
23:17: Operator '!' requires 'bool', but the operand has type 'void'.
33:17: Operator '!' requires 'bool', but the operand has type 'void'.
47:25: Operator '!' requires 'bool', but the operand has type 'void'.
Values:
run.Compiled = false |
Failed |
IN9 297
OUT440 |
| × |
LcbPerfectPowerAALiveCodeBench / atcoder / abc327_b: A^A1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PerfectPowerAA case1: expected [-1] got [10] |
Failed |
IN9 179
OUT219 |
| ✓ |
LcbRepdigitDatesLiveCodeBench / atcoder / abc328_b: 11/111 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 216
OUT266 |
| × |
LcbSalaryExpectationLiveCodeBench / atcoder / abc326_e: Revenge of "The Salary of AtCoder Inc."1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SalaryExpectation case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:16: Variable 'parts' has type 'List<long>' but its initializer has type 'List<string>'.
9:26: 'long.Parse' expects a string argument, but it has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 343
OUT387 |
Code Generation - LiveCodeBench v1 AtCoder (set 17)
5 passed3 failed |
62.5% |
IN73 854
OUT2 073 |
| ✓ |
LcbConsecutivePairsLiveCodeBench / atcoder / abc328_c: Consecutive1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 283
OUT260 |
| × |
LcbCountRepeatsLiveCodeBench / atcoder / abc329_c: Count xxx1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountRepeats case0: expected [4] got [10] |
Failed |
IN9 198
OUT131 |
| ✓ |
LcbElectionReportLiveCodeBench / atcoder / abc329_d: Election Quick Report1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 262
OUT235 |
| × |
LcbModuloMstLiveCodeBench / atcoder / abc328_e: Modulo MST1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ModuloMst case0: expected [33] got [0] |
Failed |
IN9 294
OUT714 |
| ✓ |
LcbSecondLargestLiveCodeBench / atcoder / abc329_b: Next1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 187
OUT198 |
| ✓ |
LcbSpreadCharsLiveCodeBench / atcoder / abc329_a: Spread1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT61 |
| × |
LcbStampMatchLiveCodeBench / atcoder / abc329_e: Stamp1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: StampMatch case0: expected [Yes] got [No] |
Failed |
IN9 254
OUT267 |
| ✓ |
LcbTakeAbcLiveCodeBench / atcoder / abc328_d: Take ABC1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 201
OUT207 |
Code Generation - LiveCodeBench v1 AtCoder (set 18)
6 passed3 failed |
66.7% |
IN105 096
OUT3 430 |
| × |
LcbColoredBallLiveCodeBench / atcoder / hard: Colored Ball [LCB id: abc329_f]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ColoredBall case0: expected [1
2
1
1
3] got [1
2
2
3
3] |
Failed |
IN9 247
OUT904 |
| ✓ |
LcbCountingLsLiveCodeBench / atcoder / medium: Counting Ls [LCB id: abc330_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 296
OUT395 |
| ✓ |
LcbCountingPassesLiveCodeBench / atcoder / easy: Counting Passes [LCB id: abc330_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 166
OUT116 |
| ✓ |
LcbEggPacksLiveCodeBench / atcoder / easy: Buy One Carton of Milk (egg packs) [LCB id: abc331_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 178
OUT254 |
| × |
LcbMexAndUpdateLiveCodeBench / atcoder / hard: Mex and Update [LCB id: abc330_e]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MexAndUpdate case0: expected [4
3
6
5
0] got [4
6
6
6
6] |
Failed |
IN9 252
OUT528 |
| ✓ |
LcbMinimizeAbs1LiveCodeBench / atcoder / easy: Minimize Abs 1 [LCB id: abc330_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 240
OUT172 |
| ✓ |
LcbMinimizeAbs2LiveCodeBench / atcoder / medium: Minimize Abs 2 [LCB id: abc330_c]1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN31 346
OUT533 |
| ✓ |
LcbSumGreaterThanMeLiveCodeBench / atcoder / medium: Sum of Numbers Greater Than Me [LCB id: abc331_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 188
OUT368 |
| × |
LcbTomorrowCalendarLiveCodeBench / atcoder / easy: Tomorrow [LCB id: abc331_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TomorrowCalendar case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:14: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 183
OUT160 |
Code Generation - LiveCodeBench v1 AtCoder (set 19)
6 passed2 failed |
75.0% |
IN74 311
OUT6 977 |
| ✓ |
LcbGlassAndMugLiveCodeBench / atcoder / easy: Glass and Mug [LCB id: abc332_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 269
OUT191 |
| ✓ |
LcbOnlineShoppingLiveCodeBench / atcoder / easy: Online Shopping [LCB id: abc332_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 238
OUT158 |
| ✓ |
LcbPentagonDiagonalLiveCodeBench / atcoder / easy: Pentagon [LCB id: abc333_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 250
OUT224 |
| × |
LcbSetMealLiveCodeBench / atcoder / hard: Set Meal [LCB id: abc331_e]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SetMeal case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
28:23: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
45:36: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 306
OUT416 |
| ✓ |
LcbSwappingPuzzleLiveCodeBench / atcoder / hard: Swapping Puzzle [LCB id: abc332_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 364
OUT4 007 |
| ✓ |
LcbThreeThreesLiveCodeBench / atcoder / easy: Three Threes [LCB id: abc333_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 142
OUT70 |
| × |
LcbTilePatternLiveCodeBench / atcoder / hard: Tile Pattern [LCB id: abc331_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TilePattern case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:36: Char literals must contain exactly one character and close with '''.
3:38: Unexpected character '×'.
5:54: Unexpected character '×'.
11:65: Unexpected character '×'.
16:13: Unexpected character '×'.
17:32: Unexpected character '→'.
17:172: Unexpected character '→'.
17:358: Unexpected character '→'.
23:15: Unexpected character '×'.
24:15: Unexpected character '×'.
25:18: Unexpected character '×'.
50:95: Unexpected character '≤'.
Values:
run.Compiled = false |
Failed |
IN9 346
OUT1 641 |
| ✓ |
LcbTShirtsLiveCodeBench / atcoder / medium: T-Shirts [LCB id: abc332_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 396
OUT270 |
Code Generation - LiveCodeBench v1 AtCoder (set 2)
2 passed6 failed |
25.0% |
IN99 425
OUT11 995 |
| × |
LcbAlmostEqualRearrangeLiveCodeBench / atcoder abc302_c / medium: Almost Equal (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AlmostEqualRearrange case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
77:33: The if condition must be 'bool', but it has type 'int'.
91:36: The if condition must be 'bool', but it has type 'int'.
Values:
run.Compiled = false |
Failed |
IN23 525
OUT4 217 |
| ✓ |
LcbDashHealthLiveCodeBench / atcoder abc303_c / medium: Dash (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 312
OUT366 |
| ✓ |
LcbDiscordBadMoodLiveCodeBench / atcoder abc303_b / easy: Discord (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 237
OUT224 |
| × |
LcbImpartialGiftLiveCodeBench / atcoder abc302_d / medium: Impartial Gift (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ImpartialGift case0: expected [8] got [5] |
Failed |
IN9 262
OUT280 |
| × |
LcbIsolationQueriesLiveCodeBench / atcoder abc302_e / hard: Isolation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: IsolationQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:27: Type 'List<int>' has no supported method named 'Remove' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 249
OUT344 |
| × |
LcbMergeSetLiveCodeBench / atcoder abc302_f / hard: Merge Set (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MergeSet case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
475:15: Expected ')' after the while condition, found end of file.
Values:
run.Compiled = false |
Failed |
IN20 317
OUT5 973 |
| × |
LcbShiftVsCapsLockLiveCodeBench / atcoder abc303_d / medium: Shift vs. CapsLock (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ShiftVsCapsLock case0: expected [9] got [1000000000000000000] |
Failed |
IN9 272
OUT408 |
| × |
LcbSimilarStringLiveCodeBench / atcoder abc303_a / easy: Similar String (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SimilarString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:19: Function 'Similar' must return 'string', but this return statement has type 'bool'.
4:61: Function 'Similar' must return 'string', but this return statement has type 'bool'.
5:61: Function 'Similar' must return 'string', but this return statement has type 'bool'.
6:5: Function 'Similar' must return 'string', but this return statement has type 'bool'.
16:13: Operator '!' requires 'bool', but the operand has type 'string'.
Values:
run.Compiled = false |
Failed |
IN9 251
OUT183 |
Code Generation - LiveCodeBench v1 AtCoder (set 20)
2 passed5 failed1 skipped |
28.6% |
IN93 836
OUT13 861 |
| × |
LcbChristmasColorGridLiveCodeBench / atcoder / hard: Christmas Color Grid 1 [LCB id: abc334_e]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ChristmasColorGrid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:40: Unexpected character '#'.
5:5: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:7: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:49: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:51: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:82: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:84: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:27: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:29: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:36: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:37: Unexpected character '\'.
7:46: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:116: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:122: Unexpected character '\'.
7:129: Unexpected character '\'.
7:143: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:9: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:10: Unexpected character '\'.
9:19: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:31: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:33: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:47: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:49: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:67: Char literals must contain exactly one character and close with '''.
10:6: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:12: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:38: Unexpected character '→'.
10:40: Unexpected character 'Δ'.
11:6: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:9: Unexpected character '≥'.
11:12: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:115: Unexpected character '−'.
11:118: Unexpected character '…'.
11:209: Char literals must contain exactly one character and close with '''.
11:412: Unexpected character 'Δ'.
11:418: Unexpected character '−'.
13:14: Unexpected character 'Δ'.
13:23: Unexpected character '−'.
13:121: Unexpected character 'Δ'.
13:139: Unexpected character '−'.
13:165: Unexpected character 'Δ'.
13:171: Unexpected character '−'.
15:24: Unexpected character 'Δ'.
15:33: Unexpected character '−'.
15:35: Unexpected character 'Σ'.
17:30: Unexpected character '−'.
17:32: Unexpected character 'Σ'.
17:50: Unexpected character '−'.
17:53: Unexpected character 'Σ'.
19:13: Unexpected character 'Σ'.
19:331: Unexpected character 'Σ'.
21:101: Char literals must contain exactly one character and close with '''.
25:21: Unexpected character '−'.
25:318: Unexpected character '≥'.
25:360: Char literals must contain exactly one character and close with '''.
33:25: Unexpected character '−'.
33:55: Unexpected character '−'.
51:4: Char literals must contain exactly one character and close with '''.
59:122: Unexpected character 'Δ'.
59:188: Unexpected character '−'.
59:196: Unexpected character '−'.
61:109: Unexpected character 'Δ'.
61:179: Unexpected character '−'.
63:120: Unexpected character '−'.
71:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 375
OUT2 122 |
| ✓ |
LcbChristmasPresentLiveCodeBench / atcoder / easy: Christmas Present [LCB id: abc334_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 159
OUT70 |
| × |
LcbChristmasTreesLiveCodeBench / atcoder / medium: Christmas Trees [LCB id: abc334_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ChristmasTrees case0: expected [3] got [2] |
Failed |
IN9 236
OUT146 |
| × |
LcbEraseLeavesLiveCodeBench / atcoder / medium: Erase Leaves [LCB id: abc333_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: EraseLeaves case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
47:24: Type 'List<int>' has no supported method named 'Remove' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 238
OUT414 |
| × |
LcbReindeerSleighLiveCodeBench / atcoder / medium: Reindeer and Sleigh [LCB id: abc334_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ReindeerSleigh case0: expected [3
1
4] got [4
2
4] |
Failed |
IN9 254
OUT332 |
| ✓ |
LcbRepunitTrioLiveCodeBench / atcoder / medium: Repunit Trio [LCB id: abc333_c]1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN38 319
OUT4 162 |
| × |
LcbSocks2LiveCodeBench / atcoder / medium: Socks 2 [LCB id: abc334_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Socks2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:8: Expected '(' to declare a function or '=' to declare a global variable after 'parse N', found ','. Global variables must have an initializer.
Values:
run.Compiled = false |
Failed |
IN9 255
OUT6 615 |
| – |
LcbTakahashiQuestLiveCodeBench / atcoder / hard: Takahashi Quest [LCB id: abc333_e]0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: non-unique output: any valid pick-up sequence achieving K_min is accepted, cannot value-compare against a single oracle line |
Skipped |
IN0
OUT0 |
Code Generation - LiveCodeBench v1 AtCoder (set 21)
5 passed3 failed1 skipped |
62.5% |
IN73 608
OUT3 713 |
| ✓ |
Lcb2023To2024LiveCodeBench / atcoder / abc335_a: 2023 -> 2024 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT53 |
| ✓ |
LcbCtzLiveCodeBench / atcoder / abc336_b: CTZ (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 176
OUT71 |
| × |
LcbEvenDigitsLiveCodeBench / atcoder / abc336_c: Even Digits (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: EvenDigits case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:32: 'Pow' expects 2 numeric arguments.
9:29: 'Pow' expects 2 numeric arguments.
13:24: 'Pow' expects 2 numeric arguments.
14:24: 'Pow' expects 2 numeric arguments.
18:34: 'Pow' expects 2 numeric arguments.
Values:
run.Compiled = false |
Failed |
IN9 233
OUT182 |
| ✓ |
LcbLongLoongLiveCodeBench / atcoder / abc336_a: Long Loong (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 176
OUT81 |
| – |
LcbLoongAndTakahashiLiveCodeBench / atcoder / abc335_d: Loong and Takahashi (Pattern B, stdin) [NON-UNIQUE output]0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbLoongTrackingLiveCodeBench / atcoder / abc335_c: Loong Tracking (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: LoongTracking case0: expected [3 0
2 0
1 1
1 0
1 0] got [3 0
4 0
5 1
5 0
5 0] |
Failed |
IN9 242
OUT431 |
| × |
LcbPyramidSequenceLiveCodeBench / atcoder / abc336_d: Pyramid (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PyramidSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:39: Unexpected character '`'.
3:73: Unexpected character '`'.
39:9: Char literals must contain exactly one character and close with '''.
57:110: Unexpected character '—'.
61:199: Char literals must contain exactly one character and close with '''.
65:148: Char literals must contain exactly one character and close with '''.
75:4: Char literals must contain exactly one character and close with '''.
81:16: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 230
OUT2 583 |
| ✓ |
LcbScoreboardLiveCodeBench / atcoder / abc337_a: Scoreboard (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 195
OUT132 |
| ✓ |
LcbTetrahedralNumberLiveCodeBench / atcoder / abc335_b: Tetrahedral Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 207
OUT180 |
Code Generation - LiveCodeBench v1 AtCoder (set 22)
5 passed2 failed1 skipped |
71.4% |
IN99 551
OUT9 543 |
| – |
LcbBadJuiceLiveCodeBench / atcoder / abc337_e: Bad Juice (INTERACTIVE — not expressible as batch stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: SC# subset: interactive problem (two-way judge interaction + non-unique valid output) |
Skipped |
IN0
OUT0 |
| ✓ |
LcbCapitalizedLiveCodeBench / atcoder / abc338_a: Capitalized? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 143
OUT129 |
| × |
LcbCheatingGomokuLiveCodeBench / atcoder / abc337_d: Cheating Gomoku Narabe (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CheatingGomoku case0: expected [2] got [-1] |
Failed |
IN9 232
OUT547 |
| ✓ |
LcbExtendedAbcLiveCodeBench / atcoder / abc337_b: Extended ABC (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 198
OUT114 |
| ✓ |
LcbFrequencyLiveCodeBench / atcoder / abc338_b: Frequency (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 146
OUT158 |
| ✓ |
LcbIslandTourLiveCodeBench / atcoder / abc338_d: Island Tour (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN44 409
OUT7 844 |
| × |
LcbLeftoverRecipesLiveCodeBench / atcoder / abc338_c: Leftover Recipes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: LeftoverRecipes case1 runtime error: Division by zero.
Values:
run.Success = false |
Failed |
IN9 226
OUT497 |
| ✓ |
LcbLiningUp2LiveCodeBench / atcoder / abc337_c: Lining Up 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 197
OUT254 |
Code Generation - LiveCodeBench v1 AtCoder (set 23)
7 passed1 failed |
87.5% |
IN97 921
OUT6 697 |
| ✓ |
LcbAppendQueryLiveCodeBench / atcoder / abc340_b: Append (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 185
OUT197 |
| ✓ |
LcbArithmeticProgressionLiveCodeBench / atcoder / abc340_a: Arithmetic Progression (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 150
OUT130 |
| × |
LcbChordsLiveCodeBench / atcoder / abc338_e: Chords (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Chords case1: expected [No] got [Yes] |
Failed |
IN9 193
OUT257 |
| ✓ |
LcbLangtonsTakahashiLiveCodeBench / atcoder / abc339_b: Langton's Takahashi (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 255
OUT395 |
| ✓ |
LcbNegativeTspLiveCodeBench / atcoder / abc338_f: Negative Traveling Salesman (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 244
OUT2 493 |
| ✓ |
LcbPerfectBusLiveCodeBench / atcoder / abc339_c: Perfect Bus (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 192
OUT131 |
| ✓ |
LcbSynchronizedPlayersLiveCodeBench / atcoder / abc339_d: Synchronized Players (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN33 543
OUT3 053 |
| ✓ |
LcbTldLiveCodeBench / atcoder / abc339_a: TLD (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 159
OUT41 |
Code Generation - LiveCodeBench v1 AtCoder (set 24)
3 passed7 failed |
30.0% |
IN116 939
OUT7 500 |
| × |
LcbAlternatingStringLiveCodeBench / atcoder abc341_e / hard: Alternating String (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AlternatingString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
46:17: Cannot assign a value of type 'long' to variable 'count' of type 'int'.
Values:
run.Compiled = false |
Failed |
IN9 263
OUT1 174 |
| × |
LcbBreakdownLiveCodeBench / atcoder abc341_f / hard: Breakdown (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Breakdown case0: expected [5] got [2] |
Failed |
IN33 757
OUT2 966 |
| × |
LcbDivideAndDivideLiveCodeBench / atcoder abc340_c / medium: Divide and Divide (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DivideAndDivide case0: expected [5] got [3] |
Failed |
IN9 248
OUT65 |
| × |
LcbForeignExchangeLiveCodeBench / atcoder abc341_b / easy: Foreign Exchange (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ForeignExchange case0: expected [5] got [11] |
Failed |
IN9 301
OUT168 |
| × |
LcbMancala2LiveCodeBench / atcoder abc340_e / hard: Mancala 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Mancala2 case0: expected [0 4 2 7 2] got [1 1 4 3 6] |
Failed |
IN9 303
OUT324 |
| ✓ |
LcbOnlyOneOfTwoLiveCodeBench / atcoder abc341_d / medium: Only one of two (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 187
OUT314 |
| ✓ |
LcbPrint341LiveCodeBench / atcoder abc341_a / easy: Print 341 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 148
OUT87 |
| × |
LcbSuperTakahashiBrosLiveCodeBench / atcoder abc340_d / hard: Super Takahashi Bros. (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SuperTakahashiBros case0: expected [350] got [261] |
Failed |
IN9 292
OUT362 |
| × |
LcbTakahashiGetsLostLiveCodeBench / atcoder abc341_c / medium: Takahashi Gets Lost (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TakahashiGetsLost case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
20:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 277
OUT1 881 |
| ✓ |
LcbYayLiveCodeBench / atcoder abc342_a / easy: Yay! (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT159 |
Code Generation - LiveCodeBench v1 AtCoder (set 25)
3 passed4 failed2 skipped |
42.9% |
IN64 827
OUT2 655 |
| ✓ |
LcbAdjacencyMatrixLiveCodeBench / atcoder abc343_b / easy: Adjacency Matrix (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 254
OUT151 |
| × |
LcbDiversityOfScoresLiveCodeBench / atcoder abc343_d / medium: Diversity of Scores (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DiversityOfScores case0: expected [2
3
2
2] got [3
5
7
8] |
Failed |
IN9 274
OUT368 |
| × |
LcbLastTrainLiveCodeBench / atcoder abc342_e / hard: Last Train (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: LastTrain case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
18:38: List constructor expects a List or HashSet to copy, but got 'int'.
36:33: List constructor expects a List or HashSet to copy, but got 'int'.
43:39: List constructor expects a List or HashSet to copy, but got 'int'.
Values:
run.Compiled = false |
Failed |
IN9 411
OUT780 |
| × |
LcbManyReplacementLiveCodeBench / atcoder abc342_c / medium: Many Replacement (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ManyReplacement case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
18:16: Variable 'parts' has type 'string' but its initializer has type 'List<string>'.
20:26: Type 'char' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
21:26: Type 'char' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
Values:
run.Compiled = false |
Failed |
IN9 240
OUT312 |
| ✓ |
LcbPalindromicCubeLiveCodeBench / atcoder abc343_c / medium: 343 (palindromic cube) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 185
OUT186 |
| – |
LcbSevenCubedLiveCodeBench / atcoder abc343_e / hard: 7x7x7 (Pattern B, stdin) Non-unique output: any valid 9-integer placement is accepted; exact-match oracle cannot judge the Yes case.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: Non-unique output: any valid tuple is accepted for the Yes case; exact-match oracle cannot judge |
Skipped |
IN0
OUT0 |
| × |
LcbSquarePairLiveCodeBench / atcoder abc342_d / medium: Square Pair (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: SquarePair case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 224
OUT601 |
| ✓ |
LcbWhichIsAheadLiveCodeBench / atcoder abc342_b / easy: Which is ahead? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 239
OUT257 |
| – |
LcbWrongAnswerLiveCodeBench / atcoder abc343_a / easy: Wrong Answer (Pattern B, stdin) Non-unique output: ANY integer 0..9 not equal to A+B is accepted; exact-match oracle cannot judge it.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchSkipped: Non-unique output: any integer 0..9 != A+B is a valid answer; exact-match oracle cannot judge |
Skipped |
IN0
OUT0 |
Code Generation - LiveCodeBench v1 AtCoder (set 3)
5 passed4 failed |
55.6% |
IN83 159
OUT6 004 |
| ✓ |
LcbAbcdefgDistLiveCodeBench / atcoder abc305_b / easy: ABCDEFG (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 212
OUT163 |
| × |
LcbCookiePickerLiveCodeBench / atcoder abc305_c / medium: Snuke the Cookie Picker (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CookiePicker case0: expected [2 4] got [5 6] |
Failed |
IN9 236
OUT179 |
| ✓ |
LcbFirstPlayerLiveCodeBench / atcoder abc304_a / easy: First Player (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 208
OUT224 |
| × |
LcbGiftFromStarsLiveCodeBench / atcoder abc303_e / hard: A Gift From the Stars (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: GiftFromStars case0 runtime error: The dictionary does not contain the key -1. Use ContainsKey to check for it first.
Values:
run.Success = false |
Failed |
IN9 264
OUT723 |
| × |
LcbGoodGraphLiveCodeBench / atcoder abc304_e / hard: Good Graph (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GoodGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 260
OUT3 262 |
| ✓ |
LcbPieceOfCakeLiveCodeBench / atcoder abc304_d / medium: A Piece of Cake (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 258
OUT862 |
| × |
LcbSubscribersLiveCodeBench / atcoder abc304_b / easy: Subscribers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Subscribers case0: expected [20200000] got [20000000] |
Failed |
IN9 316
OUT93 |
| ✓ |
LcbVirusSpreadLiveCodeBench / atcoder abc304_c / medium: Virus (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 207
OUT411 |
| ✓ |
LcbWaterStationLiveCodeBench / atcoder abc305_a / easy: Water Station (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 198
OUT87 |
Code Generation - LiveCodeBench v1 AtCoder (set 4)
3 passed5 failed |
37.5% |
IN73 793
OUT2 299 |
| × |
LcbArtGalleryLiveCodeBench / atcoder abc305_e / hard: Art Gallery on Graph (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: ArtGallery case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 254
OUT571 |
| ✓ |
LcbBase2LiveCodeBench / atcoder abc306_b / easy: Base 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 197
OUT100 |
| ✓ |
LcbBestPerformancesLiveCodeBench / atcoder abc306_e / hard: Best Performances (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 232
OUT282 |
| × |
LcbCentersLiveCodeBench / atcoder abc306_c / medium: Centers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: Centers case0 runtime error: List index 3 is out of range. The list has 3 element(s).
Values:
run.Success = false |
Failed |
IN9 208
OUT464 |
| ✓ |
LcbEchoStringLiveCodeBench / atcoder abc306_a / easy: Echo (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 172
OUT79 |
| × |
LcbPoisonousFullLiveCodeBench / atcoder abc306_d / medium: Poisonous Full (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PoisonousFull case0: expected [600] got [1000] |
Failed |
IN9 259
OUT189 |
| × |
LcbSleepLogLiveCodeBench / atcoder abc305_d / hard: Sleep Log (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SleepLog case0: expected [480
0
960] got [120
0
960] |
Failed |
IN9 261
OUT473 |
| × |
LcbWeeklyRecordsLiveCodeBench / atcoder abc307_a / easy: Weekly Records (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WeeklyRecords case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:16: Variable 'parts' has type 'List<long>' but its initializer has type 'List<string>'.
12:36: 'long.Parse' expects a string argument, but it has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 210
OUT141 |
Code Generation - LiveCodeBench v1 AtCoder (set 5)
6 passed2 failed |
75.0% |
IN126 512
OUT10 702 |
| ✓ |
LcbDefaultPriceLiveCodeBench / atcoder abc308_b / easy: Default Price (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 213
OUT219 |
| × |
LcbDistinctAdjacentLiveCodeBench / atcoder abc307_e / hard: Distinct Adjacent (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DistinctAdjacent case0: expected [6] got [9] |
Failed |
IN9 214
OUT207 |
| × |
LcbIdealSheetLiveCodeBench / atcoder abc307_c / medium: Ideal Sheet (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: IdealSheet case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:38: Expected ';' after the variable declaration, found ','.
Values:
run.Compiled = false |
Failed |
IN9 303
OUT3 928 |
| ✓ |
LcbMismatchedParensLiveCodeBench / atcoder abc307_d / medium: Mismatched Parentheses (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 186
OUT246 |
| ✓ |
LcbNewSchemeLiveCodeBench / atcoder abc308_a / easy: New Scheme (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 205
OUT181 |
| ✓ |
LcbRacecarPalindromeLiveCodeBench / atcoder abc307_b / easy: racecar (Pattern B, stdin)1 AI query
3 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN49 796
OUT3 411 |
| ✓ |
LcbSnukeMazeLiveCodeBench / atcoder abc308_d / medium: Snuke Maze (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN21 395
OUT2 179 |
| ✓ |
LcbStandingsLiveCodeBench / atcoder abc308_c / medium: Standings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 200
OUT331 |
Code Generation - LiveCodeBench v1 AtCoder (set 6)
3 passed6 failed |
33.3% |
IN135 795
OUT10 161 |
| × |
LcbAddEdgeMaxDistanceLiveCodeBench / atcoder / medium: abc309_d Add One Edge1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AddEdgeMaxDistance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:29: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed |
IN9 357
OUT592 |
| ✓ |
LcbDrinkCouponLiveCodeBench / atcoder / easy: abc310_a Order Something Else1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 227
OUT169 |
| × |
LcbFamilyInsuranceLiveCodeBench / atcoder / hard: abc309_e Family and Insurance1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: FamilyInsurance case0 runtime error: List index 7 is out of range. The list has 7 element(s).
Values:
run.Success = false |
Failed |
IN22 312
OUT3 423 |
| × |
LcbMedicineDayLiveCodeBench / atcoder / medium: abc309_c Medicine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MedicineDay case0: expected [3] got [1] |
Failed |
IN9 238
OUT167 |
| × |
LcbMexTriplesLiveCodeBench / atcoder / hard: abc308_e MEX1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MexTriples case0: expected [3] got [0] |
Failed |
IN9 257
OUT520 |
| ✓ |
LcbNineAdjacentLiveCodeBench / atcoder / easy: abc309_a Nine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 226
OUT151 |
| × |
LcbRotateOuterLiveCodeBench / atcoder / easy: abc309_b Rotate1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RotateOuter case0: expected [1010
1101
0111
0001] got [0011
0101
1111
0000] |
Failed |
IN9 316
OUT639 |
| ✓ |
LcbStrictlySuperiorLiveCodeBench / atcoder / easy: abc310_b Strictly Superior1 AI query
1 tool callcategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN20 902
OUT863 |
| × |
LcbVouchersDiscountLiveCodeBench / atcoder / hard: abc308_f Vouchers1 AI query
2 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: VouchersDiscount case0: expected [4] got [2] |
Failed |
IN36 960
OUT3 637 |
Code Generation - LiveCodeBench v1 AtCoder (set 7)
2 passed4 failed1 broken1 skipped |
28.6% |
IN64 957
OUT64 520 |
| – |
LcbFindCycleLiveCodeBench / atcoder / medium: abc311_c Find it! (NON-UNIQUE output: any valid directed cycle is accepted; exact-match unreliable)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: non-unique output: any valid directed cycle is accepted, exact string match unreliable |
Skipped |
IN0
OUT0 |
| ✓ |
LcbFirstAbcLiveCodeBench / atcoder / easy: abc311_a First ABC1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 191
OUT101 |
| × |
LcbGridIceFloorLiveCodeBench / atcoder / medium: abc311_d Grid Ice Floor1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: GridIceFloor case0 runtime error: List index 6 is out of range. The list has 6 element(s).
Values:
run.Success = false |
Failed |
IN9 349
OUT732 |
| × |
LcbMake10AgainLiveCodeBench / atcoder / hard: abc310_f Make 10 Again1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Make10Again case0: expected [942786334] got [512326467] |
Failed |
IN9 330
OUT886 |
| × |
LcbNandRepeatedlyLiveCodeBench / atcoder / hard: abc310_e NAND repeatedly1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NandRepeatedly case0: expected [9] got [8] |
Failed |
IN9 296
OUT248 |
| ! |
LcbPeacefulTeamsLiveCodeBench / atcoder / medium: abc310_d Peaceful Teams1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: AI output broken/partial |
Broken |
IN9 273
OUT62 144 |
| × |
LcbReversibleSticksLiveCodeBench / atcoder / medium: abc310_c Reversible1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ReversibleSticks case0: expected [3] got [5] |
Failed |
IN9 267
OUT179 |
| ✓ |
LcbVacationTogetherLiveCodeBench / atcoder / easy: abc311_b Vacation Together1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 251
OUT230 |
Code Generation - LiveCodeBench v1 AtCoder (set 8)
4 passed4 failed |
50.0% |
IN74 147
OUT3 347 |
| ✓ |
LcbBecomeStrongestLiveCodeBench / atcoder / easy: abc313_a To Be Saikyo1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 204
OUT120 |
| × |
LcbCansOpenersLiveCodeBench / atcoder / hard: abc312_f Cans and Openers1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CansOpeners case2: expected [30] got [19] |
Failed |
IN9 295
OUT427 |
| ✓ |
LcbChordCheckLiveCodeBench / atcoder / easy: abc312_a Chord1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 156
OUT73 |
| × |
LcbCountBracketsLiveCodeBench / atcoder / medium: abc312_d Count Bracket Sequences1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountBrackets case0: expected [2] got [0] |
Failed |
IN9 253
OUT380 |
| ✓ |
LcbCuboidTangencyLiveCodeBench / atcoder / hard: abc312_e Tangency of Cuboids1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 343
OUT1 097 |
| × |
LcbHolelessSquaresLiveCodeBench / atcoder / hard: abc311_e Defect1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: HolelessSquares case0: expected [6] got [5] |
Failed |
IN9 324
OUT446 |
| × |
LcbInvisibleHandLiveCodeBench / atcoder / medium: abc312_c Invisible Hand1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: InvisibleHand case0: expected [110] got [81] |
Failed |
IN9 212
OUT327 |
| ✓ |
LcbTakCodeLiveCodeBench / atcoder / easy: abc312_b TaK Code1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 360
OUT477 |
Code Generation - LiveCodeBench v1 AtCoder (set 9)
5 passed3 failed1 skipped |
62.5% |
IN73 922
OUT6 957 |
| × |
LcbApproxEqualizationLiveCodeBench / atcoder / medium: abc313_c Approximate Equalization 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ApproxEqualization case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:43: Type 'List<string>' has no supported method named 'ToList' with 0 argument(s).
8:26: 'long.Parse' expects a string argument, but it has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 191
OUT234 |
| × |
LcbCertainGameExpectedLiveCodeBench / atcoder / hard: abc314_f A Certain Game (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CertainGameExpected case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:61: Char literals must contain exactly one character and close with '''.
13:354: Char literals must contain exactly one character and close with '''.
15:139: Unexpected character '#'.
19:118: Char literals must contain exactly one character and close with '''.
19:312: Unexpected character '—'.
19:321: Char literals must contain exactly one character and close with '''.
25:349: Unexpected character '—'.
25:647: Unexpected character '—'.
27:530: Unexpected character '→'.
29:118: Unexpected character '≤'.
35:327: Unexpected character '`'.
35:337: Unexpected character '`'.
35:590: Unexpected character '—'.
35:595: Char literals must contain exactly one character and close with '''.
43:40: Unexpected character '≤'.
51:11: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 306
OUT2 321 |
| ✓ |
LcbLowerUpperOpsLiveCodeBench / atcoder / medium: abc314_d LOWER (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 225
OUT386 |
| ✓ |
LcbPiTruncateLiveCodeBench / atcoder / easy: abc314_a 3.14 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 270
OUT244 |
| ✓ |
LcbRemoveVowelsLiveCodeBench / atcoder / easy: abc315_a tcdr (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 155
OUT94 |
| ✓ |
LcbRotateColoredSubsequenceLiveCodeBench / atcoder / medium: abc314_c Rotate Colored Subsequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 254
OUT391 |
| × |
LcbRouletteFewestBetsLiveCodeBench / atcoder / easy: abc314_b Roulette (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RouletteFewestBets case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:1: Expected '(' to declare a function or '=' to declare a global variable after 'N C_1', found 'A_'. Global variables must have an initializer.
Values:
run.Compiled = false |
Failed |
IN9 271
OUT2 872 |
| – |
LcbRoulettesExpectedLiveCodeBench / atcoder / hard: abc314_e Roulettes (Pattern B, stdin) FLAG: float output (expected value); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: SC# stdin driver: float/expected-value output needs tolerance comparison; exact string compare unsafe |
Skipped |
IN0
OUT0 |
| ✓ |
LcbStrongestProgrammerLiveCodeBench / atcoder / medium: abc313_b Who is Saikyo? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 250
OUT415 |
Code Generation - LiveCodeBench v1 Codeforces (set 1)
5 passed4 failed |
55.6% |
IN83 336
OUT8 035 |
| × |
LcbChemistryPalindromeLiveCodeBench / codeforces 1883_B / medium: Chemistry (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ChemistryPalindrome case0: expected [YES
NO
YES
YES
YES
YES
NO
NO
YES
YES
YES
YES
NO
YES] got [NO
NO
YES
NO
YES
NO
NO
NO
YES
YES
NO
YES
NO
NO] |
Failed |
IN9 252
OUT435 |
| × |
LcbGameWithIntegersLiveCodeBench / codeforces 1899_A / easy: Game with Integers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GameWithIntegers case0: expected [First
Second
First
First
Second
First] got [Second
First
First
Second
First
Second] |
Failed |
IN9 257
OUT121 |
| ✓ |
LcbGoodKidLiveCodeBench / codeforces 1873_B / easy: Good Kid (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBench |
Passed |
IN9 246
OUT276 |
| ✓ |
LcbOneDEraserLiveCodeBench / codeforces 1873_D / easy: 1D Eraser (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBench |
Passed |
IN9 242
OUT205 |
| ✓ |
LcbRaspberriesLiveCodeBench / codeforces 1883_C / medium: Raspberries (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Mediumsource:LiveCodeBench |
Passed |
IN9 250
OUT3 053 |
| ✓ |
LcbShortSortLiveCodeBench / codeforces 1873_A / easy: Short Sort (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBench |
Passed |
IN9 229
OUT182 |
| × |
LcbTruckWeightDifferenceLiveCodeBench / codeforces 1899_B / hard: 250 Thousand Tons of TNT (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TruckWeightDifference case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:26: Unexpected character '→'.
7:64: Unexpected character '→'.
29:234: Unexpected character '≤'.
29:254: Char literals must contain exactly one character and close with '''.
33:88: Char literals must contain exactly one character and close with '''.
35:89: Unexpected character '≈'.
39:96: Unexpected character '≈'.
41:11: Unexpected character '→'.
41:17: Unexpected character '→'.
45:347: Char literals must contain exactly one character and close with '''.
61:13: Unexpected character '#'.
63:55: Unexpected character '→'.
67:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 308
OUT1 800 |
| × |
LcbYarikArrayAltParityLiveCodeBench / codeforces 1899_C / hard: Yarik and Array (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: YarikArrayAltParity case0: expected [15
17
8
4
-1000
101
10] got [15
17
7
4
-1000
101
10] |
Failed |
IN9 270
OUT276 |
| ✓ |
LcbYarikMusicalNotesLiveCodeBench / codeforces 1899_D / hard: Yarik and Musical Notes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBench |
Passed |
IN9 282
OUT1 687 |
Code Generation - LiveCodeBench v1 LeetCode (set 1)
5 passed4 failed |
55.6% |
IN82 636
OUT2 213 |
| ✓ |
LcbBuyChocoLiveCodeBench / leetcode / easy: buy-two-chocolates (LCB 2756)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 158
OUT78 |
| ✓ |
LcbColorTheArrayLiveCodeBench / leetcode / medium: number-of-adjacent-elements-with-the-same-color (LCB 2779)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 235
OUT264 |
| × |
LcbCountOfIntegersLiveCodeBench / leetcode / hard: count-of-integers (LCB 2757)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: count case0: expected 11 got 4 |
Failed |
IN9 202
OUT534 |
| ✓ |
LcbCountSeniorsLiveCodeBench / leetcode / easy: number-of-senior-citizens (LCB 2727)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 238
OUT87 |
| ✓ |
LcbDistinctDifferenceArrayLiveCodeBench / leetcode / easy: find-the-distinct-difference-array (LCB 2777)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 179
OUT216 |
| × |
LcbMatrixSumLiveCodeBench / leetcode / medium: sum-in-a-matrix (LCB 2728)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: matrixSum case0 runtime error: List index 7 is out of range. The list has 3 element(s).
Values:
run.Success = false |
Failed |
IN9 153
OUT245 |
| × |
LcbMaximumOrLiveCodeBench / leetcode / medium: maximum-or (LCB 2730)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maximumOr case0: expected 30 got 13 |
Failed |
IN9 165
OUT156 |
| ✓ |
LcbMaxStrengthLiveCodeBench / leetcode / medium: maximum-strength-of-a-group (LCB 2754)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 144
OUT431 |
| × |
LcbMinExtraCharLiveCodeBench / leetcode / medium: extra-characters-in-a-string (LCB 2755)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minExtraChar case0: expected 1 got 0 |
Failed |
IN9 162
OUT202 |
Code Generation - LiveCodeBench v1 LeetCode (set 10)
2 passed7 failed |
22.2% |
IN83 020
OUT4 175 |
| ✓ |
LcbCanBeEqualSwapLiveCodeBench / leetcode / easy: check-if-strings-can-be-made-equal-with-operations-i (id 2999)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 184
OUT363 |
| × |
LcbCanMakeSubsequenceCyclicLiveCodeBench / leetcode / medium: make-string-a-subsequence-using-cyclic-increments (id 3018)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: canMakeSubseq case0: expected true got false |
Failed |
IN9 249
OUT139 |
| × |
LcbConstructProductMatrixLiveCodeBench / leetcode / medium: construct-product-matrix (id 3031)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<List<int>>>(run.ResultJson) == FromJson<List<List<int>>>(expected[i])
Context: constructProduct case0: expected [[24, 12], [8, 6]] got [[6,16],[36,96]] |
Failed |
IN9 198
OUT535 |
| × |
LcbFurthestDistanceFromOriginLiveCodeBench / leetcode / easy: furthest-point-from-origin (id 3019)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: furthestOrigin case1: expected 5 got 3 |
Failed |
IN9 196
OUT70 |
| × |
LcbMaxBallPassingValueLiveCodeBench / leetcode / hard: maximize-value-of-function-in-a-ball-passing-game (id 3032)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxBallPassing case1 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 310
OUT635 |
| ✓ |
LcbMinAbsoluteDifferenceApartLiveCodeBench / leetcode / medium: minimum-absolute-difference-between-elements-with-constraint (id 3000)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 206
OUT111 |
| × |
LcbMinOperationsSubsequenceSumLiveCodeBench / leetcode / hard: minimum-operations-to-form-subsequence-with-target-sum (id 3025)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsSubseqSum case0: expected 1 got 0 |
Failed |
IN9 263
OUT421 |
| × |
LcbNumberOfBeautifulIntegersLiveCodeBench / leetcode / hard: number-of-beautiful-integers-in-the-range (id 3017)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: numBeautiful case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 184
OUT1 348 |
| × |
LcbStringTransformationWaysLiveCodeBench / leetcode / hard: string-transformation (id 3024)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: stringTransform case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
26:17: The shift count of '>>' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed |
IN9 230
OUT553 |
Code Generation - LiveCodeBench v1 LeetCode (set 11)
5 passed4 failed |
55.6% |
IN82 747
OUT2 249 |
| × |
LcbMaximumCompleteSubsetSumLiveCodeBench / leetcode / hard: maximum-element-sum-of-a-complete-subset-of-indices (id 3047)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxCompleteSubset case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:46: Expected ')' to close the argument list, found 'var'.
Values:
run.Compiled = false |
Failed |
IN9 236
OUT303 |
| ✓ |
LcbMaximumOddBinaryNumberLiveCodeBench / leetcode / easy: maximum-odd-binary-number (id 3055)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 152
OUT150 |
| ✓ |
LcbMinArrayLengthAfterRemovalsLiveCodeBench / leetcode / medium: minimum-array-length-after-pair-removals (id 3081)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 210
OUT184 |
| × |
LcbMinCostMakeStringsEqualLiveCodeBench / leetcode / medium: apply-operations-to-make-two-strings-equal (id 3033)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minCostEqual case0: expected 4 got -1 |
Failed |
IN9 283
OUT369 |
| ✓ |
LcbMinimumRightShiftsLiveCodeBench / leetcode / easy: minimum-right-shifts-to-sort-the-array (id 3045)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 161
OUT162 |
| ✓ |
LcbMinOperationsCollectElementsLiveCodeBench / leetcode / easy: minimum-operations-to-collect-elements (id 3044)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 172
OUT124 |
| × |
LcbMinOperationsSpecialNumberLiveCodeBench / leetcode / medium: minimum-operations-to-make-a-special-number (id 3046)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsSpecial case1: expected 3 got 1 |
Failed |
IN9 160
OUT699 |
| ✓ |
LcbNumberOfPointsCarsLiveCodeBench / leetcode / easy: points-that-intersect-with-cars (id 3034)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 171
OUT93 |
| × |
LcbSplitArrayMaxSubarraysLiveCodeBench / leetcode / medium: split-array-into-maximum-number-of-subarrays (id 3080)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSubarrays case0: expected 3 got 6 |
Failed |
IN9 202
OUT165 |
Code Generation - LiveCodeBench v1 LeetCode (set 12)
3 passed5 failed |
37.5% |
IN73 740
OUT2 358 |
| × |
LcbBeautifulTowersMaxSumLiveCodeBench / leetcode / medium: beautiful-towers-i (id 3114)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: beautifulTowers case0: expected 13 got 1 |
Failed |
IN9 266
OUT314 |
| ✓ |
LcbCountSubMultisetsBoundedSumLiveCodeBench / leetcode / hard: count-of-sub-multisets-with-bounded-sum (id 3091)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 293
OUT508 |
| × |
LcbCountWaysHappyStudentsLiveCodeBench / leetcode / medium: happy-students (id 3104)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countWaysHappy case0: expected 2 got 3 |
Failed |
IN9 202
OUT213 |
| × |
LcbLongestSubsequenceSumTargetLiveCodeBench / leetcode / medium: length-of-the-longest-subsequence-that-sums-to-target (id 3106)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestSubseqSum case0: expected 3 got 2 |
Failed |
IN9 202
OUT259 |
| ✓ |
LcbMinOperationsMakeArrayEmptyLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-array-empty (id 3094)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 177
OUT221 |
| × |
LcbMinSizeSubarrayInfiniteLiveCodeBench / leetcode / medium: minimum-size-subarray-in-infinite-array (id 3141)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minSizeSubarray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
31:49: Operator '+' cannot be applied to 'int' and 'bool'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 183
OUT360 |
| × |
LcbShortestBeautifulSubstringLiveCodeBench / leetcode / medium: shortest-and-lexicographically-smallest-beautiful-string (id 3150)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: shortestBeautiful case0: expected "11001" got "00011001" |
Failed |
IN9 227
OUT353 |
| ✓ |
LcbSumIndicesKSetBitsLiveCodeBench / leetcode / easy: sum-of-values-at-indices-with-k-set-bits (id 3093)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 190
OUT130 |
Code Generation - LiveCodeBench v1 LeetCode (set 13)
3 passed6 failed |
33.3% |
IN82 352
OUT2 096 |
| ✓ |
LcbDivisibleSumsDifferenceLiveCodeBench / leetcode / easy: 3172 divisible-and-non-divisible-sums-difference (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 118
OUT98 |
| ✓ |
LcbLastVisitedIntegersLiveCodeBench / leetcode / easy: 3164 last-visited-integers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 202
OUT162 |
| × |
LcbMaxOrderedTripletValueLiveCodeBench / leetcode / easy: 3154 maximum-value-of-an-ordered-triplet-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxOrderedTripletValue case0: expected 77 got 42 |
Failed |
IN9 131
OUT151 |
| × |
LcbMaxSumOfSquaresLiveCodeBench / leetcode / hard: 3153 apply-operations-on-array-to-maximize-sum-of-squares (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxSumOfSquares case0: expected 261 got 850618967 |
Failed |
IN9 188
OUT328 |
| × |
LcbMinChangesBinaryBeautifulLiveCodeBench / leetcode / medium: 3174 minimum-number-of-changes-to-make-binary-string-beautiful (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinChangesBinaryBeautiful case1: expected 1 got 2 |
Failed |
IN9 124
OUT100 |
| × |
LcbMinEqualSumReplaceZerosLiveCodeBench / leetcode / medium: 3171 minimum-equal-sum-of-two-arrays-after-replacing-zeros (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MinEqualSumReplaceZeros case0: expected 12 got 16 |
Failed |
IN9 121
OUT551 |
| × |
LcbMinGroupsValidAssignmentLiveCodeBench / leetcode / medium: 3166 minimum-number-of-groups-to-create-a-valid-assignment (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinGroupsValidAssignment case0: expected 2 got 5 |
Failed |
IN9 153
OUT449 |
| × |
LcbMinProcessingTimeLiveCodeBench / leetcode / medium: 3151 minimum-processing-time (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinProcessingTime case0: expected 16 got 17 |
Failed |
IN9 193
OUT145 |
| ✓ |
LcbSumDistinctSquaresLiveCodeBench / leetcode / easy: 3163 subarrays-distinct-element-sum-of-squares-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 122
OUT112 |
Code Generation - LiveCodeBench v1 LeetCode (set 14)
5 passed4 failed |
55.6% |
IN82 225
OUT1 644 |
| ✓ |
LcbFindChampionILiveCodeBench / leetcode / easy: 3188 find-champion-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 150
OUT143 |
| ✓ |
LcbFindKOrLiveCodeBench / leetcode / easy: 3183 find-the-k-or-of-an-array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 141
OUT137 |
| ✓ |
LcbFindWordsContainingLiveCodeBench / leetcode / easy: 3194 find-words-containing-character (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 098
OUT78 |
| × |
LcbMaxBalancedSubseqSumLiveCodeBench / leetcode / hard: 3184 maximum-balanced-subsequence-sum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxBalancedSubseqSum case0: expected 14 got 9 |
Failed |
IN9 159
OUT327 |
| × |
LcbMaximumXorProductLiveCodeBench / leetcode / medium: 3192 maximum-xor-product (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaximumXorProduct case0: expected 98 got 14 |
Failed |
IN9 121
OUT233 |
| ✓ |
LcbMaxStrongPairXorLiveCodeBench / leetcode / easy: 3193 maximum-strong-pair-xor-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT156 |
| ✓ |
LcbMinMountainTripletSumLiveCodeBench / leetcode / easy: 3176 minimum-sum-of-mountain-triplets-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 118
OUT267 |
| × |
LcbMinOpsMaximizeLastLiveCodeBench / leetcode / medium: 3190 minimum-operations-to-maximize-last-elements-in-arrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinOpsMaximizeLast case0: expected 1 got 2 |
Failed |
IN9 164
OUT205 |
| × |
LcbSeparateBlackWhiteBallsLiveCodeBench / leetcode / medium: 3195 separate-black-and-white-balls (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: SeparateBlackWhiteBalls case1: expected 2 got 0 |
Failed |
IN9 125
OUT98 |
Code Generation - LiveCodeBench v1 LeetCode (set 15)
3 passed5 failed |
37.5% |
IN73 170
OUT2 111 |
| × |
LcbCountBeautifulSubstringsLiveCodeBench / leetcode / medium: 3210 count-beautiful-substrings-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: CountBeautifulSubstrings case0: expected 2 got 0 |
Failed |
IN9 145
OUT282 |
| ✓ |
LcbDistributeCandiesILiveCodeBench / leetcode / easy: 3199 distribute-candies-among-children-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 086
OUT204 |
| ✓ |
LcbFindCommonElementsLiveCodeBench / leetcode / easy: 3206 find-common-elements-between-two-arrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 161
OUT153 |
| × |
LcbGoodStringsLeetLiveCodeBench / leetcode / medium: 3200 number-of-strings-which-can-be-rearranged-to-contain-substring (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: GoodStringsLeet case1: expected 83943898 got 929687486 |
Failed |
IN9 117
OUT117 |
| ✓ |
LcbMakeThreeStringsEqualLiveCodeBench / leetcode / easy: 3207 make-three-strings-equal (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 148
OUT167 |
| × |
LcbMaxFrequencyScoreLiveCodeBench / leetcode / hard: 3196 apply-operations-to-maximize-frequency-score (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxFrequencyScore case0: expected 3 got 2 |
Failed |
IN9 129
OUT459 |
| × |
LcbMinCoinsForFruitsLiveCodeBench / leetcode / medium: 3209 minimum-number-of-coins-for-fruits (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinCoinsForFruits case0: expected 4 got 6 |
Failed |
IN9 150
OUT219 |
| × |
LcbPalindromeRearrangeQueriesLiveCodeBench / leetcode / hard: 3203 palindrome-rearrangement-queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PalindromeRearrangeQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'canMakePalindromeQueries' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, CanMakePalindromeQueries.
Values:
run.Compiled = false |
Failed |
IN9 234
OUT510 |
Code Generation - LiveCodeBench v1 LeetCode (set 16)
5 passed4 failed |
55.6% |
IN82 354
OUT2 996 |
| ✓ |
LcbCompleteSubstringsLiveCodeBench / leetcode / hard: count-complete-substrings (id 3223)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 145
OUT412 |
| ✓ |
LcbCountTestedDevicesLiveCodeBench / leetcode / easy: count-tested-devices-after-test-operations (id 3220)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT106 |
| ✓ |
LcbFindPeaksLiveCodeBench / leetcode / easy: find-the-peaks (id 3221)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 117
OUT101 |
| × |
LcbGoodPartitionsCountLiveCodeBench / leetcode / hard: count-the-number-of-good-partitions (id 3212)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: goodPart case0: expected 8 got 4 |
Failed |
IN9 118
OUT276 |
| × |
LcbInfectionSequencesLiveCodeBench / leetcode / hard: count-the-number-of-infection-sequences (id 3224)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: infectionSeq case0: expected 4 got 144 |
Failed |
IN9 217
OUT877 |
| ✓ |
LcbLexSmallestSwapLiveCodeBench / leetcode / medium: make-lexicographically-smallest-array-by-swapping-elements (id 3219)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 174
OUT432 |
| ✓ |
LcbLongestSubarrayKFreqLiveCodeBench / leetcode / medium: length-of-longest-subarray-with-at-most-k-frequency (id 3225)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 133
OUT206 |
| × |
LcbMaxNonDecreasingLengthLiveCodeBench / leetcode / hard: find-maximum-non-decreasing-array-length (id 3211)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxNonDec case0: expected 1 got 3 |
Failed |
IN9 157
OUT315 |
| × |
LcbSubarraysMaxKTimesLiveCodeBench / leetcode / medium: count-subarrays-where-max-element-appears-at-least-k-times (id 3213)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: subMaxK case0: expected 6 got 3 |
Failed |
IN9 101
OUT271 |
Code Generation - LiveCodeBench v1 LeetCode (set 17)
3 passed6 failed |
33.3% |
IN82 456
OUT2 408 |
| × |
LcbDoubleModExpLiveCodeBench / leetcode / medium: double-modular-exponentiation (id 3234)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: doubleModExp case0: expected [0, 2] got [2] |
Failed |
IN9 150
OUT252 |
| × |
LcbEqualindromicCostLiveCodeBench / leetcode / medium: minimum-cost-to-make-array-equalindromic (id 3229)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: equalindromic case0: expected 6 got 540 |
Failed |
IN9 202
OUT356 |
| ✓ |
LcbMaxPartitionsOpsLiveCodeBench / leetcode / hard: maximize-the-number-of-partitions-after-operations (id 3233)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 166
OUT361 |
| × |
LcbMaxSetSizeRemovalsLiveCodeBench / leetcode / medium: maximum-size-of-a-set-after-removals (id 3228)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSetSize case1: expected 5 got 6 |
Failed |
IN9 131
OUT259 |
| ✓ |
LcbMinCoinsToAddLiveCodeBench / leetcode / medium: minimum-number-of-coins-to-be-added (id 3231)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 163
OUT122 |
| × |
LcbMinCostConvertStringLiveCodeBench / leetcode / medium: minimum-cost-to-convert-string-i (id 3235)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minCostConvert case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:17: Type 'long' cannot be assigned by index. Only List and Dictionary elements can be assigned.
16:24: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
16:38: Type 'long' cannot be assigned by index. Only List and Dictionary elements can be assigned.
24:25: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
28:35: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
28:50: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
29:34: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
29:50: Type 'long' cannot be assigned by index. Only List and Dictionary elements can be assigned.
39:20: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
40:23: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
Values:
run.Compiled = false |
Failed |
IN9 234
OUT449 |
| × |
LcbMinimumNumberGameLiveCodeBench / leetcode / easy: minimum-number-game (id 3226)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: numberGame case0: expected [3, 2, 5, 4] got [4,5,3,2] |
Failed |
IN9 139
OUT103 |
| ✓ |
LcbMissingAndRepeatedLiveCodeBench / leetcode / easy: find-missing-and-repeated-values (id 3227)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 153
OUT284 |
| × |
LcbRemoveAlmostEqualLiveCodeBench / leetcode / medium: remove-adjacent-almost-equal-characters (id 3230)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: removeAlmost case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:13: Variable 'c1' has type 'int' but its initializer has type 'char'.
10:13: Variable 'c2' has type 'int' but its initializer has type 'char'.
Values:
run.Compiled = false |
Failed |
IN9 118
OUT222 |
Code Generation - LiveCodeBench v1 LeetCode (set 18)
5 passed3 failed |
62.5% |
IN73 246
OUT2 157 |
| ✓ |
LcbBeautifulIndicesLiveCodeBench / leetcode / medium: find-beautiful-indices-in-the-given-array-i (id 3245)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 174
OUT247 |
| ✓ |
LcbBitwiseOrTrailingZerosLiveCodeBench / leetcode / easy: check-if-bitwise-or-has-trailing-zeros (id 3246)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 159
OUT70 |
| ✓ |
LcbMaxFrequencyElementsLiveCodeBench / leetcode / easy: count-elements-with-maximum-frequency (id 3242)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 093
OUT157 |
| × |
LcbMaxNumberPricesSumLiveCodeBench / leetcode / medium: maximum-number-that-sum-of-the-prices-is-less-than-or-equal-to-k (id 3240)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxNumberPrices case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:23: The shift count of '<<' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed |
IN9 193
OUT283 |
| × |
LcbMinArrayLengthModLiveCodeBench / leetcode / medium: minimize-length-of-array-using-operations (id 3244)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minArrayLen case0: expected 1 got 2 |
Failed |
IN9 156
OUT135 |
| ✓ |
LcbMinOpsXYEqualLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-x-and-y-equal (id 3239)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 133
OUT331 |
| ✓ |
LcbMissingSeqPrefixSumLiveCodeBench / leetcode / easy: smallest-missing-integer-greater-than-sequential-prefix-sum (id 3236)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 132
OUT174 |
| × |
LcbPowerfulIntegersLiveCodeBench / leetcode / hard: count-the-number-of-powerful-integers (id 3243)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: powerfulInt case0: expected 5 got 26 |
Failed |
IN9 206
OUT760 |
Code Generation - LiveCodeBench v1 LeetCode (set 19)
4 passed5 failed |
44.4% |
IN82 287
OUT2 010 |
| ✓ |
LcbAreaOfMaxDiagonalLiveCodeBench / leetcode / easy: maximum-area-of-longest-diagonal-rectangle (id 3251)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 130
OUT151 |
| ✓ |
LcbCountMatchingSubarraysLiveCodeBench / leetcode / medium: number-of-subarrays-that-match-a-pattern-i (id 3269)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 203
OUT237 |
| × |
LcbIncremovableSubarrayCountLiveCodeBench / leetcode / easy: count-the-number-of-incremovable-subarrays-i (id 3252)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: incremovable case1: expected 7 got 10 |
Failed |
IN9 123
OUT370 |
| × |
LcbLargestPerimeterLiveCodeBench / leetcode / medium: find-polygon-with-the-largest-perimeter (id 3262)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: largestPerimeter case1: expected 12 got -1 |
Failed |
IN9 189
OUT128 |
| × |
LcbLongestSpecialThriceLiveCodeBench / leetcode / medium: find-longest-special-substring-that-occurs-thrice-i (id 3267)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestSpecial case0: expected 2 got 3 |
Failed |
IN9 117
OUT406 |
| ✓ |
LcbMaximumGoodSubarraySumLiveCodeBench / leetcode / medium: maximum-good-subarray-sum (id 3265)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 134
OUT183 |
| × |
LcbMinimumCostThreeSubarraysLiveCodeBench / leetcode / easy: divide-an-array-into-subarrays-with-minimum-cost-i (id 3263)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minCost3 case0: expected 6 got 5 |
Failed |
IN9 098
OUT171 |
| ✓ |
LcbMinOpsXorEqualKLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-array-xor-equal-to-k (id 3249)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 141
OUT98 |
| × |
LcbMinOrAfterOperationsLiveCodeBench / leetcode / hard: minimize-or-of-remaining-elements-using-operations (id 3261)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOr case0: expected 3 got 2147483647 |
Failed |
IN9 152
OUT266 |
Code Generation - LiveCodeBench v1 LeetCode (set 2)
4 passed5 failed |
44.4% |
IN82 512
OUT1 605 |
| × |
LcbCircularGameLosersLiveCodeBench / leetcode / easy: find-the-losers-of-the-circular-game (LCB 2791)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: circularGameLosers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:19: The while condition must be 'bool', but it has type 'void'.
Values:
run.Compiled = false |
Failed |
IN9 186
OUT138 |
| × |
LcbCollectingChocolatesLiveCodeBench / leetcode / medium: collecting-chocolates (LCB 2810)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minCost case0: expected 13 got 36 |
Failed |
IN9 184
OUT205 |
| ✓ |
LcbLongestSemiRepetitiveLiveCodeBench / leetcode / medium: find-the-longest-semi-repetitive-substring (LCB 2786)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 161
OUT194 |
| ✓ |
LcbMinLengthRemoveSubstrLiveCodeBench / leetcode / easy: minimum-string-length-after-removing-substrings (LCB 2800)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 129
OUT133 |
| ✓ |
LcbNeighboringXorLiveCodeBench / leetcode / medium: neighboring-bitwise-xor (LCB 2792)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 189
OUT52 |
| × |
LcbPaintWallsLiveCodeBench / leetcode / hard: painting-the-walls (LCB 2808)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: paintWalls case0: expected 3 got 0 |
Failed |
IN9 195
OUT326 |
| × |
LcbPowerOfHeroesLiveCodeBench / leetcode / hard: power-of-heroes (LCB 2784)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: sumOfPower case0: expected 141 got 220 |
Failed |
IN9 156
OUT164 |
| × |
LcbPunishmentNumberLiveCodeBench / leetcode / medium: find-the-punishment-number-of-an-integer (LCB 2802)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: punishmentNumber case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:18: Unknown type 'String'. Supported types: bool, byte, int, decimal, double, char, string, Date, DateTime, TimeSpan, List<...>, Dictionary<...>, HashSet<...> and declared structs.
4:21: Function 'CanPartition' must return 'int', but this return statement has type 'bool'.
5:24: Function 'CanPartition' must return 'int', but this return statement has type 'bool'.
7:32: Function 'CanPartition' must return 'int', but this return statement has type 'bool'.
13:13: The if condition must be 'bool', but it has type 'int'.
13:63: Function 'CanPartition' must return 'int', but this return statement has type 'bool'.
15:5: Function 'CanPartition' must return 'int', but this return statement has type 'bool'.
25:13: The if condition must be 'bool', but it has type 'int'.
Values:
run.Compiled = false |
Failed |
IN9 149
OUT255 |
| ✓ |
LcbSemiOrderedPermutationLiveCodeBench / leetcode / easy: semi-ordered-permutation (LCB 2785)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT138 |
Code Generation - LiveCodeBench v1 LeetCode (set 20)
3 passed5 failed |
37.5% |
IN73 165
OUT1 885 |
| ✓ |
LcbCanSortArrayBySetBitsLiveCodeBench / leetcode / medium: find-if-array-can-be-sorted (id 3291)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 117
OUT332 |
| ✓ |
LcbCountPrefixSuffixPairsLiveCodeBench / leetcode / easy: count-prefix-and-suffix-pairs-i (id 3309)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 133
OUT135 |
| × |
LcbEarliestSecondToMarkLiveCodeBench / leetcode / medium: earliest-second-to-mark-indices-i (id 3292)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: earliestSecond case0 runtime error: List index 3 is out of range. The list has 3 element(s).
Values:
run.Success = false |
Failed |
IN9 194
OUT236 |
| ✓ |
LcbFlowerGameLiveCodeBench / leetcode / medium: alice-and-bob-playing-flower-game (id 3279)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 149
OUT103 |
| × |
LcbLastNonEmptyStringLiveCodeBench / leetcode / medium: apply-operations-to-make-string-empty (id 3308)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: lastNonEmpty case0: expected "ba" got "b" |
Failed |
IN9 101
OUT290 |
| × |
LcbMaxSelectedConsecutiveLiveCodeBench / leetcode / hard: maximize-consecutive-elements-in-an-array-after-modification (id 3298)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSelected case0: expected 3 got 4 |
Failed |
IN9 148
OUT199 |
| × |
LcbMaxSubsetPowerPatternLiveCodeBench / leetcode / medium: find-the-maximum-number-of-elements-in-subset (id 3299)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxSubsetPower case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 195
OUT421 |
| × |
LcbMinTimeRevertWordLiveCodeBench / leetcode / medium: minimum-time-to-revert-word-to-initial-state-i (id 3297)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minTimeRevert case0: expected 2 got 7 |
Failed |
IN9 128
OUT169 |
Code Generation - LiveCodeBench v1 LeetCode (set 21)
7 passed1 failed |
87.5% |
IN73 182
OUT1 144 |
| ✓ |
LcbAntOnBoundaryLiveCodeBench / leetcode / easy: ant-on-the-boundary (id 3311)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 156
OUT70 |
| ✓ |
LcbCountKeyChangesLiveCodeBench / leetcode / easy: number-of-changing-keys (id 3312)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 126
OUT167 |
| ✓ |
LcbDistributeTwoArraysLiveCodeBench / leetcode / easy: distribute-elements-into-two-arrays-i (id 3347)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 180
OUT205 |
| ✓ |
LcbLongestCommonPrefixDigitsLiveCodeBench / leetcode / medium: find-the-length-of-the-longest-common-prefix (id 3329)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 192
OUT151 |
| ✓ |
LcbMaxOperationsSameScoreLiveCodeBench / leetcode / easy: maximum-number-of-operations-with-the-same-score-i (id 3320)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 126
OUT126 |
| ✓ |
LcbMaxPalindromesAfterOpsLiveCodeBench / leetcode / medium: maximum-palindromes-after-operations (id 3317)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 169
OUT175 |
| × |
LcbMinOpsExceedThresholdLiveCodeBench / leetcode / easy: minimum-operations-to-exceed-threshold-value-i (id 3331)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minOpsExceed case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
12:13: Type 'List<int>' has no supported method named 'Remove' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 109
OUT121 |
| ✓ |
LcbSplitArrayDistinctLiveCodeBench / leetcode / easy: split-the-array (id 3324)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 124
OUT129 |
Code Generation - LiveCodeBench v1 LeetCode (set 3)
5 passed3 failed |
62.5% |
IN73 151
OUT1 667 |
| ✓ |
LcbGcdTraversalLiveCodeBench / leetcode / hard: greatest-common-divisor-traversal (LCB 2827)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 171
OUT608 |
| ✓ |
LcbIsFascinatingLiveCodeBench / leetcode / easy: check-if-the-number-is-fascinating (LCB 2824)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 138
OUT112 |
| × |
LcbKAvoidingArrayLiveCodeBench / leetcode / medium: determine-the-minimum-sum-of-a-k-avoiding-array (LCB 2811)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimumSum case0: expected 18 got 4 |
Failed |
IN9 125
OUT135 |
| ✓ |
LcbMaxAchievableNumberLiveCodeBench / leetcode / easy: find-the-maximum-achievable-number (LCB 2812)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 139
OUT27 |
| × |
LcbMinCostEqualCharsLiveCodeBench / leetcode / medium: minimum-cost-to-make-all-characters-equal (LCB 2817)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumCost case0: expected 2 got 0 |
Failed |
IN9 167
OUT377 |
| × |
LcbMinimizeStringLengthLiveCodeBench / leetcode / easy: minimize-string-length (LCB 2825)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimizedStringLength case0: expected 3 got 4 |
Failed |
IN9 156
OUT128 |
| ✓ |
LcbRemoveTrailingZerosLiveCodeBench / leetcode / easy: remove-trailing-zeros-from-a-string (LCB 2819)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 109
OUT79 |
| ✓ |
LcbSmallestPalindromeLiveCodeBench / leetcode / easy: lexicographically-smallest-palindrome (LCB 2816)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 146
OUT201 |
Code Generation - LiveCodeBench v1 LeetCode (set 4)
5 passed4 failed |
55.6% |
IN83 007
OUT2 424 |
| ✓ |
LcbBeautifulPairsLiveCodeBench / leetcode / easy: number-of-beautiful-pairs (id 2831)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 191
OUT240 |
| × |
LcbCountZeroRequestServersLiveCodeBench / leetcode / medium: count-zero-request-servers (id 2833)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: countServers case0: expected [1, 2] got [0,0] |
Failed |
IN9 290
OUT524 |
| ✓ |
LcbLongestEqualSubarrayLiveCodeBench / leetcode / medium: find-the-longest-equal-subarray (id 2832)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 195
OUT243 |
| × |
LcbMakeIntegerZeroLiveCodeBench / leetcode / medium: minimum-operations-to-make-the-integer-zero (id 2837)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: makeTheIntegerZero case0: expected 3 got 1 |
Failed |
IN9 182
OUT176 |
| × |
LcbMaximumSumQueriesLiveCodeBench / leetcode / hard: maximum-sum-queries (id 2839)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maximumSumQueries case0 runtime error: List index 4 is out of range. The list has 4 element(s).
Values:
run.Success = false |
Failed |
IN9 342
OUT589 |
| ✓ |
LcbRelocateMarblesLiveCodeBench / leetcode / medium: relocate-marbles (id 2834)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 268
OUT256 |
| × |
LcbSmallestStringOpLiveCodeBench / leetcode / medium: lexicographically-smallest-string-after-substring-operation (id 2828)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: smallestString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:36: Expected ')' to close the StringBuilder constructor. Use 'new StringBuilder()'., found 's'.
Values:
run.Compiled = false |
Failed |
IN9 207
OUT214 |
| ✓ |
LcbSumOfSquaresSpecialLiveCodeBench / leetcode / easy: sum-of-squares-of-special-elements (id 2844)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 156
OUT94 |
| ✓ |
LcbValueOfPartitionLiveCodeBench / leetcode / medium: find-the-value-of-the-partition (id 2845)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 176
OUT88 |
Code Generation - LiveCodeBench v1 LeetCode (set 5)
5 passed4 failed |
55.6% |
IN82 916
OUT2 732 |
| × |
LcbCountCompleteSubarraysLiveCodeBench / leetcode / medium: count-complete-subarrays-in-an-array (id 2856)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countCompleteSubarrays case0: expected 4 got 3 |
Failed |
IN9 161
OUT280 |
| × |
LcbDecrementalConcatLiveCodeBench / leetcode / medium: decremental-string-concatenation (id 2854)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimizeConcatenatedLength case0: expected 4 got 5 |
Failed |
IN9 273
OUT772 |
| ✓ |
LcbLongestEvenOddThresholdLiveCodeBench / leetcode / easy: longest-even-odd-subarray-with-threshold (id 2866)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 245
OUT270 |
| ✓ |
LcbLongestNewStringLiveCodeBench / leetcode / medium: construct-the-longest-new-string (id 2850)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 197
OUT161 |
| ✓ |
LcbMaximumJumpsLiveCodeBench / leetcode / medium: maximum-number-of-jumps-to-reach-the-last-index (id 2855)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 244
OUT211 |
| ✓ |
LcbMaxStringPairsLiveCodeBench / leetcode / easy: find-maximum-number-of-string-pairs (id 2847)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 195
OUT140 |
| × |
LcbSpecialPermutationsLiveCodeBench / leetcode / medium: special-permutations (id 2848)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: specialPerm case0: expected 2 got 7 |
Failed |
IN9 178
OUT609 |
| ✓ |
LcbSumImbalanceNumbersLiveCodeBench / leetcode / hard: sum-of-imbalance-numbers-of-all-subarrays (id 2849)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 200
OUT196 |
| × |
LcbTotalDistanceTraveledLiveCodeBench / leetcode / easy: total-distance-traveled (id 2857)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: distanceTraveled case0: expected 60 got 110 |
Failed |
IN9 223
OUT93 |
Code Generation - LiveCodeBench v1 LeetCode (set 6)
7 passed1 failed |
87.5% |
IN73 684
OUT2 246 |
| ✓ |
LcbContinuousSubarraysLiveCodeBench / leetcode / medium: continuous-subarrays (id 2868)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 186
OUT267 |
| ✓ |
LcbEmployeesMetTargetLiveCodeBench / leetcode / easy: number-of-employees-who-met-the-target (id 2876)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 200
OUT59 |
| ✓ |
LcbGoodSubarraySplitsLiveCodeBench / leetcode / medium: ways-to-split-array-into-good-subarrays (id 2867)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 181
OUT217 |
| ✓ |
LcbLongestAlternatingLiveCodeBench / leetcode / easy: longest-alternating-subarray (id 2870)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 265
OUT189 |
| ✓ |
LcbLongestNonDecreasingLiveCodeBench / leetcode / medium: longest-non-decreasing-subarray-from-two-arrays (id 2869)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 264
OUT384 |
| ✓ |
LcbMaxArrayValueMergeLiveCodeBench / leetcode / medium: largest-element-in-an-array-after-merge-operations (id 2872)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 193
OUT165 |
| × |
LcbPrimePairsSumLiveCodeBench / leetcode / medium: prime-pairs-with-target-sum (id 2873)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: findPrimePairs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:45: Type 'List<int>' has no supported method named 'GetType' with 0 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 195
OUT285 |
| ✓ |
LcbShortestThreeStringsLiveCodeBench / leetcode / medium: shortest-string-that-contains-three-strings (id 2877)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 200
OUT680 |
Code Generation - LiveCodeBench v1 LeetCode (set 7)
5 passed4 failed |
55.6% |
IN82 444
OUT2 492 |
| × |
LcbCheckArrayZeroableLiveCodeBench / leetcode / medium: apply-operations-to-make-all-array-elements-equal-to-zero (id 2878)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: CheckArrayZeroable case0: expected true got false |
Failed |
IN9 155
OUT177 |
| × |
LcbFaultyKeyboardLiveCodeBench / leetcode / easy: faulty-keyboard (id 2886)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: FaultyKeyboard case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:15: Type 'StringBuilder' has no supported method named 'Reverse' with 0 argument(s). Supported methods: Append, AppendLine, Clear, ToString.
Values:
run.Compiled = false |
Failed |
IN9 115
OUT75 |
| ✓ |
LcbLongestValidSubstringForbiddenLiveCodeBench / leetcode / hard: length-of-the-longest-valid-substring (id 2884)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 139
OUT204 |
| ✓ |
LcbMinBeautifulSubstringsLiveCodeBench / leetcode / medium: partition-string-into-minimum-beautiful-substrings (id 2883)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 149
OUT434 |
| ✓ |
LcbMinChangesKSemiPalindromesLiveCodeBench / leetcode / hard: minimum-changes-to-make-k-semi-palindromes (id 2879)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 178
OUT638 |
| × |
LcbMinIndexValidSplitLiveCodeBench / leetcode / medium: minimum-index-of-a-valid-split (id 2888)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinIndexValidSplit case0: expected 2 got -1 |
Failed |
IN9 213
OUT256 |
| ✓ |
LcbSortVowelsInStringLiveCodeBench / leetcode / medium: sort-vowels-in-a-string (id 2887)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 141
OUT291 |
| ✓ |
LcbSplitWordsBySeparatorLiveCodeBench / leetcode / easy: split-strings-by-separator (id 2881)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 179
OUT92 |
| × |
LcbWaysSumOfPowersLiveCodeBench / leetcode / medium: ways-to-express-an-integer-as-sum-of-powers (id 2882)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: WaysSumOfPowers case0: expected 1 got 4 |
Failed |
IN9 175
OUT325 |
Code Generation - LiveCodeBench v1 LeetCode (set 8)
5 passed4 failed |
55.6% |
IN82 568
OUT2 601 |
| × |
LcbCanSplitArrayMLiveCodeBench / leetcode / medium: check-if-it-is-possible-to-split-array (id 2916)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CanSplitArrayM case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 199
OUT495 |
| × |
LcbCountInterestingSubarraysLiveCodeBench / leetcode / medium: count-of-interesting-subarrays (id 2915)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountInterestingSubarrays case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
36:6: Unexpected character '—'.
36:15: Unexpected character '`'.
36:21: Unexpected character '`'.
Values:
run.Compiled = false |
Failed |
IN9 186
OUT546 |
| ✓ |
LcbCountPairsSumLessTargetLiveCodeBench / leetcode / easy: count-pairs-whose-sum-is-less-than-target (id 2917)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 146
OUT133 |
| ✓ |
LcbIsArrayGoodLiveCodeBench / leetcode / easy: check-if-array-is-good (id 2892)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 194
OUT226 |
| ✓ |
LcbMaxBeautyAfterOpLiveCodeBench / leetcode / medium: maximum-beauty-of-an-array-after-applying-operation (id 2891)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 179
OUT127 |
| × |
LcbMaxIncreasingGroupsLiveCodeBench / leetcode / hard: maximum-number-of-groups-with-increasing-length (id 2919)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxIncreasingGroups case1: expected 2 got 3 |
Failed |
IN9 173
OUT348 |
| ✓ |
LcbMaxPairSumEqualDigitLiveCodeBench / leetcode / easy: max-pair-sum-in-an-array (id 2902)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 139
OUT209 |
| × |
LcbMaxScoreVisitPositionsLiveCodeBench / leetcode / medium: visit-array-positions-to-maximize-score (id 2893)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxScoreVisitPositions case0: expected 13 got 10 |
Failed |
IN9 186
OUT248 |
| ✓ |
LcbMinSecondsCircularEqualLiveCodeBench / leetcode / medium: minimum-seconds-to-equalize-a-circular-array (id 2920)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 166
OUT269 |
Code Generation - LiveCodeBench v1 LeetCode (set 9)
3 passed5 failed |
37.5% |
IN73 483
OUT1 881 |
| × |
LcbAccountBalanceRoundedLiveCodeBench / leetcode / easy: account-balance-after-rounded-purchase (id 2955)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: AccountBalanceRounded case1: expected 80 got 90 |
Failed |
IN9 152
OUT87 |
| ✓ |
LcbCountPairsDistanceKLiveCodeBench / leetcode / medium: count-pairs-of-points-with-distance-k (id 2953)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 206
OUT156 |
| × |
LcbCountSteppingNumbersLiveCodeBench / leetcode / hard: count-stepping-numbers-in-range (id 2921)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountSteppingNumbers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'countSteppingNumbers' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, CountSteppingNumbers.
Values:
run.Compiled = false |
Failed |
IN9 158
OUT334 |
| ✓ |
LcbCountSymmetricIntegersLiveCodeBench / leetcode / easy: count-symmetric-integers (id 2998)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 151
OUT211 |
| ✓ |
LcbIsAcronymOfWordsLiveCodeBench / leetcode / easy: check-if-a-string-is-an-acronym-of-words (id 2977)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 165
OUT82 |
| × |
LcbMaximizeSalesmanProfitLiveCodeBench / leetcode / medium: maximize-the-profit-as-the-salesman (id 2979)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: MaximizeSalesmanProfit case0 runtime error: List index -1 is out of range. The list has 6 element(s).
Values:
run.Success = false |
Failed |
IN9 215
OUT206 |
| × |
LcbMaxSumAlmostUniqueLiveCodeBench / leetcode / medium: maximum-sum-of-almost-unique-subarray (id 2954)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaxSumAlmostUnique case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:37: Expected ')' to close the argument list, found 'c'.
Values:
run.Compiled = false |
Failed |
IN9 210
OUT410 |
| × |
LcbMinTimeSumAtMostXLiveCodeBench / leetcode / hard: minimum-time-to-make-array-sum-at-most-x (id 2952)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: MinTimeSumAtMostX case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 226
OUT395 |
Code Generation - LiveCodeBench v2 AtCoder (set 1)
6 passed2 failed |
75.0% |
IN73 816
OUT2 136 |
| ✓ |
LcbABCSumQueryLiveCodeBench / atcoder abc344_c / medium: A+B+C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 263
OUT480 |
| ✓ |
LcbCeilDivTenLiveCodeBench / atcoder abc345_b / easy: Integer Division Returns (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT83 |
| ✓ |
LcbDelimiterReverseLiveCodeBench / atcoder abc344_b / easy: Delimiter (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 196
OUT101 |
| × |
LcbGomamayoSequenceLiveCodeBench / atcoder abc346_d / medium: Gomamayo Sequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GomamayoSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 259
OUT357 |
| ✓ |
LcbInsertOrEraseLiveCodeBench / atcoder abc344_e / hard: Insert or Erase (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 251
OUT429 |
| ✓ |
LcbSpoilerRemoveLiveCodeBench / atcoder abc344_a / easy: Spoiler (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 151
OUT62 |
| ✓ |
LcbStringBagsLiveCodeBench / atcoder abc344_d / hard: String Bags (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 299
OUT411 |
| × |
LcbTilingGridLiveCodeBench / atcoder abc345_d / hard: Tiling (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TilingGrid case0: expected [Yes] got [No] |
Failed |
IN9 248
OUT213 |
Code Generation - LiveCodeBench v2 AtCoder (set 2)
2 passed5 failed |
28.6% |
IN64 489
OUT1 928 |
| ✓ |
LcbAdjacentProductLiveCodeBench / atcoder abc346_a / easy: Adjacent Product (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 174
OUT149 |
| ✓ |
LcbDivisibleQuotientsLiveCodeBench / atcoder abc347_a / easy: Divisible (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 184
OUT161 |
| × |
LcbIdealHolidaysLiveCodeBench / atcoder abc347_c / medium: Ideal Holidays (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: IdealHolidays case1: expected [No] got [Yes] |
Failed |
IN9 220
OUT307 |
| × |
LcbPaintGridLiveCodeBench / atcoder abc346_e / hard: Paint (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PaintGrid case0: expected [3
0 5
2 4
5 3] got [3
0 2
2 4
5 3] |
Failed |
IN9 305
OUT743 |
| × |
LcbPianoSubstringLiveCodeBench / atcoder abc346_b / easy: Piano (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PianoSubstring case0: expected [Yes] got [No] |
Failed |
IN9 190
OUT116 |
| × |
LcbSetAddQueryLiveCodeBench / atcoder abc347_e / hard: Set Add Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: SetAddQuery case0 runtime error: The dictionary does not contain the key 1. Use ContainsKey to check for it first.
Values:
run.Success = false |
Failed |
IN9 249
OUT289 |
| × |
LcbSigmaMissingSumLiveCodeBench / atcoder abc346_c / medium: Sigma (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: SigmaMissingSum case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 167
OUT163 |
Code Generation - LiveCodeBench v2 AtCoder (set 3)
5 passed3 failed |
62.5% |
IN74 431
OUT5 627 |
| × |
LcbAirportCodeLiveCodeBench / atcoder / abc349_c: Airport Code (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AirportCode case1: expected [Yes] got [No] |
Failed |
IN9 292
OUT410 |
| ✓ |
LcbColorfulBeansLiveCodeBench / atcoder / abc348_c: Colorful Beans (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 281
OUT203 |
| ✓ |
LcbDistinctSubstringsLiveCodeBench / atcoder / abc347_b: Substring (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT110 |
| ✓ |
LcbFarthestPointLiveCodeBench / atcoder / abc348_b: Farthest Point (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 333
OUT307 |
| × |
LcbMedicinesOnGridLiveCodeBench / atcoder / abc348_d: Medicines on Grid (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MedicinesOnGrid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
53:29: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed |
IN9 512
OUT3 629 |
| × |
LcbMinimizeSumOfDistancesLiveCodeBench / atcoder / abc348_e: Minimize Sum of Distances (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimizeSumOfDistances case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
27:30: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed |
IN9 323
OUT799 |
| ✓ |
LcbPenaltyKickLiveCodeBench / atcoder / abc348_a: Penalty Kick (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 226
OUT86 |
| ✓ |
LcbZeroSumGameLiveCodeBench / atcoder / abc349_a: Zero Sum Game (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 272
OUT83 |
Code Generation - LiveCodeBench v2 AtCoder (set 4)
2 passed3 failed2 skipped |
40.0% |
IN46 659
OUT3 568 |
| ✓ |
LcbCommencementLiveCodeBench / atcoder / abc349_b: Commencement (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 215
OUT252 |
| × |
LcbDentistAokiLiveCodeBench / atcoder / abc350_b: Dentist Aoki (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DentistAoki case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 279
OUT186 |
| × |
LcbDivideIntervalLiveCodeBench / atcoder / abc349_d: Divide Interval (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DivideInterval case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:27: Operator '+' cannot be applied to 'long' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 413
OUT204 |
| ✓ |
LcbPastABCsLiveCodeBench / atcoder / abc350_a: Past ABCs (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 297
OUT92 |
| – |
LcbSortPermutationLiveCodeBench / atcoder / abc350_c: Sort (Pattern B, stdin) TRIAGE: 0 public tests in dump + non-unique output (any valid swap sequence accepted) — cannot oracle-compare.0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: no public tests in dump; output non-unique (any valid swap sequence is accepted) — exact-compare oracle inapplicable, coordinator triage |
Skipped |
IN0
OUT0 |
| – |
LcbTowardZeroLiveCodeBench / atcoder / abc350_e: Toward 0 (Pattern B, stdin) TRIAGE: float output (absolute/relative error <= 1e-6) — RunLcbStdin does exact string compare; needs tolerance.0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: float output requires error tolerance (<=1e-6); RunLcbStdin does exact string compare — coordinator triage |
Skipped |
IN0
OUT0 |
| × |
LcbWeightedTicLiveCodeBench / atcoder / abc349_e: Weighted Tic (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WeightedTic case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
81:17: Integer literal '9223372036854775808' is out of range for type long.
Values:
run.Compiled = false |
Failed |
IN9 455
OUT2 834 |
Code Generation - LiveCodeBench v2 AtCoder (set 5)
3 passed5 failed |
37.5% |
IN97 960
OUT8 543 |
| ✓ |
LcbBottomNinthLiveCodeBench / atcoder / abc351_a: The bottom of the ninth (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 330
OUT135 |
| ✓ |
LcbCliqueConnectLiveCodeBench / atcoder / abc352_e: Clique Connect (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN20 383
OUT4 479 |
| × |
LcbDoubleSumMaxDiffLiveCodeBench / atcoder / abc351_f: Double Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DoubleSumMaxDiff case0: expected [4] got [6] |
Failed |
IN9 193
OUT147 |
| × |
LcbGridAndMagnetLiveCodeBench / atcoder / abc351_d: Grid and Magnet (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GridAndMagnet case0: expected [9] got [8] |
Failed |
IN9 296
OUT855 |
| ✓ |
LcbJumpDistanceSumLiveCodeBench / atcoder / abc351_e: Jump Distance Sum (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN21 955
OUT2 187 |
| × |
LcbMergeTheBallsLiveCodeBench / atcoder / abc351_c: Merge the balls (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MergeTheBalls case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:15: Variable 'parts' has type 'List<int>' but its initializer has type 'List<string>'.
Values:
run.Compiled = false |
Failed |
IN9 275
OUT177 |
| × |
LcbNewFriendsLiveCodeBench / atcoder / abc350_d: New Friends (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NewFriends case2: expected [12] got [4] |
Failed |
IN9 224
OUT387 |
| × |
LcbSpotTheDifferenceLiveCodeBench / atcoder / abc351_b: Spot the Difference (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SpotTheDifference case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:14: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 304
OUT176 |
Code Generation - LiveCodeBench v2 AtCoder (set 6)
6 passed1 failed |
85.7% |
IN75 865
OUT3 258 |
| ✓ |
LcbAmusementParkLiveCodeBench / atcoder / abc353_b: AtCoder Amusement Park (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 286
OUT172 |
| ✓ |
LcbAtcoderLineLiveCodeBench / atcoder / abc352_a: AtCoder Line (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 239
OUT129 |
| ✓ |
LcbBuildingsTallerLiveCodeBench / atcoder / abc353_a: Buildings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 213
OUT123 |
| × |
LcbPermutationSubsequenceLiveCodeBench / atcoder / abc352_d: Permutation Subsequence (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PermutationSubsequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:1: Unknown type 'Fenwick'. Supported types: bool, byte, int, decimal, double, char, string, Date, DateTime, TimeSpan, List<...>, Dictionary<...>, HashSet<...> and declared structs.
17:17: Unknown type 'Fenwick'. Supported types: bool, byte, int, decimal, double, char, string, Date, DateTime, TimeSpan, List<...>, Dictionary<...>, HashSet<...> and declared structs.
22:20: Unknown type 'Fenwick'. Supported types: bool, byte, int, decimal, double, char, string, Date, DateTime, TimeSpan, List<...>, Dictionary<...>, HashSet<...> and declared structs.
29:24: Unknown type 'Fenwick'. Supported types: bool, byte, int, decimal, double, char, string, Date, DateTime, TimeSpan, List<...>, Dictionary<...>, HashSet<...> and declared structs.
14:12: Unknown struct 'Fenwick'.
Values:
run.Compiled = false |
Failed |
IN20 325
OUT2 238 |
| ✓ |
LcbSigmaRemainderSumLiveCodeBench / atcoder / abc353_c: Sigma Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 219
OUT239 |
| ✓ |
LcbStandingOnShouldersLiveCodeBench / atcoder / abc352_c: Standing On The Shoulders (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 312
OUT208 |
| ✓ |
LcbTypingBackspaceLiveCodeBench / atcoder / abc352_b: Typing (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 271
OUT149 |
Code Generation - LiveCodeBench v2 AtCoder (set 7)
6 failed |
0.0% |
IN55 858
OUT2 065 |
| × |
LcbAnotherSigmaProblemLiveCodeBench / atcoder / abc353_d: Another Sigma Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AnotherSigmaProblem case0: expected [2044] got [3790] |
Failed |
IN9 320
OUT473 |
| × |
LcbAtcoderJanken2LiveCodeBench / atcoder / abc354_b: AtCoder Janken 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AtcoderJanken2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
31:33: Expected '{' to start a block, found '('.
Values:
run.Compiled = false |
Failed |
IN9 304
OUT318 |
| × |
LcbAtcoderMagicsLiveCodeBench / atcoder / abc354_c: AtCoder Magics (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AtcoderMagics case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:26: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 373
OUT515 |
| × |
LcbExponentialPlantLiveCodeBench / atcoder / abc354_a: Exponential Plant (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ExponentialPlant case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:25: The shift count of '<<' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed |
IN9 257
OUT80 |
| × |
LcbRemovePairsLiveCodeBench / atcoder / abc354_e: Remove Pairs (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RemovePairs case0: expected [Aoki] got [Takahashi] |
Failed |
IN9 350
OUT496 |
| × |
LcbYetAnotherSigmaProblemLiveCodeBench / atcoder / abc353_e: Yet Another Sigma Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: YetAnotherSigmaProblem case0: expected [4] got [5] |
Failed |
IN9 254
OUT183 |
Code Generation - LiveCodeBench v2 AtCoder (set 8)
2 passed3 failed1 skipped |
40.0% |
IN46 533
OUT1 875 |
| × |
LcbAtcoderWallpaperLiveCodeBench / atcoder / abc354_d: AtCoder Wallpaper (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AtcoderWallpaper case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:50: Char literals must contain exactly one character and close with '''.
10:140: Char literals must contain exactly one character and close with '''.
12:46: Unexpected character '∈'.
12:53: Unexpected character '∈'.
12:63: Unexpected character '∈'.
12:129: Unexpected character '→'.
12:132: Unexpected character '≡'.
12:144: Unexpected character '≡'.
16:338: Unexpected character '→'.
16:400: Unexpected character '→'.
16:432: Unexpected character '—'.
16:573: Unexpected character '—'.
18:70: Char literals must contain exactly one character and close with '''.
20:524: Char literals must contain exactly one character and close with '''.
22:222: Unexpected character '—'.
24:98: Char literals must contain exactly one character and close with '''.
26:32: Char literals must contain exactly one character and close with '''.
28:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 365
OUT981 |
| ✓ |
LcbBingo2LiveCodeBench / atcoder / abc355_c: Bingo 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 378
OUT333 |
| – |
LcbGuessTheSumLiveCodeBench / atcoder / abc355_e: Guess the Sum (Pattern B, stdin) — INTERACTIVE, cannot run in the batch harness0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: SC# subset: interactive problem (adaptive query/response dialogue; not expressible as a single stdin->stdout Solve) |
Skipped |
IN0
OUT0 |
| × |
LcbIntersectingIntervalsLiveCodeBench / atcoder / abc355_d: Intersecting Intervals (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: IntersectingIntervals case0: expected [2] got [7] |
Failed |
IN9 235
OUT203 |
| ✓ |
LcbPiano2LiveCodeBench / atcoder / abc355_b: Piano 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 303
OUT259 |
| × |
LcbWhoAteTheCakeLiveCodeBench / atcoder / abc355_a: Who Ate the Cake? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WhoAteTheCake case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:15: Type 'List<int>' has no supported method named 'Remove' with 1 argument(s).
8:15: Type 'List<int>' has no supported method named 'Remove' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 252
OUT99 |
Code Generation - LiveCodeBench v2 LeetCode (set 1)
6 passed3 failed |
66.7% |
IN82 585
OUT2 601 |
| ✓ |
LcbCountSubstringsStartEndCharLiveCodeBench / leetcode / medium: count-substrings-starting-and-ending-with-given-character (LCB 3337)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 105
OUT117 |
| ✓ |
LcbMaximumHappinessSumLiveCodeBench / leetcode / medium: maximize-happiness-of-selected-children (LCB 3351)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 200
OUT109 |
| × |
LcbMaximumStrengthSubarraysLiveCodeBench / leetcode / hard: maximum-strength-of-k-disjoint-subarrays (LCB 3313)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maximumStrength case0: expected 22 got 15 |
Failed |
IN9 255
OUT421 |
| ✓ |
LcbMaxLengthTwoOccurrencesLiveCodeBench / leetcode / easy: maximum-length-substring-with-two-occurrences (LCB 3349)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 087
OUT243 |
| × |
LcbMinDeletionsKSpecialLiveCodeBench / leetcode / medium: minimum-deletions-to-make-string-k-special (LCB 3360)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minimumDeletions case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 160
OUT215 |
| ✓ |
LcbMinimumBoxesApplesLiveCodeBench / leetcode / easy: apple-redistribution-into-boxes (LCB 3334)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 190
OUT114 |
| × |
LcbMinMovesPickKOnesLiveCodeBench / leetcode / hard: minimum-moves-to-pick-k-ones (LCB 3327)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumMoves case0: expected 3 got 4 |
Failed |
IN9 314
OUT621 |
| ✓ |
LcbShortestUncommonSubstringsLiveCodeBench / leetcode / medium: shortest-uncommon-substring-in-an-array (LCB 3356)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 168
OUT631 |
| ✓ |
LcbSubstringInReverseLiveCodeBench / leetcode / easy: existence-of-a-substring-in-a-string-and-its-reverse (LCB 3353)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 106
OUT130 |
Code Generation - LiveCodeBench v2 LeetCode (set 2)
7 passed2 failed |
77.8% |
IN82 482
OUT2 151 |
| ✓ |
LcbCountAlternatingSubarraysLiveCodeBench / leetcode / medium: count-alternating-subarrays (LCB 3374)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 118
OUT137 |
| ✓ |
LcbHarshadDigitSumLiveCodeBench / leetcode / easy: harshad-number (LCB 3371)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 117
OUT108 |
| ✓ |
LcbLexSmallestWithinDistanceLiveCodeBench / leetcode / medium: lexicographically-smallest-string-after-operations-with-constraint (LCB 3346)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 213
OUT301 |
| ✓ |
LcbLongestCommonSuffixQueriesLiveCodeBench / leetcode / hard: longest-common-suffix-queries (LCB 3376)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 212
OUT258 |
| × |
LcbLongestMonotonicSubarrayLiveCodeBench / leetcode / easy: longest-strictly-increasing-or-decreasing-subarray (LCB 3372)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestMonotonicSubarray case0: expected 2 got 1 |
Failed |
IN9 126
OUT230 |
| ✓ |
LcbMinimizeManhattanDistanceLiveCodeBench / leetcode / hard: minimize-manhattan-distances (LCB 3344)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 149
OUT413 |
| × |
LcbMinOpsMedianKLiveCodeBench / leetcode / medium: minimum-operations-to-make-median-of-array-equal-to-k (LCB 3387)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minOperationsToMakeMedianK case0: expected 2 got 8 |
Failed |
IN9 189
OUT301 |
| ✓ |
LcbMinOpsSumAtLeastKLiveCodeBench / leetcode / medium: apply-operations-to-make-sum-of-array-ge-k (LCB 3328)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 144
OUT160 |
| ✓ |
LcbMostFrequentIdsLiveCodeBench / leetcode / medium: most-frequent-ids (LCB 3363)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 214
OUT243 |
Code Generation - LiveCodeBench v2 LeetCode (set 3)
3 passed6 failed |
33.3% |
IN82 832
OUT2 602 |
| × |
LcbKthSmallestAmountLiveCodeBench 3375 / leetcode / hard: kth-smallest-amount-with-single-denomination-combination (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: kthSmallestAmount case0: expected 9 got 6 |
Failed |
IN9 189
OUT167 |
| × |
LcbLatestTimeReplaceLiveCodeBench 3361 / leetcode / easy: latest-time-you-can-obtain-after-replacing-characters (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: latestTimeReplace case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
46:20: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed |
IN9 213
OUT297 |
| ✓ |
LcbMaximumPrimeDifferenceLiveCodeBench 3373 / leetcode / medium: maximum-prime-difference (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 158
OUT285 |
| × |
LcbMinimizeStringValueLiveCodeBench 3354 / leetcode / medium: replace-question-marks-in-string-to-minimize-its-value (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: minimizeStringValue case0 runtime error: List index 2 is out of range. The list has 2 element(s).
Values:
run.Success = false |
Failed |
IN9 225
OUT573 |
| × |
LcbMinSumValuesDividingLiveCodeBench 3364 / leetcode / hard: minimum-sum-of-values-by-dividing-array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minSumValuesDividing case0: expected 12 got -1 |
Failed |
IN9 277
OUT350 |
| × |
LcbShortestSubarrayOrKLiveCodeBench 3381 / leetcode / easy: shortest-subarray-with-or-at-least-k-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: shortestSubarrayOrK case2 runtime error: List index 2 is out of range. The list has 2 element(s).
Values:
run.Success = false |
Failed |
IN9 181
OUT159 |
| ✓ |
LcbSumOfEncryptedIntsLiveCodeBench 3367 / leetcode / easy: find-the-sum-of-encrypted-integers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 159
OUT182 |
| × |
LcbSumOfPowerSubsequencesLiveCodeBench 3345 / leetcode / hard: find-the-sum-of-the-power-of-all-subsequences (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: sumOfPowerSubsequences case0: expected 6 got 10 |
Failed |
IN9 174
OUT222 |
| ✓ |
LcbUnmarkedSumArrayLiveCodeBench 3306 / leetcode / medium: mark-elements-on-array-by-performing-queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 256
OUT367 |
Code Generation - LiveCodeBench v2 LeetCode (set 4)
3 passed6 failed |
33.3% |
IN82 587
OUT2 041 |
| ✓ |
LcbAddedIntegerLiveCodeBench 3397 / leetcode / easy: find-the-integer-added-to-array-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 202
OUT101 |
| × |
LcbMedianUniquenessArrayLiveCodeBench 3362 / leetcode / hard: find-the-median-of-the-uniqueness-array (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: medianUniquenessArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
36:40: Expected ')' to close the argument list, found 'fR'.
Values:
run.Compiled = false |
Failed |
IN9 222
OUT416 |
| × |
LcbMinimumArrayEndLiveCodeBench 3394 / leetcode / medium: minimum-array-end (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minimumArrayEnd case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:22: Unknown function '__builtin_ctzll'.
Values:
run.Compiled = false |
Failed |
IN9 166
OUT67 |
| × |
LcbMinimumLevelsGainPointsLiveCodeBench 3355 / leetcode / medium: minimum-levels-to-gain-more-points (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimumLevelsGainPoints case0: expected 1 got 2 |
Failed |
IN9 225
OUT336 |
| ✓ |
LcbNumberOfSpecialCharsLiveCodeBench 3408 / leetcode / easy: count-the-number-of-special-characters-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 121
OUT151 |
| ✓ |
LcbScoreOfStringLiveCodeBench 3379 / leetcode / easy: score-of-a-string (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 172
OUT86 |
| × |
LcbSubarraysBoundaryMaxLiveCodeBench 3382 / leetcode / hard: find-the-number-of-subarrays-where-boundary-elements-are-maximum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: subarraysBoundaryMax case0: expected 6 got 11 |
Failed |
IN9 127
OUT119 |
| × |
LcbSumOfSubsequencePowersLiveCodeBench 3316 / leetcode / hard: find-the-sum-of-subsequence-powers (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: sumOfSubsequencePowers case0: expected 4 got 256 |
Failed |
IN9 182
OUT506 |
| × |
LcbValidWordLiveCodeBench 3396 / leetcode / easy: valid-word (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: validWord case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'isValid' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, IsValid.
Values:
run.Compiled = false |
Failed |
IN9 170
OUT259 |
Code Generation - LiveCodeBench v2 LeetCode (set 5)
7 passed2 failed |
77.8% |
IN82 723
OUT2 761 |
| ✓ |
LcbCanMakeSquareColorLiveCodeBench / leetcode / easy: make-a-square-with-the-same-color1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 217
OUT173 |
| ✓ |
LcbIsArraySpecialILiveCodeBench / leetcode / easy: special-array-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 127
OUT79 |
| ✓ |
LcbMakeWordKPeriodicLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-word-k-periodic1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 197
OUT191 |
| ✓ |
LcbMinAnagramLengthLiveCodeBench / leetcode / medium: minimum-length-of-anagram-concatenation1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 131
OUT207 |
| × |
LcbMinCostEqualizeArrayLiveCodeBench / leetcode / hard: minimum-cost-to-equalize-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinCostEqualizeArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:101: Unexpected character '≥'.
9:95: Char literals must contain exactly one character and close with '''.
18:20: Unexpected character '→'.
20:100: Char literals must contain exactly one character and close with '''.
22:21: Unexpected character '≥'.
22:260: Unexpected character '—'.
22:337: Unexpected character '→'.
22:376: Unexpected character '⇒'.
22:389: Unexpected character '⇒'.
26:242: Unexpected character '—'.
26:270: Char literals must contain exactly one character and close with '''.
28:178: Unexpected character '→'.
28:182: Unexpected character '✓'.
30:88: Unexpected character '→'.
30:112: Unexpected character '✓'.
Values:
run.Compiled = false |
Failed |
IN9 261
OUT1 112 |
| ✓ |
LcbMysticDungeonEnergyLiveCodeBench / leetcode / medium: taking-maximum-energy-from-the-mystic-dungeon1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 188
OUT119 |
| ✓ |
LcbNumberOfRightTrianglesLiveCodeBench / leetcode / medium: right-triangles1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 223
OUT304 |
| × |
LcbNumberOfStableArraysLiveCodeBench / leetcode / medium: find-all-possible-stable-binary-arrays-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: NumberOfStableArrays case1: expected 1 got 2 |
Failed |
IN9 217
OUT462 |
| ✓ |
LcbPermutationDifferenceLiveCodeBench / leetcode / easy: permutation-difference-between-two-strings1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 162
OUT114 |
Code Generation - LiveCodeBench v2 LeetCode (set 6)
5 passed4 failed |
55.6% |
IN82 717
OUT3 655 |
| ✓ |
LcbMaxSumNonAdjacentQueriesLiveCodeBench / leetcode / hard: maximum-sum-of-subsequence-with-non-adjacent-elements1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 228
OUT483 |
| × |
LcbMinBalancedPartitionLiveCodeBench / leetcode / medium: minimum-substring-partition-of-equal-character-frequency1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinBalancedPartition case0: expected 3 got 1 |
Failed |
IN9 125
OUT613 |
| ✓ |
LcbNumberOfGoodPairsILiveCodeBench / leetcode / easy: find-the-number-of-good-pairs-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 207
OUT103 |
| × |
LcbOccurrencesOfElementLiveCodeBench / leetcode / medium: find-occurrences-of-an-element-in-an-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: OccurrencesOfElement case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:16: Expected a loop variable name, found 'in'.
Values:
run.Compiled = false |
Failed |
IN9 205
OUT166 |
| × |
LcbProductsBigArrayLiveCodeBench / leetcode / hard: find-products-of-elements-of-big-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ProductsBigArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
113:31: Expected ')' to close the argument list, found 'long'.
Values:
run.Compiled = false |
Failed |
IN9 310
OUT1 228 |
| ✓ |
LcbStringCompressionIIILiveCodeBench / leetcode / medium: string-compression-iii1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 149
OUT121 |
| ✓ |
LcbSumDigitDifferencesLiveCodeBench / leetcode / medium: sum-of-digit-differences-of-all-pairs1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 147
OUT639 |
| × |
LcbWaysToReachStairLiveCodeBench / leetcode / hard: find-number-of-ways-to-reach-the-k-th-stair1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: WaysToReachStair case0 runtime error: Integer overflow.
Values:
run.Success = false |
Failed |
IN9 204
OUT164 |
| ✓ |
LcbXorAppearingTwiceLiveCodeBench / leetcode / easy: find-the-xor-of-numbers-which-appear-twice1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 142
OUT138 |
Code Generation - LiveCodeBench v3 AtCoder (set 1)
4 passed3 failed |
57.1% |
IN87 045
OUT2 133 |
| × |
LcbKeysCombinationsLiveCodeBench / atcoder / medium: abc356_c Keys (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: KeysCombinations case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:26: Expected ';' after the for initializer, found ':'.
Values:
run.Compiled = false |
Failed |
IN9 392
OUT407 |
| × |
LcbMaskedPopcountLiveCodeBench / atcoder / medium: abc356_d Masked Popcount (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaskedPopcount case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:27: The shift count of '<<' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed |
IN31 320
OUT340 |
| × |
LcbMaxMinFloorSumLiveCodeBench / atcoder / hard: abc356_e Max/Min (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaxMinFloorSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 216
OUT585 |
| ✓ |
LcbNutrientsLiveCodeBench / atcoder / easy: abc356_b Nutrients (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 293
OUT227 |
| ✓ |
LcbSanitizeHandsLiveCodeBench / atcoder / easy: abc357_a Sanitize Hands (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 268
OUT144 |
| ✓ |
LcbSierpinskiCarpetLiveCodeBench / atcoder / medium: abc357_c Sierpinski carpet (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 321
OUT246 |
| ✓ |
LcbSubsegmentReverseLiveCodeBench / atcoder / easy: abc356_a Subsegment Reverse (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 235
OUT184 |
Code Generation - LiveCodeBench v3 AtCoder (set 2)
3 passed4 failed |
42.9% |
IN77 672
OUT4 452 |
| × |
LcbAlphabetTilesLiveCodeBench / atcoder / hard: abc358_e Alphabet Tiles (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AlphabetTiles case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
26:30: Math has no supported method named 'min'. Supported methods: Abs, Acos, Asin, Atan, Atan2, Ceiling, Cos, Exp, Floor, Log, Log10, Log2, Max, Min, Pow, Round, Sin, Sqrt, Tan, ToDegrees, ToRadians.
Values:
run.Compiled = false |
Failed |
IN9 289
OUT632 |
| × |
LcbConcatModuloLiveCodeBench / atcoder / medium: abc357_d 88888888 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ConcatModulo case0: expected [55555] got [236507738] |
Failed |
IN9 280
OUT306 |
| × |
LcbFunctionalGraphReachLiveCodeBench / atcoder / hard: abc357_e Reachability in Functional Graph (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: FunctionalGraphReach case0 runtime error: A list of length 1000001 exceeded the list length limit of 1000000.
Values:
run.Success = false |
Failed |
IN21 922
OUT2 539 |
| ✓ |
LcbPopcornStandsLiveCodeBench / atcoder / medium: abc358_c Popcorn (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 292
OUT309 |
| × |
LcbSouvenirsLiveCodeBench / atcoder / medium: abc358_d Souvenirs (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Souvenirs case0: expected [7] got [9] |
Failed |
IN9 335
OUT211 |
| ✓ |
LcbTicketCounterLiveCodeBench / atcoder / easy: abc358_b Ticket Counter (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 333
OUT178 |
| ✓ |
LcbUppercaseLowercaseLiveCodeBench / atcoder / easy: abc357_b Uppercase and Lowercase (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 221
OUT277 |
Code Generation - LiveCodeBench v3 AtCoder (set 3)
3 passed4 failed |
42.9% |
IN64 635
OUT5 229 |
| × |
LcbAvoidKPalindromeLiveCodeBench / atcoder abc359_d / hard: Avoid K Palindrome (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AvoidKPalindrome case0: expected [1] got [4] |
Failed |
IN9 223
OUT706 |
| ✓ |
LcbCountTakahashiLiveCodeBench / atcoder abc359_a / easy: Count Takahashi (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 179
OUT76 |
| ✓ |
LcbCouplesBetweenLiveCodeBench / atcoder abc359_b / easy: Couples (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT158 |
| × |
LcbMoveItBoxesLiveCodeBench / atcoder abc360_c / medium: Move It (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: MoveItBoxes case0 runtime error: List index 5 is out of range. The list has 5 element(s).
Values:
run.Success = false |
Failed |
IN9 232
OUT317 |
| × |
LcbTileDistance2TollLiveCodeBench / atcoder abc359_c / medium: Tile Distance 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TileDistance2Toll case0: expected [5] got [4] |
Failed |
IN9 322
OUT3 637 |
| × |
LcbWaterTankFillLiveCodeBench / atcoder abc359_e / hard: Water Tank (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WaterTankFill case0: expected [4 5 13 14 26] got [4 5 9 10 15] |
Failed |
IN9 319
OUT265 |
| ✓ |
LcbWelcomeAtCoderLandLiveCodeBench / atcoder abc358_a / easy: Welcome to AtCoder Land (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 168
OUT70 |
Code Generation - LiveCodeBench v3 AtCoder (set 4)
4 passed3 failed |
57.1% |
IN75 698
OUT4 723 |
| × |
LcbGhostAntsPassLiveCodeBench / atcoder abc360_d / medium: Ghost Ants (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GhostAntsPass case0: expected [5] got [12] |
Failed |
IN9 302
OUT576 |
| × |
LcbGoStonePuzzleLiveCodeBench / atcoder abc361_d / hard: Go Stone Puzzle (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: GoStonePuzzle case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:5: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN20 273
OUT1 132 |
| ✓ |
LcbHealthyBreakfastLiveCodeBench / atcoder abc360_a / easy: A Healthy Breakfast (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 228
OUT61 |
| ✓ |
LcbInsertAfterKLiveCodeBench / atcoder abc361_a / easy: Insert (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 177
OUT158 |
| ✓ |
LcbIntersectCuboidsLiveCodeBench / atcoder abc361_b / medium: Intersection of Cuboids (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 209
OUT483 |
| × |
LcbRandomSwapsBallsLiveCodeBench / atcoder abc360_e / hard: Random Swaps of Balls (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RandomSwapsBalls case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:148: Unexpected character '∈'.
4:161: Unexpected character '≠'.
10:268: Unexpected character '—'.
10:380: Unexpected character '≠'.
10:391: Char literals must contain exactly one character and close with '''.
12:45: Unexpected character '≥'.
17:39: Unexpected character '∈'.
20:7: Unexpected character '∉'.
24:43: Unexpected character '≠'.
24:121: Unexpected character '→'.
24:131: Unexpected character '²'.
25:104: Unexpected character '≠'.
25:112: Unexpected character '∉'.
25:188: Char literals must contain exactly one character and close with '''.
27:32: Unexpected character '²'.
27:41: Unexpected character '²'.
27:57: Unexpected character '≠'.
27:64: Unexpected character '²'.
27:70: Unexpected character '²'.
27:88: Unexpected character '²'.
27:94: Unexpected character '²'.
27:108: Unexpected character '²'.
27:113: Unexpected character '²'.
27:116: Unexpected character '²'.
27:118: Unexpected character '✓'.
29:59: Unexpected character 'Σ'.
29:63: Unexpected character '≥'.
29:81: Unexpected character '→'.
30:13: Unexpected character '²'.
30:22: Unexpected character '²'.
30:28: Unexpected character '→'.
30:35: Unexpected character '²'.
31:1: Unexpected character 'Σ'.
31:5: Unexpected character '≥'.
35:9: Unexpected character '²'.
35:20: Unexpected character '²'.
35:26: Unexpected character '²'.
35:33: Unexpected character '²'.
35:45: Unexpected character '²'.
36:8: Unexpected character '²'.
37:33: Unexpected character '²'.
39:32: Unexpected character '⇒'.
39:44: Unexpected character '⇒'.
39:64: Unexpected character '²'.
46:8: Unexpected character 'Σ'.
46:38: Unexpected character 'Σ'.
47:1: Unexpected character 'Σ'.
50:65: Unexpected character '²'.
65:139: Unexpected character '✓'.
65:290: Unexpected character '≤'.
Values:
run.Compiled = false |
Failed |
IN9 282
OUT2 112 |
| ✓ |
LcbVerticalReadingPairLiveCodeBench / atcoder abc360_b / easy: Vertical Reading (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 227
OUT201 |
Code Generation - LiveCodeBench v3 AtCoder (set 5)
2 passed4 failed1 skipped |
33.3% |
IN77 549
OUT2 574 |
| ✓ |
LcbBuyAPenLiveCodeBench / atcoder / easy: abc362_a Buy a Pen (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 205
OUT131 |
| × |
LcbCountArithmeticSubsequencesLiveCodeBench / atcoder / hard: abc362_e Count Arithmetic Subsequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountArithmeticSubsequences case0: expected [5 10 3 0 0] got [5 1 1 1 0] |
Failed |
IN9 263
OUT543 |
| ✓ |
LcbMakeThemNarrowLiveCodeBench / atcoder / medium: abc361_c Make Them Narrow (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN31 394
OUT340 |
| × |
LcbPerfectPowerCountLiveCodeBench / atcoder / hard: abc361_f x = a^b (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PerfectPowerCount case0: expected [12] got [11] |
Failed |
IN9 158
OUT133 |
| × |
LcbShortestPath3LiveCodeBench / atcoder / hard: abc362_d Shortest Path 3 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ShortestPath3 case0: expected [4 9] got [4 10] |
Failed |
IN9 282
OUT696 |
| – |
LcbSumZeroSequenceLiveCodeBench / atcoder / medium: abc362_c Sum = 0 (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbTreeHamiltonPathLiveCodeBench / atcoder / hard: abc361_e Tree and Hamilton Path 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TreeHamiltonPath case0: expected [11] got [7] |
Failed |
IN9 247
OUT731 |
Code Generation - LiveCodeBench v3 AtCoder (set 6)
3 passed3 failed1 skipped |
50.0% |
IN55 524
OUT2 186 |
| × |
LcbAvoidKPalindrome2LiveCodeBench / atcoder / medium: abc363_c Avoid K Palindrome 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AvoidKPalindrome2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 271
OUT119 |
| ✓ |
LcbCursedDollLiveCodeBench / atcoder / easy: abc363_b Japanese Cursed Doll (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 241
OUT370 |
| × |
LcbNthPalindromeLiveCodeBench / atcoder / medium: abc363_d Palindromic Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NthPalindrome case0: expected [363] got [3663] |
Failed |
IN9 181
OUT299 |
| – |
LcbPalindromicExpressionLiveCodeBench / atcoder / hard: abc363_f Palindromic Expression (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbPilingUpLiveCodeBench / atcoder / easy: abc363_a Piling Up (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 261
OUT56 |
| ✓ |
LcbRightTriangleCheckLiveCodeBench / atcoder / easy: abc362_b Right Triangle (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 211
OUT293 |
| × |
LcbSinkingLandLiveCodeBench / atcoder / hard: abc363_e Sinking Land (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SinkingLand case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
17:14: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 359
OUT1 049 |
Code Generation - LiveCodeBench v3 AtCoder (set 7)
3 passed2 failed1 broken |
50.0% |
IN78 212
OUT66 265 |
| × |
LcbGluttonTakahashiLiveCodeBench / atcoder / easy: Glutton Takahashi (abc364_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GluttonTakahashi case1: expected [Yes] got [No] |
Failed |
IN9 193
OUT98 |
| ✓ |
LcbGridWalkLiveCodeBench / atcoder / easy: Grid Walk (abc364_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 373
OUT399 |
| ✓ |
LcbKthClosestPointLiveCodeBench / atcoder / hard: K — k-th closest point (abc364_d)1 AI query
2 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN31 949
OUT1 060 |
| ! |
LcbMaximumGluttonLiveCodeBench / atcoder / hard: Maximum Glutton (abc364_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: AI output broken/partial |
Broken |
IN9 236
OUT62 144 |
| ✓ |
LcbMinimumGluttonLiveCodeBench / atcoder / medium: Minimum Glutton (abc364_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 200
OUT1 948 |
| × |
LcbRangeConnectMstLiveCodeBench / atcoder / hard: Range Connect MST (abc364_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RangeConnectMst case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 261
OUT616 |
Code Generation - LiveCodeBench v3 AtCoder (set 8)
1 passed4 failed |
20.0% |
IN57 013
OUT1 574 |
| × |
LcbAtcoderJanken3LiveCodeBench / atcoder / medium: AtCoder Janken 3 (abc365_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AtcoderJanken3 case0: expected [5] got [2] |
Failed |
IN9 269
OUT449 |
| ✓ |
LcbLeapYearLiveCodeBench / atcoder / easy: Leap Year (abc365_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 243
OUT87 |
| × |
LcbSecondBestLiveCodeBench / atcoder / easy: Second Best (abc365_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SecondBest case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:16: Variable 'parts' has type 'List<long>' but its initializer has type 'List<string>'.
8:26: 'long.Parse' expects a string argument, but it has type 'long'.
Values:
run.Compiled = false |
Failed |
IN9 175
OUT187 |
| × |
LcbTransportationExpensesLiveCodeBench / atcoder / medium: Transportation Expenses (abc365_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TransportationExpenses case0: expected [2] got [-1] |
Failed |
IN9 235
OUT252 |
| × |
LcbXorSigmaProblemLiveCodeBench / atcoder / hard: Xor Sigma Problem (abc365_e)1 AI query
1 tool callcategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: XorSigmaProblem case0 runtime error: Cannot parse '1 3 2' as long.
Values:
run.Success = false |
Failed |
IN20 091
OUT599 |
Code Generation - LiveCodeBench v3 LeetCode (set 1)
5 passed3 failed |
62.5% |
IN73 393
OUT1 325 |
| ✓ |
LcbChildWithBallLiveCodeBench / leetcode / easy: find-the-child-who-has-the-ball-after-k-seconds (id 3450)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 181
OUT60 |
| ✓ |
LcbCompleteDayPairsLiveCodeBench / leetcode / easy: count-pairs-that-form-a-complete-day-i (id 3421)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 140
OUT178 |
| ✓ |
LcbCountDaysWithoutMeetingsLiveCodeBench / leetcode / medium: count-days-without-meetings (id 3430)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 198
OUT189 |
| × |
LcbMaxTotalDamageLiveCodeBench / leetcode / medium: maximum-total-damage-with-spell-casting (id 3437)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxTotalDamage case0: expected 6 got 4 |
Failed |
IN9 160
OUT335 |
| × |
LcbMaxTotalRewardLiveCodeBench / leetcode / medium: maximum-total-reward-using-operations-i (id 3442)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxTotalReward case1: expected 11 got 7 |
Failed |
IN9 174
OUT83 |
| ✓ |
LcbMinimumChairsLiveCodeBench / leetcode / easy: minimum-number-of-chairs-in-a-waiting-room (id 3426)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT104 |
| ✓ |
LcbSubarrayOrClosestKLiveCodeBench / leetcode / hard: find-subarray-with-bitwise-or-closest-to-k (id 3436)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 182
OUT135 |
| × |
LcbValueAfterKSecondsLiveCodeBench / leetcode / medium: find-the-n-th-value-after-k-seconds (id 3422)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: ValueAfterKSeconds case0: expected 56 got 4 |
Failed |
IN9 195
OUT241 |
Code Generation - LiveCodeBench v3 LeetCode (set 2)
3 passed5 failed |
37.5% |
IN73 563
OUT2 082 |
| ✓ |
LcbEncryptedStringLiveCodeBench / leetcode / easy: find-the-encrypted-string (id 3468)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 127
OUT109 |
| × |
LcbMaxTotalCostAlternatingLiveCodeBench / leetcode / medium: maximize-total-cost-of-alternating-subarrays (id 3464)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxTotalCostAlternating case0: expected 10 got 7 |
Failed |
IN9 218
OUT165 |
| × |
LcbMaxTriangleHeightLiveCodeBench / leetcode / easy: maximum-height-of-a-triangle (id 3469)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxTriangleHeight case1: expected 2 got 1 |
Failed |
IN9 184
OUT154 |
| ✓ |
LcbMaxValidSubsequenceLiveCodeBench / leetcode / medium: find-the-maximum-length-of-valid-subsequence-i (id 3490)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 185
OUT186 |
| × |
LcbMinDiameterMergeTreesLiveCodeBench / leetcode / hard: find-minimum-diameter-after-merging-two-trees (id 3439)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinDiameterMergeTrees case0: expected 3 got 1 |
Failed |
IN9 284
OUT692 |
| ✓ |
LcbMinimumAreaCoverOnesLiveCodeBench / leetcode / medium: find-the-minimum-area-to-cover-all-ones-i (id 3461)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 130
OUT224 |
| × |
LcbMinimumAverageLiveCodeBench / leetcode / easy: minimum-average-of-smallest-and-largest-elements (id 3471)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: Math.Abs(got - exp) < 1e-4
Context: MinimumAverage case0: expected 5.5 got 6 |
Failed |
IN9 185
OUT124 |
| × |
LcbPeaksInArrayLiveCodeBench / leetcode / hard: peaks-in-array (id 3438)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: PeaksInArray case1: expected [0, 1] got [1,1] |
Failed |
IN9 250
OUT428 |
Code Generation - LiveCodeBench v3 LeetCode (set 3)
4 passed3 failed1 skipped |
57.1% |
IN64 797
OUT1 336 |
| ✓ |
LcbBitChangesEqualLiveCodeBench / leetcode / easy: 3508 number-of-bit-changes-to-make-two-integers-equal1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 223
OUT92 |
| × |
LcbCountEqualXYSubmatricesLiveCodeBench / leetcode / medium: 3492 count-submatrices-with-equal-frequency-of-x-and-y1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfSubmatrices case0: expected 3 got 1 |
Failed |
IN9 238
OUT235 |
| × |
LcbMaxOpsMoveOnesLiveCodeBench / leetcode / medium: 3493 maximum-number-of-operations-to-move-ones-to-the-end1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxOperations case0: expected 4 got 7 |
Failed |
IN9 220
OUT105 |
| ✓ |
LcbMinCostConstructStringLiveCodeBench / leetcode / hard: 3482 construct-string-with-minimum-cost1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 334
OUT302 |
| ✓ |
LcbMinCostCutCakeLiveCodeBench / leetcode / medium: 3494 minimum-cost-for-cutting-cake-i1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 350
OUT257 |
| ✓ |
LcbSmallestStringAfterSwapLiveCodeBench / leetcode / easy: 3484 lexicographically-smallest-string-after-a-swap1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 216
OUT253 |
| – |
LcbValidBinaryStringsLiveCodeBench / leetcode / medium: 3453 generate-binary-strings-without-adjacent-zeros0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbVowelsGameLiveCodeBench / leetcode / medium: 3462 vowels-game-in-a-string1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: doesAliceWin case0: expected true got false |
Failed |
IN9 216
OUT92 |
Code Generation - LiveCodeBench v3 LeetCode (set 4)
3 passed5 failed |
37.5% |
IN74 297
OUT3 690 |
| × |
LcbAlternatingGroupsIIILiveCodeBench / leetcode / hard: 3527 alternating-groups-iii1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: numberOfAlternatingGroups case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
95:21: Variable 'oldPrevAlt' has type 'int' but its initializer has type 'bool'.
96:21: Variable 'oldNextAlt' has type 'int' but its initializer has type 'bool'.
97:21: Variable 'newPrevAlt' has type 'int' but its initializer has type 'bool'.
98:21: Variable 'newNextAlt' has type 'int' but its initializer has type 'bool'.
109:24: Cannot assign a value of type 'int' to an element of type 'bool'.
124:24: Cannot assign a value of type 'int' to an element of type 'bool'.
Values:
run.Compiled = false |
Failed |
IN9 403
OUT1 376 |
| ✓ |
LcbDigitGameWinLiveCodeBench / leetcode / easy: 3515 find-if-digit-game-can-be-won1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 257
OUT122 |
| × |
LcbDominantOnesSubstringsLiveCodeBench / leetcode / medium: 3479 count-the-number-of-substrings-with-dominant-ones1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfSubstrings case0: expected 5 got 3 |
Failed |
IN9 187
OUT415 |
| × |
LcbMinOpsArrayToTargetLiveCodeBench / leetcode / hard: 3454 minimum-operations-to-make-array-equal-to-target1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumOperations case1: expected 5 got 8 |
Failed |
IN9 216
OUT103 |
| ✓ |
LcbNonSpecialCountLiveCodeBench / leetcode / medium: 3507 find-the-count-of-numbers-which-are-not-special1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 241
OUT206 |
| × |
LcbRectangleCornerReachableLiveCodeBench / leetcode / hard: 3478 check-if-the-rectangle-corner-is-reachable1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: canReachCorner case0: expected true got false |
Failed |
IN9 392
OUT1 082 |
| × |
LcbShortestDistanceQueriesLiveCodeBench / leetcode / medium: 3517 shortest-distance-after-road-addition-queries-i1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: shortestDistanceAfterQueries case0 runtime error: List index 5 is out of range. The list has 5 element(s).
Values:
run.Success = false |
Failed |
IN9 333
OUT278 |
| ✓ |
LcbSnakeFinalPositionLiveCodeBench / leetcode / easy: 3533 snake-in-matrix1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 268
OUT108 |
Code Generation - LiveCodeBench v3 LeetCode (set 5)
2 passed6 failed |
25.0% |
IN73 510
OUT3 105 |
| ✓ |
LcbAlternatingGroupsCircleLiveCodeBench / leetcode / easy: alternating-groups-i (LCB id 3463)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 180
OUT137 |
| ✓ |
LcbClearDigitsLiveCodeBench / leetcode / easy: clear-digits (LCB id 3447)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 127
OUT132 |
| × |
LcbCountInversionPermutationsLiveCodeBench / leetcode / hard: count-the-number-of-inversions (LCB id 3460)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countInversionPermutations case0: expected 2 got 0 |
Failed |
IN9 282
OUT1 600 |
| × |
LcbCountMonotonicPairsLiveCodeBench / leetcode / hard: find-the-count-of-monotonic-pairs-i (LCB id 3535)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countMonotonicPairs case0: expected 4 got 0 |
Failed |
IN9 244
OUT201 |
| × |
LcbFirstPlayerWinKGamesLiveCodeBench / leetcode / medium: find-the-first-player-to-win-k-games-in-a-row (LCB id 3413)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: firstPlayerWinKGames case0: expected 2 got 4 |
Failed |
IN9 228
OUT401 |
| × |
LcbMaxGoodSubsequenceLengthLiveCodeBench / leetcode / medium: find-the-maximum-length-of-a-good-subsequence-i (LCB id 3456)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxGoodSubsequenceLength case0: expected 4 got 5 |
Failed |
IN9 167
OUT301 |
| × |
LcbMinOpsBinaryToOneLiveCodeBench / leetcode / medium: minimum-operations-to-make-binary-array-elements-equal-to-one-i (LCB id 3475)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minOpsBinaryToOne case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:17: Type 'List<int>' has no supported method named 'AllEqualToOne' with 0 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 161
OUT243 |
| × |
LcbMinOpsDivisibleThreeLiveCodeBench / leetcode / easy: find-minimum-operations-to-make-all-elements-divisible-by-three (LCB id 3476)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsDivisibleThree case0: expected 3 got 4 |
Failed |
IN9 121
OUT90 |
Code Generation - LiveCodeBench v3 LeetCode (set 6)
4 passed4 failed |
50.0% |
IN73 675
OUT6 403 |
| ✓ |
LcbCoinGameWinnerLiveCodeBench / leetcode / easy: find-the-winning-player-in-coin-game (LCB id 3511)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 176
OUT100 |
| ✓ |
LcbCountSubarraysAndKLiveCodeBench / leetcode / hard: number-of-subarrays-with-and-value-of-k (LCB id 3466)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 146
OUT143 |
| ✓ |
LcbCountWinningPlayersLiveCodeBench / leetcode / easy: find-the-number-of-winning-players (LCB id 3519)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 278
OUT265 |
| × |
LcbMaxPointsEnemyBattlesLiveCodeBench / leetcode / medium: maximum-points-after-enemy-battles (LCB id 3264)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxPointsEnemyBattles case0: expected 3 got 4 |
Failed |
IN9 243
OUT235 |
| × |
LcbMinChangesEqualDiffsLiveCodeBench / leetcode / medium: minimum-array-changes-to-make-differences-equal (LCB id 3498)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minChangesEqualDiffs case0: expected 2 got 0 |
Failed |
IN9 197
OUT256 |
| × |
LcbMinFlipsPalindromicGridLiveCodeBench / leetcode / medium: minimum-number-of-flips-to-make-binary-grid-palindromic-i (LCB id 3526)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minFlipsPalindromicGrid case0: expected 2 got 0 |
Failed |
IN9 192
OUT335 |
| ✓ |
LcbMinLengthAfterOpsLiveCodeBench / leetcode / medium: minimum-length-of-string-after-operations (LCB id 3455)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 164
OUT164 |
| × |
LcbTimeTakenMarkNodesLiveCodeBench / leetcode / hard: time-taken-to-mark-all-nodes (LCB id 3532)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: timeTakenMarkNodes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:48: Unexpected character '→'.
11:50: Unexpected character '→'.
15:21: Char literals must contain exactly one character and close with '''.
22:10: Unexpected character '–'.
22:20: Char literals must contain exactly one character and close with '''.
24:128: Char literals must contain exactly one character and close with '''.
26:9: Char literals must contain exactly one character and close with '''.
36:98: Char literals must contain exactly one character and close with '''.
38:166: Unexpected character '…'.
38:215: Unexpected character 'Σ'.
38:235: Unexpected character '≠'.
40:257: Unexpected character 'Σ'.
40:272: Unexpected character '≠'.
42:36: Unexpected character 'Σ'.
42:55: Unexpected character '→'.
46:136: Unexpected character 'Σ'.
46:155: Unexpected character '→'.
48:59: Unexpected character 'Σ'.
48:78: Unexpected character '→'.
49:16: Unexpected character '→'.
50:87: Unexpected character '∉'.
50:102: Unexpected character 'Σ'.
50:110: Unexpected character '→'.
50:134: Unexpected character '→'.
50:180: Unexpected character 'Σ'.
50:188: Unexpected character '→'.
50:412: Unexpected character 'Σ'.
50:420: Unexpected character '→'.
52:225: Unexpected character 'Σ'.
52:233: Unexpected character '→'.
52:581: Char literals must contain exactly one character and close with '''.
52:734: Unexpected character 'Σ'.
52:742: Unexpected character '→'.
52:840: Char literals must contain exactly one character and close with '''.
54:54: Unexpected character '∪'.
54:137: Char literals must contain exactly one character and close with '''.
56:96: Unexpected character '≤'.
56:133: Unexpected character '#'.
56:215: Char literals must contain exactly one character and close with '''.
61:25: Unexpected character '→'.
61:34: Unexpected character '→'.
62:29: Unexpected character '→'.
62:40: Unexpected character '✓'.
63:29: Unexpected character '→'.
63:40: Unexpected character '✓'.
66:39: Unexpected character '→'.
66:43: Unexpected character '→'.
66:47: Unexpected character '→'.
67:25: Unexpected character '✓'.
68:67: Unexpected character '✓'.
69:29: Unexpected character '✓'.
81:2: Char literals must contain exactly one character and close with '''.
93:4: Char literals must contain exactly one character and close with '''.
120:141: Unexpected character '≥'.
131:26: Unexpected character '≥'.
131:39: Unexpected character '≥'.
131:44: Unexpected character '≥'.
135:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 279
OUT4 905 |
Code Generation - LiveCodeBench v4 AtCoder (set 1)
5 passed3 failed1 skipped |
62.5% |
IN117 029
OUT35 867 |
| ✓ |
LcbBallsAndBagQueryLiveCodeBench / atcoder / medium: abc366_c Balls and Bag Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 213
OUT264 |
| × |
LcbCuboidSumQueryLiveCodeBench / atcoder / medium: abc366_d Cuboid Sum Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CuboidSumQuery case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 323
OUT574 |
| ✓ |
LcbCutTrailingZerosLiveCodeBench / atcoder / easy: abc367_b Cut .0 (Pattern B, stdin) NOTE (triage): float-formatting output — model must print X with no trailing zeros and no trailing decimal point. Compared as a trimmed string. Flagged for coordinator.1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 205
OUT147 |
| ✓ |
LcbElection2LiveCodeBench / atcoder / easy: abc366_a Election 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 219
OUT149 |
| ✓ |
LcbManhattanEllipseLiveCodeBench / atcoder / hard: abc366_e Manhattan Multifocal Ellipse (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN52 356
OUT33 926 |
| × |
LcbMaximumCompositionLiveCodeBench / atcoder / hard: abc366_f Maximum Composition (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MaximumComposition case0: expected [26] got [15] |
Failed |
IN9 240
OUT318 |
| × |
LcbShoutEverydayLiveCodeBench / atcoder / easy: abc367_a Shout Everyday (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ShoutEveryday case0: expected [Yes] got [No] |
Failed |
IN9 200
OUT127 |
| ✓ |
LcbVerticalWritingLiveCodeBench / atcoder / easy: abc366_b Vertical Writing (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 273
OUT362 |
| – |
LcbXorNeighborsLiveCodeBench / atcoder / hard: abc366_g XOR Neighbors (Pattern B, stdin) NOTE (triage): non-unique answer — any valid assignment is accepted, so exact-string comparison against the sample output will reject other correct solutions. Flagged for coordinator.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
Code Generation - LiveCodeBench v4 AtCoder (set 2)
1 passed7 failed |
12.5% |
IN85 062
OUT12 897 |
| × |
LcbAddMultiplyQueriesLiveCodeBench / atcoder / hard: abc368_g Add and Multiply Queries (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AddMultiplyQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
12:40: Unexpected character '≥'.
13:22: Unexpected character '≥'.
14:14: Unexpected character '≥'.
16:58: Unexpected character '≥'.
18:23: Char literals must contain exactly one character and close with '''.
28:97: Char literals must contain exactly one character and close with '''.
33:59: Unexpected character '≥'.
34:10: Unexpected character '≥'.
34:68: Unexpected character '≥'.
34:144: Unexpected character '→'.
36:125: Char literals must contain exactly one character and close with '''.
36:190: Char literals must contain exactly one character and close with '''.
36:230: Char literals must contain exactly one character and close with '''.
36:281: Char literals must contain exactly one character and close with '''.
36:313: Char literals must contain exactly one character and close with '''.
36:323: Char literals must contain exactly one character and close with '''.
36:358: Char literals must contain exactly one character and close with '''.
36:369: Char literals must contain exactly one character and close with '''.
38:69: Char literals must contain exactly one character and close with '''.
48:104: Char literals must contain exactly one character and close with '''.
50:46: Unexpected character 'φ'.
50:84: Unexpected character 'φ'.
50:137: Unexpected character 'φ'.
50:141: Unexpected character '∘'.
50:147: Unexpected character '∘'.
50:149: Unexpected character 'φ'.
50:159: Unexpected character 'φ'.
50:219: Char literals must contain exactly one character and close with '''.
52:6: Unexpected character 'φ'.
52:35: Unexpected character 'φ'.
52:68: Unexpected character 'φ'.
54:14: Unexpected character 'ψ'.
54:19: Unexpected character 'φ'.
54:23: Unexpected character 'φ'.
54:32: Unexpected character 'φ'.
55:1: Unexpected character 'ψ'.
55:11: Unexpected character 'φ'.
55:27: Unexpected character 'φ'.
57:28: Unexpected character '∈'.
59:110: Unexpected character '→'.
61:45: Char literals must contain exactly one character and close with '''.
61:143: Char literals must contain exactly one character and close with '''.
61:162: Char literals must contain exactly one character and close with '''.
63:32: Unexpected character '≤'.
65:218: Char literals must contain exactly one character and close with '''.
67:116: Unexpected character '→'.
69:268: Unexpected character '→'.
77:8: Unexpected character '—'.
77:21: Char literals must contain exactly one character and close with '''.
79:70: Unexpected character 'φ'.
79:281: Unexpected character 'φ'.
81:85: Unexpected character '≥'.
81:145: Unexpected character '≥'.
83:968: Char literals must contain exactly one character and close with '''.
83:1228: Char literals must contain exactly one character and close with '''.
85:35: Unexpected character '⊆'.
85:47: Unexpected character '∏'.
85:51: Unexpected character '∈'.
85:121: Unexpected character '≤'.
85:124: Char literals must contain exactly one character and close with '''.
85:134: Char literals must contain exactly one character and close with '''.
85:144: Unexpected character '≥'.
89:2: Char literals must contain exactly one character and close with '''.
91:309: Char literals must contain exactly one character and close with '''.
93:42: Char literals must contain exactly one character and close with '''.
95:27: Char literals must contain exactly one character and close with '''.
97:86: Unexpected character '≤'.
97:223: Unexpected character '→'.
97:305: Unexpected character '≤'.
97:316: Unexpected character '≥'.
97:365: Unexpected character '≥'.
99:29: Char literals must contain exactly one character and close with '''.
101:233: Char literals must contain exactly one character and close with '''.
101:382: Char literals must contain exactly one character and close with '''.
109:4: Char literals must contain exactly one character and close with '''.
115:276: Unexpected character '→'.
123:275: Unexpected character '∘'.
123:323: Unexpected character '∘'.
123:458: Unexp ... (message truncated) |
Failed |
IN20 315
OUT3 414 |
| × |
LcbCardCutLiveCodeBench / atcoder / easy: abc368_a Cut (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CardCut case0: expected [3 4 5 1 2] got [4 5 1 2 3] |
Failed |
IN9 219
OUT176 |
| ✓ |
LcbDecreaseTwoMaxLiveCodeBench / atcoder / easy: abc368_b Decrease 2 max elements (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 204
OUT190 |
| × |
LcbEnumerateSequencesLiveCodeBench / atcoder / medium: abc367_c Enumerate Sequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchExternal function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed |
IN9 216
OUT383 |
| × |
LcbPedometerLiveCodeBench / atcoder / medium: abc367_d Pedometer (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Pedometer case0: expected [4] got [-6] |
Failed |
IN9 269
OUT355 |
| × |
LcbPermuteKTimesLiveCodeBench / atcoder / hard: abc367_e Permute K times (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PermuteKTimes case0: expected [7 2 3 5 1 9 3] got [7 2 3 5 7 9 3] |
Failed |
IN9 244
OUT408 |
| × |
LcbRearrangeQueryLiveCodeBench / atcoder / hard: abc367_f Rearrange Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RearrangeQuery case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
7:426: Unexpected character '≤'.
9:45: Unexpected character '≤'.
11:9: Unexpected character '—'.
11:136: Unexpected character '→'.
13:51: Char literals must contain exactly one character and close with '''.
15:2: Char literals must contain exactly one character and close with '''.
22:102: Unexpected character '#'.
22:120: Unexpected character '#'.
25:49: Char literals must contain exactly one character and close with '''.
33:24: Unexpected character '\'.
44:19: Integer literal '0x9E3779B97F4A7C15L' is out of range for type long.
44:64: Integer literal '0xBF58476D1CE4E5B9L' is out of range for type long.
44:99: Integer literal '0x94D049BB133111EBL' is out of range for type long.
45:31: Integer literal '0x9E3779B97F4A7C15' is out of range for type long.
Values:
run.Compiled = false |
Failed |
IN9 297
OUT1 791 |
| × |
LcbSumXorKLiveCodeBench / atcoder / hard: abc367_g Sum of (XOR^K or 0) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SumXorK case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
11:50: Char literals must contain exactly one character and close with '''.
13:138: Unexpected character '≥'.
15:20: Unexpected character '⊆'.
19:67: Unexpected character '∈'.
23:76: Unexpected character '—'.
23:115: Char literals must contain exactly one character and close with '''.
25:22: Char literals must contain exactly one character and close with '''.
27:358: Char literals must contain exactly one character and close with '''.
29:325: Unexpected character '≡'.
31:78: Unexpected character '·'.
34:92: Unexpected character '·'.
41:204: Unexpected character '∈'.
41:232: Unexpected character '≥'.
43:105: Unexpected character '≥'.
45:51: Unexpected character '≥'.
45:150: Unexpected character '→'.
45:230: Char literals must contain exactly one character and close with '''.
47:47: Unexpected character '⊇'.
47:78: Unexpected character '#'.
49:22: Unexpected character '#'.
52:14: Unexpected character '⊆'.
52:75: Unexpected character 'ö'.
52:109: Char literals must contain exactly one character and close with '''.
55:13: Unexpected character '⊆'.
55:26: Unexpected character '∏'.
55:30: Unexpected character '∈'.
55:262: Unexpected character '⊆'.
55:328: Unexpected character '∈'.
55:375: Unexpected character '∈'.
57:57: Unexpected character '∏'.
57:61: Unexpected character '∈'.
57:74: Unexpected character '#'.
61:88: Char literals must contain exactly one character and close with '''.
63:93: Unexpected character '∈'.
65:58: Unexpected character '≤'.
65:66: Unexpected character '≤'.
69:25: Char literals must contain exactly one character and close with '''.
71:58: Char literals must contain exactly one character and close with '''.
73:8: Unexpected character '—'.
75:30: Char literals must contain exactly one character and close with '''.
79:114: Unexpected character '∈'.
79:362: Unexpected character '∈'.
79:396: Unexpected character '∈'.
79:528: Unexpected character '∈'.
81:36: Unexpected character '∈'.
81:60: Unexpected character '±'.
81:126: Unexpected character '≡'.
81:168: Unexpected character 'ω'.
85:72: Char literals must contain exactly one character and close with '''.
87:37: Char literals must contain exactly one character and close with '''.
93:57: Unexpected character '—'.
93:221: Unexpected character 'ω'.
95:108: Unexpected character 'ω'.
95:125: Unexpected character 'χ'.
95:195: Unexpected character 'χ'.
95:219: Unexpected character 'ω'.
97:33: Unexpected character 'χ'.
97:51: Unexpected character '⊆'.
97:67: Unexpected character 'χ'.
97:106: Unexpected character 'χ'.
99:101: Unexpected character 'χ'.
99:129: Unexpected character 'χ'.
101:15: Unexpected character 'χ'.
101:24: Unexpected character '±'.
101:183: Unexpected character 'σ'.
101:207: Unexpected character 'σ'.
103:136: Unexpected character 'σ'.
103:154: Unexpected character 'σ'.
103:175: Unexpected character 'σ'.
103:211: Unexpected character 'σ'.
105:219: Unexpected character '→'.
107:145: Unexpected character '≥'.
107:212: Unexpected character '≈'.
107:233: Unexpected character '—'.
109:98: Char literals must contain exactly one character and close with '''.
111:69: Unexpected character '≈'.
111:120: Unexpected character '→'.
115:562: Unexpected character '→'.
121:12: Unexpected character '→'.
122:14: Unexpected character '→'.
124:83: Unexpected character 'ω'.
124:90: Unexpected character 'ω'.
124:141: Unexpected character 'ω'.
124:237: Unexpected character '≤'.
124:283: Unexpected character '≤'.
124:379: Unexpected character '∤'.
124:391: Char literals must contain exactly one character and close with '''.
126:347: Unexpected character 'ω'.
128:156: Unexpected character '≡'.
130:80: Unexpected character 'χ'.
132:131: Unexpected character '≡'.
132:166: Unexpected character 'Σ'.
132:172: Unexpected character 'ω'.
132:311: Unexpected character '≡'.
132:479: Unexpected character '≡'.
132:516: Unexpected character '≡'.
13 ... (message truncated) |
Failed |
IN9 298
OUT6 180 |
Code Generation - LiveCodeBench v4 AtCoder (set 3)
9 failed |
0.0% |
IN83 493
OUT7 467 |
| × |
LcbArrangeArithmeticLiveCodeBench / atcoder / abc369_a: 369 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ArrangeArithmetic case1: expected [2] got [3] |
Failed |
IN9 211
OUT81 |
| × |
LcbAsFarAsPossibleLiveCodeBench / atcoder / abc369_g: As far as possible (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AsFarAsPossible case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:42: Char literals must contain exactly one character and close with '''.
11:26: Char literals must contain exactly one character and close with '''.
15:69: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:72: Unexpected character '\'.
15:80: Unexpected character '\'.
15:149: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:166: Unexpected character '—'.
15:404: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:407: Unexpected character '\'.
15:417: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:425: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:430: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:484: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:485: Unexpected character '\'.
15:488: Unexpected character '\'.
15:491: Unexpected character '\'.
15:497: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:206: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
17:209: Unexpected character '\'.
17:215: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:13: Char literals must contain exactly one character and close with '''.
21:15: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
21:20: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
21:44: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
21:49: Unexpected character '\'.
21:57: Unexpected character '\'.
21:110: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
23:7: Char literals must contain exactly one character and close with '''.
25:7: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
25:15: Unexpected character '\'.
25:26: Unexpected character '\'.
25:33: Unexpected character '\'.
25:43: Unexpected character '\'.
25:53: Unexpected character '\'.
25:82: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
25:187: Unexpected character '≥'.
27:35: Unexpected character '≠'.
29:103: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
29:108: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
29:337: Unexpected character '→'.
29:420: Unexpected character '→'.
31:138: Char literals must contain exactly one character and close with '''.
31:187: Unexpected character '→'.
31:251: Unexpected character '…'.
31:413: Char literals must contain exactly one character and close with '''.
33:52: Unexpected character '≠'.
33:303: Char literals must contain exactly one character and close with '''.
35:160: Unexpected character '→'.
35:228: Char literals must contain exactly one character and close with '''.
37:213: Char literals must contain exactly one character and close with '''.
39:93: Unexpected character '·'.
39:111: Unexpected character '≥'.
39:180: Unexpected character '−'.
39:191: Unexpected character '·'.
41:38: Char literals must contain exactly one character and close with '''.
43:29: Unexpected character 'Σ'.
43:37: Unexpected character '·'.
43:81: Unexpected character '⊇'.
49:189: Unexpected character '—'.
51:76: Unexpected character 'Σ'.
51:86: Unexpected character '·'.
51:165: Char literals must contain exactly one character and close with '''.
53:80: Unexpected character '→'.
55:328: Unexpected character '→'.
60:35: Unexpected character '⇒'.
60:62: Unexpected character '⇒'.
60:74: Unexpected character 'Σ'.
61:7: Unexpected character '≥'.
63:34: Unexpected character '≥'.
66:10: Unexpected character 'Σ'.
67:6: Unexpected character '≥ ... (message truncated) |
Failed |
IN9 370
OUT4 694 |
| × |
LcbBonusExpLiveCodeBench / atcoder / abc369_d: Bonus EXP (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BonusExp case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:29: Expected ')' to close the argument list, found '{'.
Values:
run.Compiled = false |
Failed |
IN9 275
OUT233 |
| × |
LcbCountArithmeticSubarraysLiveCodeBench / atcoder / abc369_c: Count Arithmetic Subarrays (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountArithmeticSubarrays case0: expected [8] got [9] |
Failed |
IN9 256
OUT218 |
| × |
LcbDividingGameLiveCodeBench / atcoder / abc368_f: Dividing Game (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DividingGame case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 260
OUT261 |
| × |
LcbMinimumSteinerTreeLiveCodeBench / atcoder / abc368_d: Minimum Steiner Tree (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimumSteinerTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
17:22: Expected a type name after 'new', found 'bool'.
Values:
run.Compiled = false |
Failed |
IN9 218
OUT855 |
| × |
LcbPiano3LiveCodeBench / atcoder / abc369_b: Piano 3 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Piano3 case0: expected [11] got [107] |
Failed |
IN9 308
OUT230 |
| × |
LcbTrainDelayLiveCodeBench / atcoder / abc368_e: Train Delay (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TrainDelay case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
38:29: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed |
IN9 342
OUT726 |
| × |
LcbTripleAttackLiveCodeBench / atcoder / abc368_c: Triple Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TripleAttack case0: expected [8] got [18] |
Failed |
IN9 253
OUT169 |
Code Generation - LiveCodeBench v4 AtCoder (set 4)
2 passed6 failed |
25.0% |
IN111 294
OUT11 711 |
| × |
LcbAvoidKPartitionLiveCodeBench / atcoder / abc370_e: Avoid K Partition (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AvoidKPartition case0: expected [2] got [3] |
Failed |
IN9 318
OUT305 |
| ✓ |
LcbBinaryAlchemyLiveCodeBench / atcoder / abc370_b: Binary Alchemy (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 250
OUT221 |
| × |
LcbCakeDivisionLiveCodeBench / atcoder / abc370_f: Cake Division (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CakeDivision case0: expected [13 1] got [13 4] |
Failed |
IN33 389
OUT3 028 |
| × |
LcbCrossExplosionLiveCodeBench / atcoder / abc370_d: Cross Explosion (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: CrossExplosion case0 runtime error: List index 0 is out of range. The list has 0 element(s).
Values:
run.Success = false |
Failed |
IN9 346
OUT932 |
| × |
LcbDivisibleByThreeLiveCodeBench / atcoder / abc370_g: Divisible by 3 (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DivisibleByThree case0: expected [5] got [6] |
Failed |
IN22 085
OUT6 538 |
| ✓ |
LcbRaiseBothHandsLiveCodeBench / atcoder / abc370_a: Raise Both Hands (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 265
OUT92 |
| × |
LcbSightseeingTourLiveCodeBench / atcoder / abc369_e: Sightseeing Tour (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SightseeingTour case0: expected [25
70] got [10
55] |
Failed |
IN9 366
OUT327 |
| × |
LcbWordLadderLiveCodeBench / atcoder / abc370_c: Word Ladder (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WordLadder case0: expected [3
acbe
acbc
bcbc] got [3
bdbe
bcbe
bcbc] |
Failed |
IN9 275
OUT268 |
Code Generation - LiveCodeBench v4 AtCoder (set 5)
4 passed5 failed |
44.4% |
IN83 455
OUT5 796 |
| ✓ |
LcbCountABCAgainLiveCodeBench / atcoder / medium: abc372_c Count ABC Again (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 241
OUT419 |
| × |
LcbDistinctSubarraySumLiveCodeBench / atcoder / hard: abc371_e I Hate Sigma Problems (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DistinctSubarraySum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
28:49: Unexpected character '#'.
28:56: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 209
OUT505 |
| × |
LcbKthLargestConnectedLiveCodeBench / atcoder / hard: abc372_e K (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchExternal function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed |
IN9 279
OUT293 |
| ✓ |
LcbLexSmallestPermLiveCodeBench / atcoder / hard: abc371_g Lexicographically Smallest Permutation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 248
OUT464 |
| × |
LcbMakeIsomorphicLiveCodeBench / atcoder / medium: abc371_c Make Isomorphic (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MakeIsomorphic case0: expected [9] got [20] |
Failed |
IN9 402
OUT685 |
| × |
LcbMiddleBrotherLiveCodeBench / atcoder / easy: abc371_a Jiro (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MiddleBrother case0: expected [B] got [] |
Failed |
IN9 228
OUT140 |
| × |
LcbNarrowRoadLiveCodeBench / atcoder / hard: abc371_f Takahashi in Narrow Road (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NarrowRoad case0: expected [239] got [103] |
Failed |
IN9 289
OUT2 542 |
| ✓ |
LcbOneDCountryLiveCodeBench / atcoder / medium: abc371_d 1D Country (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 281
OUT520 |
| ✓ |
LcbTaroNameLiveCodeBench / atcoder / easy: abc371_b Taro (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 278
OUT228 |
Code Generation - LiveCodeBench v4 AtCoder (set 6)
4 passed3 failed1 skipped |
57.1% |
IN64 731
OUT1 891 |
| × |
LcbAxByLessCLiveCodeBench / atcoder / hard: abc372_g Ax + By < C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AxByLessC case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:20: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed |
IN9 304
OUT586 |
| ✓ |
LcbDeleteDotsLiveCodeBench / atcoder / easy: abc372_a delete . (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 153
OUT28 |
| ✓ |
LcbDiminishingKnapsackLiveCodeBench / atcoder / hard: abc373_f Knapsack with Diminishing Values (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 245
OUT404 |
| – |
LcbNoCrossMatchingLiveCodeBench / atcoder / hard: abc373_g No Cross Matching (Pattern B, stdin) NOTE: non-unique answer — any non-crossing perfect matching R is valid; the exact-match oracle may reject a different valid R.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbOneDKeyboardLiveCodeBench / atcoder / easy: abc373_b 1D Keyboard (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 246
OUT152 |
| ✓ |
LcbSeptemberLengthsLiveCodeBench / atcoder / easy: abc373_a September (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 183
OUT74 |
| × |
LcbTeleportingWalksLiveCodeBench / atcoder / hard: abc372_f Teleporting Takahashi 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: TeleportingWalks case1 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 364
OUT324 |
| × |
LcbVisibleBuildingsLiveCodeBench / atcoder / medium: abc372_d Buildings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: VisibleBuildings case0: expected [3 2 2 1 0] got [2 1 2 1 0] |
Failed |
IN9 236
OUT323 |
Code Generation - LiveCodeBench v4 AtCoder (set 7)
1 passed5 failed1 skipped |
16.7% |
IN67 359
OUT11 524 |
| × |
LcbAnnoyingStringLiveCodeBench / atcoder / hard: arc181_b Annoying String Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AnnoyingString case0: expected [Yes
No
No] got [Yes
Yes
Yes] |
Failed |
IN9 263
OUT309 |
| × |
LcbIncDecAgainLiveCodeBench / atcoder / hard: arc182_d Increment Decrement Again (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: IncDecAgain case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
2:12: Expected a field name after '.', found '.'.
Values:
run.Compiled = false |
Failed |
IN21 089
OUT4 482 |
| ✓ |
LcbMaxAiBjLiveCodeBench / atcoder / medium: abc373_c Max Ai+Bj (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 174
OUT179 |
| × |
LcbPrefixBubbleSortLiveCodeBench / atcoder / hard: arc181_d Prefix Bubble Sort (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PrefixBubbleSort case0: expected [3
1] got [1
-8] |
Failed |
IN9 294
OUT410 |
| – |
LcbRowColumnOrderLiveCodeBench / atcoder / hard: arc181_c Row and Column Order (Pattern B, stdin) TRIAGE: non-unique answer (any valid grid accepted) — exact string compare may fail correct solutions.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbSortLeftRightLiveCodeBench / atcoder / medium: arc181_a Sort Left and Right (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SortLeftRight case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:56: Unexpected character '∈'.
4:7: Unexpected character '≥'.
5:7: Unexpected character '≤'.
11:171: Unexpected character '→'.
11:272: Unexpected character '…'.
13:23: Char literals must contain exactly one character and close with '''.
19:42: Char literals must contain exactly one character and close with '''.
21:246: Char literals must contain exactly one character and close with '''.
23:134: Unexpected character '≠'.
23:144: Unexpected character '…'.
27:39: Unexpected character '→'.
28:30: Unexpected character '→'.
29:34: Unexpected character '→'.
29:69: Unexpected character '→'.
29:136: Unexpected character '→'.
29:169: Unexpected character '→'.
29:250: Unexpected character '→'.
31:49: Unexpected character '→'.
31:82: Unexpected character '→'.
31:122: Unexpected character '→'.
31:142: Unexpected character '…'.
31:191: Unexpected character '→'.
31:226: Unexpected character '→'.
31:255: Unexpected character '→'.
31:311: Char literals must contain exactly one character and close with '''.
33:30: Unexpected character '→'.
33:46: Unexpected character '→'.
33:59: Unexpected character '→'.
41:21: Unexpected character '#'.
43:225: Unexpected character '…'.
43:439: Unexpected character '→'.
43:447: Unexpected character '✓'.
43:485: Unexpected character '→'.
43:489: Unexpected character '→'.
43:525: Unexpected character '→'.
43:528: Unexpected character '✗'.
43:561: Unexpected character '—'.
43:642: Unexpected character '≤'.
46:84: Unexpected character '→'.
46:123: Unexpected character '→'.
47:45: Unexpected character '→'.
47:58: Unexpected character '→'.
48:45: Unexpected character '→'.
49:11: Unexpected character '≥'.
53:176: Char literals must contain exactly one character and close with '''.
55:186: Unexpected character '…'.
57:227: Unexpected character '→'.
57:245: Unexpected character '→'.
61:2: Char literals must contain exactly one character and close with '''.
63:16: Char literals must contain exactly one character and close with '''.
65:21: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 266
OUT3 614 |
| × |
LcbWinElectionLiveCodeBench / atcoder / hard: abc373_e How to Win the Election (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WinElection case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:3: Expected a name, found '='.
Values:
run.Compiled = false |
Failed |
IN9 273
OUT2 530 |
Code Generation - LiveCodeBench v4 AtCoder (set 8)
6 failed1 skipped |
0.0% |
IN66 743
OUT13 907 |
| × |
LcbChmaxRushLiveCodeBench / atcoder / hard: arc182_a Chmax Rush! (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ChmaxRush case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:5: Unexpected character '#'.
13:13: Unexpected character '#'.
25:13: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN20 711
OUT9 627 |
| × |
LcbDivisorsOfProductLiveCodeBench / atcoder / hard: arc182_c Sum of Number of Divisors of Product (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DivisorsOfProduct case0: expected [16] got [499122178] |
Failed |
IN9 205
OUT261 |
| – |
LcbKeepMatchedLiveCodeBench / atcoder / hard: arc183_d Keep Perfectly Matched (Pattern B, stdin) TRIAGE: non-unique answer (any maximizing procedure accepted) — exact string compare may fail correct solutions.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbMedianGoodSeqLiveCodeBench / atcoder / medium: arc183_a Median of Good Sequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MedianGoodSeq case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:9: Char literals must contain exactly one character and close with '''.
12:26: Char literals must contain exactly one character and close with '''.
16:231: Unexpected character '→'.
18:151: Char literals must contain exactly one character and close with '''.
20:5: Unexpected character '—'.
20:8: Unexpected character '#'.
26:35: Unexpected character '—'.
26:195: Unexpected character '≈'.
28:48: Unexpected character '≤'.
30:2: Char literals must contain exactly one character and close with '''.
32:55: Unexpected character '∏'.
32:213: Unexpected character '∏'.
Values:
run.Compiled = false |
Failed |
IN9 177
OUT1 956 |
| × |
LcbNearAssignmentLiveCodeBench / atcoder / hard: arc183_b Near Assignment (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: NearAssignment case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
55:41: Type 'Dictionary<long, int>' has no supported method named 'GetValueOrDefault' with 1 argument(s).
62:41: Type 'Dictionary<long, int>' has no supported method named 'GetValueOrDefault' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 242
OUT1 040 |
| × |
LcbNotArgmaxLiveCodeBench / atcoder / hard: arc183_c Not Argmax (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: NotArgmax case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 227
OUT614 |
| × |
LcbSumMinModLinearLiveCodeBench / atcoder / hard: arc182_e Sum of Min of Mod of Linear (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SumMinModLinear case0: expected [4] got [3] |
Failed |
IN9 181
OUT409 |
Code Generation - LiveCodeBench v4 LeetCode (set 1)
1 passed5 failed |
16.7% |
IN55 193
OUT2 140 |
| × |
LcbAlmostEqualPairsLiveCodeBench / leetcode / medium: count-almost-equal-pairs-i (id 3534)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: almostequal case0: expected 2 got 1 |
Failed |
IN9 200
OUT568 |
| × |
LcbBeautifulArraySumLiveCodeBench / leetcode / medium: find-the-minimum-possible-sum-of-a-beautiful-array (id 3026)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: beautiful case0: expected 4 got 3 |
Failed |
IN9 189
OUT172 |
| ✓ |
LcbFinalArrayStateLiveCodeBench / leetcode / easy: final-array-state-after-k-multiplication-operations-i (id 3555)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 212
OUT135 |
| × |
LcbKConstraintSubstringsLiveCodeBench / leetcode / easy: count-substrings-that-satisfy-k-constraint-i (id 3543)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: kconstraint case0: expected 12 got 8 |
Failed |
IN9 193
OUT251 |
| × |
LcbLargestKPalindromeLiveCodeBench / leetcode / hard: find-the-largest-palindrome-divisible-by-k (id 3552)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: kpalindrome case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:27: Unknown function 'BuildPalindrome'.
30:17: Unknown function 'ModPow'.
57:27: Unknown function 'BuildPalindromeWithTail'.
58:17: Unknown function 'ModPow'.
68:25: Unknown function 'BuildPalindrome'.
69:13: Unknown function 'ModPow'.
Values:
run.Compiled = false |
Failed |
IN9 164
OUT759 |
| × |
LcbMaxEnergyBoostLiveCodeBench / leetcode / medium: maximum-energy-boost-from-two-drinks (id 3525)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: energyboost case1: expected 7 got 6 |
Failed |
IN9 235
OUT255 |
Code Generation - LiveCodeBench v4 LeetCode (set 2)
2 passed4 failed |
33.3% |
IN55 280
OUT5 218 |
| × |
LcbConvertDateBinaryLiveCodeBench / leetcode / easy: convert-date-to-binary (id 3567)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: datebinary case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'convertDateToBinary' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, ConvertDateToBinary, IntToBinary.
Values:
run.Compiled = false |
Failed |
IN9 183
OUT215 |
| ✓ |
LcbKthNearestObstacleLiveCodeBench / leetcode / medium: k-th-nearest-obstacle-queries (id 3495)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 252
OUT154 |
| × |
LcbMaxCellScoreLiveCodeBench / leetcode / hard: select-cells-in-grid-with-maximum-score (id 3563)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxcell case0: expected 8 got 10 |
Failed |
IN9 180
OUT251 |
| × |
LcbMaxRangeScoreLiveCodeBench / leetcode / medium: maximize-score-of-numbers-in-ranges (id 3485)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: rangescore case0: expected 4 got 0 |
Failed |
IN9 175
OUT330 |
| × |
LcbMaxXorSubarrayLiveCodeBench / leetcode / hard: maximum-xor-score-subarray-queries (id 3551)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxxor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:13: Unexpected character '`'.
5:15: Unexpected character '`'.
6:12: Unexpected character '`'.
6:17: Unexpected character '`'.
6:24: Unexpected character '`'.
6:40: Unexpected character '`'.
6:50: Unexpected character '`'.
6:52: Unexpected character '`'.
9:8: Unexpected character '`'.
9:18: Unexpected character '`'.
10:11: Unexpected character '`'.
10:22: Unexpected character '`'.
10:24: Unexpected character '→'.
10:38: Unexpected character '→'.
16:5: Unexpected character '`'.
16:15: Unexpected character '`'.
17:16: Unexpected character '→'.
17:28: Unexpected character '→'.
17:48: Unexpected character '`'.
17:62: Unexpected character '`'.
17:77: Unexpected character '→'.
17:79: Unexpected character '`'.
17:90: Unexpected character '`'.
19:21: Char literals must contain exactly one character and close with '''.
19:207: Unexpected character '`'.
19:218: Unexpected character '`'.
21:16: Unexpected character '`'.
21:27: Unexpected character '`'.
22:44: Unexpected character '`'.
22:63: Unexpected character '`'.
22:78: Unexpected character '→'.
22:80: Unexpected character '`'.
22:94: Unexpected character '`'.
30:24: Unexpected character '`'.
30:30: Unexpected character '`'.
30:32: Unexpected character '→'.
30:47: Unexpected character '→'.
30:59: Unexpected character '`'.
30:68: Unexpected character '`'.
30:83: Unexpected character '→'.
30:85: Unexpected character '`'.
30:91: Unexpected character '`'.
34:22: Unexpected character '`'.
34:32: Unexpected character '`'.
35:37: Unexpected character '→'.
35:39: Unexpected character '`'.
35:58: Unexpected character '`'.
35:73: Unexpected character '→'.
35:75: Unexpected character '`'.
35:91: Unexpected character '`'.
36:59: Unexpected character '→'.
36:61: Unexpected character '`'.
36:77: Unexpected character '`'.
36:92: Unexpected character '→'.
36:94: Unexpected character '`'.
36:105: Unexpected character '`'.
37:34: Unexpected character '→'.
37:36: Unexpected character '`'.
37:55: Unexpected character '`'.
37:70: Unexpected character '→'.
37:72: Unexpected character '`'.
37:86: Unexpected character '`'.
39:32: Char literals must contain exactly one character and close with '''.
45:184: Char literals must contain exactly one character and close with '''.
53:17: Char literals must contain exactly one character and close with '''.
55:150: Unexpected character '→'.
59:157: Unexpected character '→'.
61:103: Unexpected character '⊆'.
65:89: Unexpected character '⊆'.
67:26: Unexpected character '⊆'.
69:60: Char literals must contain exactly one character and close with '''.
73:127: Unexpected character '#'.
73:249: Unexpected character '≈'.
79:121: Unexpected character '≡'.
87:196: Unexpected character '→'.
89:111: Unexpected character '≤'.
89:124: Unexpected character '≤'.
89:142: Unexpected character '∈'.
89:156: Unexpected character '∈'.
91:128: Unexpected character '≤'.
91:176: Unexpected character '∈'.
91:185: Unexpected character '∈'.
93:28: Char literals must contain exactly one character and close with '''.
95:2: Char literals must contain exactly one character and close with '''.
155:96: Unexpected character '✓'.
157:202: Unexpected character '→'.
157:332: Unexpected character '✓'.
157:427: Unexpected character '→'.
157:438: Unexpected character '✓'.
159:92: Unexpected character '⊆'.
159:107: Unexpected character '→'.
159:147: Char literals must contain exactly one character and close with '''.
160:50: Unexpected character '⊆'.
160:86: Unexpected character '✓'.
161:53: Unexpected character '⊆'.
161:63: Unexpected character '∈'.
161:111: Unexpected character '✓'.
163:83: Unexpected character '✓'.
164:69: Unexpected character '✓'.
165:75: Unexpected character '✓'.
Values:
run.Compiled = false |
Failed |
IN9 315
OUT4 149 |
| ✓ |
LcbSameChessboardColorLiveCodeBench / leetcode / easy: check-if-two-chessboard-squares-have-the-same-color (id 3553)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT119 |
Code Generation - LiveCodeBench v4 LeetCode (set 3)
4 passed2 failed |
66.7% |
IN55 521
OUT2 943 |
| × |
LcbKnightKillPawnsLiveCodeBench / leetcode / hard: maximum-number-of-moves-to-kill-all-pawns (LCB 3560)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: knightPawns case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
62:9: Variable 'aliceTurn' has type 'int' but its initializer has type 'bool'.
63:9: The if condition must be 'bool', but it has type 'int'.
Values:
run.Compiled = false |
Failed |
IN9 345
OUT1 706 |
| ✓ |
LcbMaxMultiplicationScoreLiveCodeBench / leetcode / medium: maximum-multiplication-score (LCB 3518)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 290
OUT281 |
| × |
LcbMinValidStringsLiveCodeBench / leetcode / medium: minimum-number-of-valid-strings-to-form-target-i (LCB 3559)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minValid case0: expected 3 got -1 |
Failed |
IN9 187
OUT293 |
| ✓ |
LcbPowerKSizeSubarraysLiveCodeBench / leetcode / medium: find-the-power-of-k-size-subarrays-i (LCB 3522)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 255
OUT195 |
| ✓ |
LcbReachEndMaxScoreLiveCodeBench / leetcode / medium: reach-end-of-array-with-max-score (LCB 3528)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 211
OUT176 |
| ✓ |
LcbThreeRooksMaxSumLiveCodeBench / leetcode / hard: maximum-value-sum-by-placing-three-rooks-i (LCB 3550)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 233
OUT292 |
Code Generation - LiveCodeBench v4 LeetCode (set 4)
2 passed4 failed |
33.3% |
IN55 672
OUT6 146 |
| × |
LcbCountGoodIntegersLiveCodeBench / leetcode / hard: find-the-count-of-good-integers (LCB 3548)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: goodInt case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
83:3: Char literals must contain exactly one character and close with '''.
87:107: Char literals must contain exactly one character and close with '''.
93:178: Char literals must contain exactly one character and close with '''.
103:5: Char literals must contain exactly one character and close with '''.
107:149: Char literals must contain exactly one character and close with '''.
145:8: Char literals must contain exactly one character and close with '''.
157:35: Char literals must contain exactly one character and close with '''.
208:21: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 272
OUT4 300 |
| ✓ |
LcbGenerateKeyNumbersLiveCodeBench / leetcode / easy: find-the-key-of-the-numbers (LCB 3568)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 296
OUT239 |
| ✓ |
LcbHashDividedStringLiveCodeBench / leetcode / medium: hash-divided-string (LCB 3540)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 273
OUT124 |
| × |
LcbMaxSequenceValueLiveCodeBench / leetcode / hard: find-the-maximum-sequence-value-of-array (LCB 3575)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxSeq case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:48: List constructor expects a List or HashSet to copy, but got 'int'.
20:48: List constructor expects a List or HashSet to copy, but got 'int'.
24:32: List constructor expects a List or HashSet to copy, but got 'int'.
25:32: List constructor expects a List or HashSet to copy, but got 'int'.
Values:
run.Compiled = false |
Failed |
IN9 236
OUT489 |
| × |
LcbMinDamageToBobLiveCodeBench / leetcode / hard: minimum-amount-of-damage-dealt-to-bob (LCB 3531)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minDmg case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
38:14: Unknown function 'HealthAt'.
39:14: Unknown function 'HealthAt'.
42:14: Unknown function 'DamageAt'.
43:14: Unknown function 'DamageAt'.
Values:
run.Compiled = false |
Failed |
IN9 271
OUT411 |
| × |
LcbSafeWalkGridLiveCodeBench / leetcode / medium: find-a-safe-walk-through-a-grid (LCB 3558)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: safeWalk case2: expected true got false |
Failed |
IN9 324
OUT583 |
Code Generation - LiveCodeBench v4 LeetCode (set 5)
3 passed3 failed |
50.0% |
IN55 287
OUT1 523 |
| × |
LcbKthCharacterGameLiveCodeBench 3600 / leetcode / easy: find the k-th character in string game I (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: kthCharacterGame case0: expected "b" got "a" |
Failed |
IN9 208
OUT95 |
| ✓ |
LcbLongestIncreasingPathLiveCodeBench 3571 / leetcode / hard: length of the longest increasing path (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 281
OUT354 |
| ✓ |
LcbMountainHeightZeroLiveCodeBench 3496 / leetcode / medium: minimum number of seconds to make mountain height zero (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 281
OUT322 |
| ✓ |
LcbReportSpamLiveCodeBench 3541 / leetcode / medium: report spam message (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 164
OUT82 |
| × |
LcbSubstringsVowelsKConsonantsLiveCodeBench 3570 / leetcode / medium: count of substrings containing every vowel and k consonants I (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: substringsVowelsKConsonants case2: expected 3 got 0 |
Failed |
IN9 168
OUT339 |
| × |
LcbValidSubstringCountLiveCodeBench 3573 / leetcode / medium: count substrings that can be rearranged to contain a string I (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: validSubstringCount case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 185
OUT331 |
Code Generation - LiveCodeBench v4 LeetCode (set 6)
2 passed4 failed |
33.3% |
IN55 312
OUT4 895 |
| ✓ |
LcbFirstAlmostEqualSubstringLiveCodeBench 3580 / leetcode / hard: find the occurrence of first almost equal substring (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 200
OUT180 |
| × |
LcbLexSmallestValidSequenceLiveCodeBench 3584 / leetcode / medium: find the lexicographically smallest valid sequence (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: lexSmallestValidSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
9:22: Unexpected character '`'.
9:31: Unexpected character '`'.
9:33: Unexpected character '∈'.
9:167: Unexpected character '≤'.
11:52: Unexpected character '`'.
11:56: Unexpected character '`'.
11:89: Unexpected character '≥'.
11:135: Unexpected character '`'.
11:140: Unexpected character '`'.
13:23: Unexpected character '`'.
13:41: Unexpected character '`'.
13:43: Unexpected character '–'.
13:116: Unexpected character '≤'.
13:189: Char literals must contain exactly one character and close with '''.
15:216: Unexpected character '≠'.
15:394: Unexpected character '≠'.
17:48: Unexpected character '–'.
17:113: Unexpected character '≠'.
17:184: Char literals must contain exactly one character and close with '''.
19:326: Unexpected character '→'.
22:37: Unexpected character '≥'.
22:53: Unexpected character '→'.
22:59: Char literals must contain exactly one character and close with '''.
22:445: Char literals must contain exactly one character and close with '''.
23:30: Unexpected character '≥'.
25:481: Unexpected character '→'.
27:80: Unexpected character '⟺'.
29:112: Unexpected character '≥'.
29:217: Unexpected character '→'.
29:222: Unexpected character '²'.
35:143: Char literals must contain exactly one character and close with '''.
37:20: Unexpected character '#'.
39:55: Unexpected character '→'.
39:119: Char literals must contain exactly one character and close with '''.
55:59: Unexpected character '≥'.
55:82: Unexpected character '→'.
61:222: Char literals must contain exactly one character and close with '''.
61:364: Char literals must contain exactly one character and close with '''.
61:457: Char literals must contain exactly one character and close with '''.
67:88: Char literals must contain exactly one character and close with '''.
74:35: Unexpected character '≥'.
74:57: Unexpected character '≥'.
74:65: Unexpected character '→'.
75:39: Unexpected character '≥'.
75:46: Unexpected character '≥'.
75:51: Unexpected character '→'.
75:61: Unexpected character '✓'.
76:7: Unexpected character '≥'.
76:40: Unexpected character '≥'.
76:45: Unexpected character '→'.
78:12: Unexpected character '≥'.
79:14: Unexpected character '✓'.
83:12: Unexpected character '≥'.
84:19: Unexpected character '≥'.
84:26: Unexpected character '≥'.
84:31: Unexpected character '→'.
84:210: Unexpected character '→'.
84:269: Char literals must contain exactly one character and close with '''.
84:374: Char literals must contain exactly one character and close with '''.
84:392: Char literals must contain exactly one character and close with '''.
84:419: Char literals must contain exactly one character and close with '''.
84:478: Char literals must contain exactly one character and close with '''.
84:487: Char literals must contain exactly one character and close with '''.
84:536: Char literals must contain exactly one character and close with '''.
84:614: Char literals must contain exactly one character and close with '''.
85:13: Unexpected character '≥'.
85:52: Unexpected character '≥'.
85:57: Unexpected character '→'.
85:67: Unexpected character '✓'.
88:175: Unexpected character '→'.
88:334: Unexpected character '→'.
88:342: Unexpected character '✓'.
90:25: Unexpected character '→'.
90:32: Unexpected character '✓'.
92:29: Unexpected character '#'.
98:30: Unexpected character '≥'.
98:57: Unexpected character '→'.
100:118: Unexpected character '`'.
100:195: Unexpected character '`'.
102:22: Unexpected character '`'.
102:82: Unexpected character '`'.
106:23: Unexpected character '`'.
106:63: Unexpected character '`'.
114:31: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 281
OUT3 397 |
| × |
LcbMaxBinaryConcatenationLiveCodeBench 3579 / leetcode / medium: maximum possible number by binary concatenation (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxBinaryConcatenation case0: expected 30 got 23 |
Failed |
IN9 168
OUT358 |
| × |
LcbMaxUniqueTowerHeightsLiveCodeBench 3510 / leetcode / medium: maximize the total height of unique towers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxUniqueTowerHeights case0: expected 10 got 12 |
Failed |
IN9 205
OUT244 |
| ✓ |
LcbMinAfterDigitSumLiveCodeBench 3606 / leetcode / easy: minimum element after replacement with digit sum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 161
OUT150 |
| × |
LcbSortedGcdPairQueriesLiveCodeBench 3583 / leetcode / hard: sorted GCD pair queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: sortedGcdPairQueries case0: expected [1, 2, 2] got [1,1,1] |
Failed |
IN9 297
OUT566 |
Code Generation - LiveCodeBench v5 AtCoder (set 1)
3 passed7 failed2 skipped |
30.0% |
IN141 118
OUT22 359 |
| × |
LcbAbaPalindromeLiveCodeBench / atcoder / medium: abc375_d ABA (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AbaPalindrome case0: expected [5] got [4] |
Failed |
IN9 191
OUT126 |
| – |
LcbLaserMarkingLiveCodeBench / atcoder / medium: abc374_d Laser Marking (Pattern B, stdin) [float output, tolerance 1e-6]0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchSkipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped |
IN0
OUT0 |
| × |
LcbOnlyOneProductNameLiveCodeBench / atcoder / hard: abc374_g Only One Product Name (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OnlyOneProductName case1: expected [2] got [3] |
Failed |
IN35 106
OUT15 606 |
| ✓ |
LcbSeatsLiveCodeBench / atcoder / easy: abc375_a Seats (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 184
OUT109 |
| × |
LcbSensorOptimizationLiveCodeBench / atcoder / hard: abc374_e Sensor Optimization Dilemma 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SensorOptimization case1: expected [1000000000] got [10000000] |
Failed |
IN9 280
OUT386 |
| × |
LcbSeparatedLunchLiveCodeBench / atcoder / medium: abc374_c Separated Lunch (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SeparatedLunch case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 201
OUT291 |
| × |
LcbShippingLiveCodeBench / atcoder / hard: abc374_f Shipping (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Shipping case0: expected [2] got [0] |
Failed |
IN9 283
OUT647 |
| × |
LcbSpiralRotationLiveCodeBench / atcoder / medium: abc375_c Spiral Rotation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SpiralRotation case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:13: Unexpected character '×'.
4:29: Unexpected character '≤'.
4:35: Unexpected character '≤'.
4:40: Unexpected character '−'.
4:73: Unexpected character '−'.
6:63: Unexpected character '−'.
6:82: Unexpected character '−'.
6:156: Unexpected character '−'.
6:197: Unexpected character '−'.
6:238: Unexpected character '∈'.
6:247: Unexpected character '−'.
6:257: Char literals must contain exactly one character and close with '''.
8:56: Unexpected character '−'.
8:107: Unexpected character '−'.
8:136: Unexpected character '−'.
10:47: Unexpected character '’'.
10:100: Unexpected character '−'.
10:111: Unexpected character '−'.
10:147: Unexpected character '−'.
10:176: Unexpected character '−'.
10:186: Unexpected character '→'.
10:205: Unexpected character '−'.
10:229: Char literals must contain exactly one character and close with '''.
12:33: Unexpected character '°'.
12:90: Unexpected character '°'.
12:124: Unexpected character '°'.
12:169: Unexpected character '°'.
14:108: Unexpected character '°'.
14:214: Unexpected character '°'.
16:9: Unexpected character '—'.
16:120: Char literals must contain exactly one character and close with '''.
18:87: Unexpected character '−'.
18:96: Unexpected character '−'.
20:85: Unexpected character '°'.
20:226: Unexpected character '−'.
20:302: Unexpected character '≤'.
20:308: Unexpected character '≤'.
20:313: Unexpected character '−'.
22:35: Unexpected character '×'.
22:45: Unexpected character '°'.
22:84: Unexpected character '−'.
22:87: Unexpected character '×'.
22:90: Unexpected character '−'.
22:103: Unexpected character '°'.
24:84: Unexpected character '°'.
24:107: Unexpected character '°'.
24:124: Char literals must contain exactly one character and close with '''.
30:21: Unexpected character '°'.
30:77: Unexpected character '−'.
30:126: Unexpected character '∘'.
30:132: Unexpected character '∘'.
32:61: Char literals must contain exactly one character and close with '''.
34:98: Unexpected character '°'.
36:45: Unexpected character '−'.
36:114: Unexpected character '°'.
36:158: Unexpected character '−'.
36:195: Unexpected character '−'.
36:224: Unexpected character '−'.
36:235: Unexpected character '−'.
36:260: Unexpected character '−'.
36:262: Char literals must contain exactly one character and close with '''.
36:331: Unexpected character '−'.
36:379: Unexpected character '−'.
36:397: Unexpected character '−'.
36:495: Unexpected character '−'.
38:151: Unexpected character '−'.
40:5: Char literals must contain exactly one character and close with '''.
42:93: Unexpected character 'Σ'.
42:154: Unexpected character 'Σ'.
42:226: Unexpected character 'Σ'.
42:232: Unexpected character '−'.
42:241: Unexpected character '≈'.
42:243: Unexpected character 'Σ'.
42:254: Unexpected character '≈'.
42:260: Unexpected character '≈'.
44:234: Unexpected character '≤'.
44:240: Unexpected character '≤'.
44:245: Unexpected character '−'.
44:248: Unexpected character '⇔'.
44:252: Unexpected character '≤'.
44:265: Unexpected character '−'.
44:271: Unexpected character '−'.
44:295: Unexpected character '−'.
44:301: Unexpected character '−'.
44:329: Unexpected character '≤'.
44:425: Unexpected character '≤'.
44:452: Char literals must contain exactly one character and close with '''.
48:99: Unexpected character '°'.
48:299: Unexpected character '±'.
48:302: Unexpected character '°'.
50:39: Unexpected character '≤'.
50:127: Unexpected character '°'.
50:229: Unexpected character '—'.
50:242: Char literals must contain exactly one character and close with '''.
52:50: Unexpected character 'θ'.
52:56: Unexpected character '°'.
52:127: Unexpected character 'θ'.
54:123: Char literals must contain exactly one character and close with '''.
58:42: Unexpected character '°'.
58:80: Unexpected character '°'.
60:2: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 251
OUT3 408 |
| ✓ |
LcbTakahashiSanLiveCodeBench / atcoder / easy: abc374_a Takahashi san 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 153
OUT42 |
| × |
LcbThreeTeamDivisionLiveCodeBench / atcoder / hard: abc375_e 3 Team Division (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ThreeTeamDivision case0: expected [2] got [-1] |
Failed |
IN32 288
OUT1 630 |
| – |
LcbTravelingTakahashiLiveCodeBench / atcoder / easy: abc375_b Traveling Takahashi Problem (Pattern B, stdin) [float output, tolerance 1e-6]0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBenchSkipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped |
IN0
OUT0 |
| ✓ |
LcbUnvarnishedReportLiveCodeBench / atcoder / easy: abc374_b Unvarnished Report (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 181
OUT114 |
Code Generation - LiveCodeBench v5 AtCoder (set 2)
2 passed9 failed |
18.2% |
IN102 102
OUT13 442 |
| × |
LcbAvoidKnightAttackLiveCodeBench / atcoder / medium: abc377_c Avoid Knight Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AvoidKnightAttack case0: expected [38] got [44] |
Failed |
IN9 307
OUT681 |
| ✓ |
LcbAvoidRookAttackLiveCodeBench / atcoder / easy: abc377_b Avoid Rook Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 255
OUT289 |
| ✓ |
LcbCandyButtonLiveCodeBench / atcoder / easy: abc376_a Candy Button (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 223
OUT144 |
| × |
LcbCycleMinEdgesLiveCodeBench / atcoder / medium: abc376_d Cycle (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CycleMinEdges case0: expected [3] got [2] |
Failed |
IN9 245
OUT406 |
| × |
LcbHandsOnRingEasyLiveCodeBench / atcoder / medium: abc376_b Hands on Ring (Easy) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: HandsOnRingEasy case0: expected [8] got [6] |
Failed |
IN9 289
OUT267 |
| × |
LcbHandsOnRingHardLiveCodeBench / atcoder / hard: abc376_f Hands on Ring (Hard) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: HandsOnRingHard case0: expected [6] got [5] |
Failed |
IN9 294
OUT477 |
| × |
LcbMaxTimesSumLiveCodeBench / atcoder / hard: abc376_e Max x Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaxTimesSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:34: Expected '>' to close the List type, found '['.
Values:
run.Compiled = false |
Failed |
IN9 281
OUT504 |
| × |
LcbPrepareAnotherBoxLiveCodeBench / atcoder / medium: abc376_c Prepare Another Box (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PrepareAnotherBox case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
43:51: Cannot assign a value of type 'long' to variable 'best' of type 'int'.
Values:
run.Compiled = false |
Failed |
IN9 278
OUT362 |
| × |
LcbRoadBlockedLiveCodeBench / atcoder / hard: abc375_f Road Blocked (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RoadBlocked case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:32: Unexpected character '≤'.
3:66: Unexpected character '×'.
3:69: Unexpected character '⁵'.
7:24: Unexpected character '≤'.
7:228: Char literals must contain exactly one character and close with '''.
9:22: Unexpected character '≤'.
9:94: Unexpected character '≤'.
9:127: Unexpected character '³'.
9:130: Unexpected character '≈'.
9:135: Unexpected character '×'.
9:138: Unexpected character '⁷'.
9:140: Unexpected character '→'.
9:146: Unexpected character '×'.
9:151: Unexpected character '×'.
9:154: Unexpected character '⁷'.
9:156: Unexpected character '≈'.
9:159: Unexpected character '×'.
9:162: Unexpected character '⁹'.
13:77: Unexpected character '≤'.
13:238: Unexpected character '³'.
15:67: Unexpected character '≤'.
15:880: Unexpected character '²'.
15:883: Unexpected character '≈'.
15:926: Unexpected character '≈'.
15:940: Unexpected character '≈'.
15:976: Unexpected character '×'.
15:979: Unexpected character '⁵'.
15:986: Unexpected character '×'.
15:989: Unexpected character '¹'.
15:990: Unexpected character '⁰'.
17:19: Unexpected character '≤'.
19:266: Unexpected character '∪'.
21:11: Unexpected character '≤'.
21:74: Unexpected character '≤'.
21:149: Unexpected character '≤'.
23:91: Unexpected character '≤'.
23:101: Unexpected character '≤'.
23:488: Unexpected character '→'.
23:619: Unexpected character '∈'.
23:621: Unexpected character '∪'.
23:633: Unexpected character '∈'.
23:635: Unexpected character '∪'.
23:655: Char literals must contain exactly one character and close with '''.
23:666: Char literals must contain exactly one character and close with '''.
23:685: Char literals must contain exactly one character and close with '''.
27:126: Char literals must contain exactly one character and close with '''.
29:128: Unexpected character '≈'.
29:193: Unexpected character '≤'.
29:204: Unexpected character '≈'.
29:235: Unexpected character '≈'.
29:246: Unexpected character '≈'.
29:262: Unexpected character '×'.
31:67: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 304
OUT2 210 |
| × |
LcbRoadBlocked2LiveCodeBench / atcoder / hard: abc375_g Road Blocked 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RoadBlocked2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:91: Char literals must contain exactly one character and close with '''.
13:132: Unexpected character '#'.
13:199: Unexpected character '#'.
13:349: Char literals must contain exactly one character and close with '''.
15:9: Unexpected character '—'.
19:19: Char literals must contain exactly one character and close with '''.
21:42: Unexpected character '→'.
23:49: Unexpected character '→'.
23:370: Unexpected character '→'.
23:380: Unexpected character '—'.
23:455: Char literals must contain exactly one character and close with '''.
25:71: Unexpected character '→'.
25:146: Unexpected character '→'.
25:169: Unexpected character '→'.
29:57: Unexpected character '→'.
33:129: Unexpected character '→'.
41:52: Char literals must contain exactly one character and close with '''.
63:13: Unexpected character '#'.
67:2: Char literals must contain exactly one character and close with '''.
69:161: Char literals must contain exactly one character and close with '''.
89:163: Char literals must contain exactly one character and close with '''.
117:4: Char literals must contain exactly one character and close with '''.
123:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 293
OUT4 675 |
| × |
LcbTreasureHuntingLiveCodeBench / atcoder / hard: abc376_g Treasure Hunting (Pattern B, stdin) [answer modulo 998244353]1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TreasureHunting case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:310: Unexpected character '#'.
3:381: Char literals must contain exactly one character and close with '''.
5:18: Char literals must contain exactly one character and close with '''.
7:296: Char literals must contain exactly one character and close with '''.
7:436: Char literals must contain exactly one character and close with '''.
11:256: Char literals must contain exactly one character and close with '''.
15:111: Char literals must contain exactly one character and close with '''.
15:223: Char literals must contain exactly one character and close with '''.
15:483: Unexpected character '—'.
15:834: Char literals must contain exactly one character and close with '''.
21:600: Char literals must contain exactly one character and close with '''.
23:44: Char literals must contain exactly one character and close with '''.
23:495: Unexpected character '—'.
35:17: Unexpected character '#'.
37:34: Char literals must contain exactly one character and close with '''.
39:97: Char literals must contain exactly one character and close with '''.
39:113: Char literals must contain exactly one character and close with '''.
47:4: Char literals must contain exactly one character and close with '''.
49:64: Char literals must contain exactly one character and close with '''.
55:2: Char literals must contain exactly one character and close with '''.
77:71: Unexpected character '—'.
79:4: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 333
OUT3 427 |
Code Generation - LiveCodeBench v5 AtCoder (set 3)
4 passed10 failed |
28.6% |
IN151 534
OUT12 438 |
| × |
LcbAddOneEdge2LiveCodeBench / atcoder abc378_f: Add One Edge 2 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AddOneEdge2 case0: expected [1] got [0] |
Failed |
IN9 176
OUT1 732 |
| × |
LcbAvoidQueenAttackLiveCodeBench / atcoder abc377_f: Avoid Queen Attack (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AvoidQueenAttack case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:13: Unexpected character '×'.
3:132: Unexpected character '−'.
6:73: Unexpected character '−'.
7:34: Char literals must contain exactly one character and close with '''.
7:129: Unexpected character '→'.
7:134: Char literals must contain exactly one character and close with '''.
9:190: Char literals must contain exactly one character and close with '''.
11:76: Unexpected character '²'.
11:191: Unexpected character '²'.
13:13: Char literals must contain exactly one character and close with '''.
15:258: Unexpected character '²'.
15:330: Unexpected character '—'.
17:45: Unexpected character '²'.
17:197: Unexpected character '≤'.
17:208: Unexpected character '²'.
17:275: Unexpected character '≥'.
17:287: Unexpected character '≥'.
17:317: Unexpected character '²'.
19:21: Char literals must contain exactly one character and close with '''.
25:2: Char literals must contain exactly one character and close with '''.
25:230: Unexpected character '—'.
25:239: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 247
OUT1 409 |
| × |
LcbCountSimplePathsLiveCodeBench / atcoder abc378_d: Count Simple Paths (hard)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountSimplePaths case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
39:30: Unknown function 'Dfs'.
Values:
run.Compiled = false |
Failed |
IN20 078
OUT3 087 |
| × |
LcbEditToMatchLiveCodeBench / atcoder abc377_g: Edit to Match (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: EditToMatch case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:28: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
run.Compiled = false |
Failed |
IN9 250
OUT381 |
| × |
LcbEverlastingLidsLiveCodeBench / atcoder abc378_g: Everlasting LIDS (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: EverlastingLids case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:42: Char literals must contain exactly one character and close with '''.
3:57: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:63: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:90: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:91: Unexpected character '\'.
3:98: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:106: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:108: Unexpected character '\'.
3:121: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:178: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:179: Unexpected character '\'.
3:190: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:193: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
3:194: Unexpected character '\'.
3:201: Char literals must contain exactly one character and close with '''.
5:24: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:30: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:37: Char literals must contain exactly one character and close with '''.
7:15: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:23: Unexpected character '\'.
7:28: Unexpected character '\'.
7:38: Unexpected character '\'.
7:49: Unexpected character '\'.
7:56: Char literals must contain exactly one character and close with '''.
7:156: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:25: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:26: Unexpected character '\'.
9:37: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:43: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:44: Unexpected character '\'.
9:51: Char literals must contain exactly one character and close with '''.
9:267: Unexpected character '₁'.
9:273: Unexpected character '✓'.
11:111: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:114: Unexpected character '\'.
11:121: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:174: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:179: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:234: Char literals must contain exactly one character and close with '''.
13:141: Unexpected character '×'.
13:152: Unexpected character 'λ'.
13:153: Unexpected character '₁'.
13:154: Unexpected character '≥'.
13:156: Unexpected character '−'.
13:168: Unexpected character 'λ'.
13:169: Unexpected character '₁'.
13:170: Unexpected character '≤'.
13:177: Unexpected character '≤'.
13:186: Unexpected character 'λ'.
13:187: Unexpected character '₁'.
13:203: Unexpected character 'λ'.
13:204: Char literals must contain exactly one character and close with '''.
15:76: Unexpected character '×'.
15:123: Unexpected character 'λ'.
15:210: Char literals must contain exactly one character and close with '''.
17:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 227
OUT676 |
| ✓ |
LcbGarbageCollectionLiveCodeBench / atcoder abc378_b: Garbage Collection (easy)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN21 066
OUT3 438 |
| × |
LcbManySegments2LiveCodeBench / atcoder abc377_d: Many Segments 2 (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ManySegments2 case0: expected [5] got [6] |
Failed |
IN9 207
OUT379 |
| × |
LcbModSigmaProblemLiveCodeBench / atcoder abc378_e: Mod Sigma Problem (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ModSigmaProblem case1: expected [2736] got [2312] |
Failed |
IN9 183
OUT309 |
| ✓ |
LcbPairingLiveCodeBench / atcoder abc378_a: Pairing (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 168
OUT184 |
| × |
LcbPermuteKTimesLiveCodeBench / atcoder abc377_e: Permute K times 2 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: PermuteKTimes case1 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 203
OUT278 |
| ✓ |
LcbRearrangingAbcLiveCodeBench / atcoder abc377_a: Rearranging ABC (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 164
OUT147 |
| × |
LcbRepeatingLiveCodeBench / atcoder abc378_c: Repeating (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Repeating case0: expected [-1 -1 1 3 -1] got [-1 -1 1 1 -1] |
Failed |
IN9 185
OUT151 |
| ✓ |
LcbStrawberriesLiveCodeBench / atcoder abc379_b: Strawberries (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 188
OUT161 |
| × |
LcbSumOfAllSubstringsLiveCodeBench / atcoder abc379_e: Sum of All Substrings (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SumOfAllSubstrings case0: expected [514] got [419] |
Failed |
IN9 192
OUT106 |
Code Generation - LiveCodeBench v5 AtCoder (set 4)
5 passed9 failed |
35.7% |
IN156 944
OUT18 347 |
| ✓ |
Lcb1122StringLiveCodeBench / atcoder abc381_a: 11/22 String (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 273
OUT161 |
| × |
Lcb1122SubsequenceLiveCodeBench / atcoder abc381_e: 11/22 Subsequence (hard)1 AI query
3 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchExternal function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed |
IN37 201
OUT1 918 |
| × |
LcbAnotherShuffleWindowLiveCodeBench / atcoder abc380_g: Another Shuffle Window (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AnotherShuffleWindow case0: expected [166374061] got [499122181] |
Failed |
IN9 275
OUT598 |
| × |
LcbBucketToolLiveCodeBench / atcoder abc380_e: 1D Bucket Tool (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchExternal function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed |
IN9 211
OUT3 167 |
| × |
LcbBuildings2LiveCodeBench / atcoder abc379_f: Buildings 2 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Buildings2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:20: Unexpected character '→'.
4:140: Unexpected character '∈'.
6:79: Unexpected character '≤'.
6:84: Unexpected character '…'.
6:205: Unexpected character '∈'.
8:21: Unexpected character '⟺'.
8:32: Unexpected character '≤'.
8:60: Unexpected character '⟺'.
8:71: Unexpected character '≤'.
8:131: Unexpected character '—'.
8:281: Unexpected character '≤'.
8:296: Unexpected character '≤'.
8:314: Unexpected character '⊆'.
8:357: Unexpected character '≥'.
8:373: Unexpected character '≤'.
8:392: Unexpected character '≤'.
8:426: Unexpected character '≤'.
8:433: Unexpected character '≤'.
8:524: Unexpected character '≤'.
8:578: Unexpected character '≤'.
10:8: Unexpected character '—'.
10:281: Unexpected character '—'.
10:466: Unexpected character '—'.
12:15: Unexpected character '#'.
12:44: Unexpected character '≤'.
12:137: Unexpected character '…'.
12:300: Unexpected character '→'.
12:375: Unexpected character '—'.
12:382: Char literals must contain exactly one character and close with '''.
14:49: Unexpected character '→'.
14:214: Unexpected character '≥'.
16:48: Unexpected character '≤'.
16:209: Unexpected character '≤'.
16:215: Unexpected character '⇔'.
16:265: Unexpected character '⇔'.
16:315: Unexpected character '≤'.
16:391: Unexpected character '⇔'.
16:408: Unexpected character '≤'.
16:450: Unexpected character '#'.
16:462: Unexpected character '≤'.
16:485: Unexpected character '≤'.
18:100: Unexpected character '≤'.
18:245: Unexpected character '≤'.
18:307: Char literals must contain exactly one character and close with '''.
22:47: Unexpected character '→'.
22:61: Unexpected character '≥'.
22:72: Unexpected character '≥'.
22:87: Unexpected character '—'.
22:92: Char literals must contain exactly one character and close with '''.
28:53: Unexpected character '≤'.
34:60: Unexpected character '≤'.
36:17: Unexpected character '#'.
36:79: Unexpected character '—'.
36:175: Char literals must contain exactly one character and close with '''.
38:4: Char literals must contain exactly one character and close with '''.
52:8: Unexpected character '≥'.
58:19: Unexpected character '\'.
66:29: Unexpected character '—'.
70:4: Char literals must contain exactly one character and close with '''.
172:6: Unexpected character '`'.
172:11: Unexpected character '`'.
172:31: Unexpected character '`'.
172:44: Unexpected character '`'.
Values:
run.Compiled = false |
Failed |
IN9 221
OUT3 659 |
| × |
LcbCountGrid3LiveCodeBench / atcoder abc379_g: Count Grid 3 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountGrid3 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:12: Expected '(' to declare a function or '=' to declare a global variable after 'Looking at', found 'this'. Global variables must have an initializer.
Values:
run.Compiled = false |
Failed |
IN9 203
OUT4 241 |
| ✓ |
LcbCyclicLiveCodeBench / atcoder abc379_a: Cyclic (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 177
OUT116 |
| ✓ |
LcbDigitPattern123233LiveCodeBench / atcoder abc380_a: 123233 (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 164
OUT143 |
| × |
LcbExchangeGameLiveCodeBench / atcoder abc380_f: Exchange Game (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: ExchangeGame case0 runtime error: List index -1 is out of range. The list has 1 element(s).
Values:
run.Success = false |
Failed |
IN9 255
OUT443 |
| × |
LcbHomeGardenLiveCodeBench / atcoder abc379_d: Home Garden (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: HomeGarden case0: expected [1
1] got [0
2] |
Failed |
IN9 191
OUT367 |
| ✓ |
LcbHurdleParsingLiveCodeBench / atcoder abc380_b: Hurdle Parsing (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT171 |
| × |
LcbMoveSegmentLiveCodeBench / atcoder abc380_c: Move Segment (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MoveSegment case0: expected [010011111000001] got [01001110000111] |
Failed |
IN9 186
OUT391 |
| ✓ |
LcbSowingStonesLiveCodeBench / atcoder abc379_c: Sowing Stones (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 218
OUT304 |
| × |
LcbStrangeMirroringLiveCodeBench / atcoder abc380_d: Strange Mirroring (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: StrangeMirroring case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
15:17: Char literals must contain exactly one character and close with '''.
35:135: Char literals must contain exactly one character and close with '''.
47:48: Char literals must contain exactly one character and close with '''.
51:10: Char literals must contain exactly one character and close with '''.
53:128: Unexpected character '`'.
53:134: Unexpected character '`'.
55:122: Unexpected character '—'.
59:168: Unexpected character '→'.
61:56: Unexpected character '—'.
67:86: Char literals must contain exactly one character and close with '''.
69:17: Char literals must contain exactly one character and close with '''.
73:237: Unexpected character '→'.
73:290: Char literals must contain exactly one character and close with '''.
79:176: Unexpected character '≈'.
79:186: Unexpected character '≈'.
83:47: Char literals must contain exactly one character and close with '''.
86:70: Unexpected character '÷'.
88:33: Unexpected character '→'.
88:99: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 194
OUT2 668 |
Code Generation - LiveCodeBench v5 AtCoder (set 5)
6 passed7 failed |
46.2% |
IN132 253
OUT7 126 |
| ✓ |
Lcb1122StringLiveCodeBench / atcoder / easy: 1122 String (abc381_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 256
OUT217 |
| × |
Lcb1122SubarrayLiveCodeBench / atcoder / hard: 1122 Substring (abc381_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: 1122Subarray case0 runtime error: List index 8 is out of range. The list has 8 element(s).
Values:
run.Success = false |
Failed |
IN9 296
OUT532 |
| ✓ |
LcbDailyCookieLiveCodeBench / atcoder / easy: Daily Cookie (abc382_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 262
OUT108 |
| ✓ |
LcbDailyCookie2LiveCodeBench / atcoder / easy: Daily Cookie 2 (abc382_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 284
OUT215 |
| × |
LcbFallingBarsLiveCodeBench / atcoder / hard: Falling Bars (abc382_f)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: FallingBars case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
52:27: Type 'List<int>' has no supported method named 'Remove' with 1 argument(s).
Values:
run.Compiled = false |
Failed |
IN20 535
OUT786 |
| ✓ |
LcbHumidifier1LiveCodeBench / atcoder / easy: Humidifier 1 (abc383_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 255
OUT146 |
| ✓ |
LcbHumidifier2LiveCodeBench / atcoder / medium: Humidifier 2 (abc383_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 319
OUT522 |
| ✓ |
LcbHumidifier3LiveCodeBench / atcoder / medium: Humidifier 3 (abc383_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 309
OUT999 |
| × |
LcbKaitenSushiLiveCodeBench / atcoder / medium: Kaiten Sushi (abc382_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: KaitenSushi case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:56: List constructor expects a List or HashSet to copy, but got 'int'.
30:44: List constructor expects a List or HashSet to copy, but got 'int'.
Values:
run.Compiled = false |
Failed |
IN9 315
OUT530 |
| × |
LcbKeepDistanceLiveCodeBench / atcoder / hard: Keep Distance (abc382_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchExternal function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed |
IN9 278
OUT349 |
| × |
LcbMax1122SubstringLiveCodeBench / atcoder / medium: 11/22 Substring (abc381_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Max1122Substring case2: expected [7] got [1] |
Failed |
IN9 381
OUT211 |
| × |
LcbSumOfMaxMatchingLiveCodeBench / atcoder / hard: Sum of Max Matching (abc383_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: SumOfMaxMatching case0 runtime error: The dictionary does not contain the key 4. Use ContainsKey to check for it first.
Values:
run.Success = false |
Failed |
IN9 367
OUT1 390 |
| × |
LcbTileDistance3LiveCodeBench / atcoder / hard: Tile Distance 3 (abc382_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TileDistance3 case0: expected [4
4
800000000000] got [3
4
800000000000] |
Failed |
IN9 396
OUT1 121 |
Code Generation - LiveCodeBench v5 AtCoder (set 6)
4 passed9 failed1 skipped |
30.8% |
IN325 851
OUT18 272 |
| × |
LcbAbsSumLiveCodeBench / atcoder / hard: Abs Sum (abc384_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AbsSum case1: expected [13331322
2209746
6366712
207690
20241215] got [10010877
6683134
4468958
1708516
20241215] |
Failed |
IN9 316
OUT587 |
| ✓ |
LcbArcDivisionLiveCodeBench / atcoder / easy: ARC Division (abc384_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 363
OUT207 |
| ✓ |
LcbDoubleSum2LiveCodeBench / atcoder / hard: Double Sum 2 (abc384_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 253
OUT186 |
| × |
LcbEquallyLiveCodeBench / atcoder / easy: Equally (abc385_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Equally case0: expected [Yes] got [No] |
Failed |
IN9 186
OUT147 |
| × |
LcbIlluminateBuildingsLiveCodeBench / atcoder / medium: Illuminate Buildings (abc385_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: IlluminateBuildings case0: expected [3] got [5] |
Failed |
IN9 226
OUT388 |
| × |
LcbMaximizeXorLiveCodeBench / atcoder / hard: Maximize XOR (abc386_e)1 AI query
9 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaximizeXor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:36: Expected ')' to close the argument list, found 'best'.
Values:
run.Compiled = false |
Failed |
IN166 092
OUT8 853 |
| ✓ |
LcbPerfectStandingsLiveCodeBench / atcoder / medium: Perfect Standings (abc384_c)1 AI query
4 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN57 378
OUT2 233 |
| × |
LcbRepeatedSequenceLiveCodeBench / atcoder / medium: Repeated Sequence (abc384_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RepeatedSequence case0: expected [Yes] got [No] |
Failed |
IN9 248
OUT633 |
| ✓ |
LcbReplaceExceptCharLiveCodeBench / atcoder / easy: aaaadaa (abc384_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 204
OUT149 |
| × |
LcbSantaClaus1LiveCodeBench / atcoder / easy: Santa Claus 1 (abc385_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SantaClaus1 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:47: Expected ';' after the variable declaration, found 'y'.
Values:
run.Compiled = false |
Failed |
IN9 469
OUT433 |
| × |
LcbSantaClaus2LiveCodeBench / atcoder / hard: Santa Claus 2 (abc385_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SantaClaus2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
89:52: Expected ')' to close the argument list, found '='.
Values:
run.Compiled = false |
Failed |
IN9 444
OUT1 648 |
| × |
LcbSlime2LiveCodeBench / atcoder / hard: Takahashi is Slime 2 (abc384_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Slime2 case0: expected [28] got [13] |
Failed |
IN9 378
OUT485 |
| × |
LcbSnowflakeTreeLiveCodeBench / atcoder / hard: Snowflake Tree (abc385_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SnowflakeTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
12:106: Char literals must contain exactly one character and close with '''.
14:15: Char literals must contain exactly one character and close with '''.
18:195: Char literals must contain exactly one character and close with '''.
20:238: Unexpected character '—'.
20:340: Char literals must contain exactly one character and close with '''.
24:107: Unexpected character '—'.
26:145: Char literals must contain exactly one character and close with '''.
27:4: Char literals must contain exactly one character and close with '''.
27:410: Unexpected character '≥'.
27:425: Unexpected character '≥'.
27:467: Unexpected character '≥'.
27:521: Unexpected character '≥'.
33:50: Unexpected character '—'.
33:249: Char literals must contain exactly one character and close with '''.
33:338: Char literals must contain exactly one character and close with '''.
33:425: Unexpected character '—'.
35:150: Char literals must contain exactly one character and close with '''.
35:293: Char literals must contain exactly one character and close with '''.
35:450: Char literals must contain exactly one character and close with '''.
35:627: Unexpected character '∪'.
35:631: Unexpected character '∪'.
35:717: Char literals must contain exactly one character and close with '''.
37:29: Unexpected character '⊆'.
37:59: Unexpected character '∈'.
37:79: Unexpected character '∈'.
37:111: Unexpected character '≤'.
37:160: Unexpected character '≥'.
37:188: Unexpected character 'Σ'.
41:57: Unexpected character '≤'.
41:61: Unexpected character '≤'.
41:69: Unexpected character '∈'.
41:163: Unexpected character '∈'.
43:42: Unexpected character '∈'.
43:108: Unexpected character '≥'.
43:138: Unexpected character '≥'.
43:184: Unexpected character '≥'.
43:194: Unexpected character '≥'.
43:197: Unexpected character '⇒'.
43:207: Unexpected character '≥'.
43:238: Unexpected character '≥'.
45:70: Unexpected character '≥'.
47:152: Unexpected character '≥'.
47:208: Unexpected character '≥'.
47:259: Unexpected character '≥'.
47:285: Unexpected character '—'.
47:314: Unexpected character '≥'.
47:489: Unexpected character '#'.
47:508: Unexpected character '≥'.
47:618: Unexpected character '≥'.
47:637: Unexpected character '≥'.
47:666: Unexpected character '≥'.
49:37: Unexpected character '≤'.
49:40: Unexpected character '≤'.
49:41: Unexpected character '…'.
49:42: Unexpected character '≤'.
49:49: Unexpected character '#'.
49:66: Unexpected character '≥'.
49:138: Unexpected character '…'.
49:270: Unexpected character '≥'.
49:384: Unexpected character '≥'.
53:14: Unexpected character '≥'.
53:29: Unexpected character '≥'.
53:34: Unexpected character '≥'.
53:49: Unexpected character '≥'.
57:52: Char literals must contain exactly one character and close with '''.
57:207: Char literals must contain exactly one character and close with '''.
59:49: Unexpected character '≥'.
61:99: Unexpected character '≥'.
61:102: Unexpected character '→'.
63:156: Unexpected character '≥'.
Values:
run.Compiled = false |
Failed |
IN9 294
OUT2 323 |
| – |
LcbVisibleBuildingsLiveCodeBench / atcoder / hard: Visible Buildings (abc385_f)0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped |
IN0
OUT0 |
Code Generation - LiveCodeBench v5 AtCoder (set 7)
1 passed11 failed1 skipped |
8.3% |
IN193 302
OUT35 227 |
| × |
LcbAccumulatingManyTimesLiveCodeBench / atcoder / hard: arc184_e Accumulating Many Times (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AccumulatingManyTimes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:16: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:18: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:44: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:48: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:55: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:57: Unexpected character '\'.
4:75: Unexpected character '\'.
4:81: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:115: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:126: Unexpected character '\'.
4:146: Unexpected character '\'.
4:153: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
6:4: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
6:26: Unexpected character '\'.
6:39: Unexpected character '\'.
6:46: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:11: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:37: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:51: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:77: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:92: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:103: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:109: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:111: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:135: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
8:163: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:12: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:14: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:31: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
10:43: Unexpected character '\'.
10:65: Unexpected character '\'.
10:78: Unexpected character '\'.
10:97: Unexpected character '\'.
10:122: Unexpected character '\'.
10:129: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
12:60: Char literals must contain exactly one character and close with '''.
14:7: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:9: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:36: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:64: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:122: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:124: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:159: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:161: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:192: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:196: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:212: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:221: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:227: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
14:241: Unexpected character '\'.
14:247: Unex ... (message truncated) |
Failed |
IN9 291
OUT5 613 |
| × |
LcbAdjacentGcdLiveCodeBench / atcoder / hard: arc185_e Adjacent GCD (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AdjacentGcd case0: expected [0
3
11] got [0
5
17] |
Failed |
IN22 240
OUT3 140 |
| ✓ |
LcbCalculatorMinPressesLiveCodeBench / atcoder / easy: abc386_b Calculator (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 208
OUT182 |
| × |
LcbDiagonalSeparationLiveCodeBench / atcoder / hard: abc386_d Diagonal Separation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DiagonalSeparation case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:13: Unexpected character '×'.
3:120: Unexpected character '≤'.
3:229: Unexpected character '≤'.
5:30: Unexpected character '≤'.
5:42: Unexpected character '≤'.
5:101: Unexpected character '⟺'.
5:106: Unexpected character '≤'.
5:113: Unexpected character '⟺'.
5:118: Unexpected character '≤'.
9:126: Unexpected character '≤'.
9:132: Unexpected character '⟺'.
9:136: Unexpected character '≤'.
11:31: Unexpected character '≤'.
11:43: Unexpected character '≤'.
13:255: Char literals must contain exactly one character and close with '''.
15:27: Unexpected character '∈'.
15:51: Unexpected character '≤'.
15:61: Unexpected character '≤'.
15:88: Unexpected character '≤'.
15:116: Unexpected character '≤'.
15:165: Unexpected character '≤'.
15:172: Unexpected character '⇔'.
15:177: Unexpected character '≤'.
17:22: Unexpected character '≤'.
17:63: Unexpected character '≤'.
19:22: Unexpected character '≤'.
19:67: Unexpected character '≤'.
19:241: Unexpected character '∈'.
19:253: Char literals must contain exactly one character and close with '''.
19:300: Unexpected character '∈'.
19:310: Char literals must contain exactly one character and close with '''.
19:418: Unexpected character '∈'.
19:428: Char literals must contain exactly one character and close with '''.
19:433: Unexpected character '∈'.
19:446: Char literals must contain exactly one character and close with '''.
19:453: Unexpected character '≤'.
19:526: Unexpected character '∈'.
19:562: Unexpected character '∈'.
21:71: Unexpected character '²'.
21:94: Unexpected character '≤'.
21:116: Unexpected character '#'.
21:127: Unexpected character '∈'.
21:143: Unexpected character '—'.
21:221: Unexpected character '∈'.
21:233: Char literals must contain exactly one character and close with '''.
21:416: Unexpected character '∩'.
21:444: Unexpected character '∈'.
21:452: Unexpected character '≤'.
23:13: Unexpected character '⟺'.
23:168: Unexpected character '²'.
25:236: Char literals must contain exactly one character and close with '''.
25:271: Char literals must contain exactly one character and close with '''.
25:299: Unexpected character '≤'.
25:391: Unexpected character '≤'.
25:396: Unexpected character '⇔'.
25:399: Unexpected character '≤'.
25:438: Unexpected character '≤'.
25:457: Unexpected character '≥'.
25:596: Unexpected character '≥'.
25:597: Unexpected character '…'.
25:598: Unexpected character '≥'.
25:609: Unexpected character '≤'.
25:613: Unexpected character '≤'.
25:689: Unexpected character '—'.
25:817: Unexpected character '≤'.
25:932: Unexpected character '∉'.
27:9: Unexpected character '—'.
27:188: Unexpected character '#'.
27:204: Unexpected character '≥'.
27:224: Unexpected character '≥'.
27:264: Unexpected character '∈'.
27:267: Unexpected character '⟺'.
27:272: Unexpected character '≥'.
27:275: Unexpected character '⟺'.
27:278: Unexpected character '≤'.
27:324: Unexpected character '≥'.
27:439: Unexpected character '∈'.
28:35: Unexpected character '≥'.
29:49: Unexpected character '≤'.
31:168: Unexpected character '≤'.
31:239: Unexpected character '∈'.
33:134: Unexpected character '≥'.
33:140: Unexpected character '≥'.
33:142: Unexpected character '…'.
33:144: Unexpected character '≥'.
33:224: Unexpected character '≤'.
33:238: Unexpected character '≤'.
33:247: Unexpected character '≥'.
33:290: Unexpected character '∞'.
33:347: Unexpected character '→'.
39:113: Char literals must contain exactly one character and close with '''.
39:462: Unexpected character '≤'.
39:477: Unexpected character '≥'.
Values:
run.Compiled = false |
Failed |
IN9 287
OUT2 153 |
| × |
LcbEraseBalls2DLiveCodeBench / atcoder / hard: arc184_d Erase Balls 2D (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: EraseBalls2D case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
17:72: Unexpected character '→'.
17:109: Unexpected character '→'.
17:195: Unexpected character '→'.
17:216: Char literals must contain exactly one character and close with '''.
17:280: Unexpected character '—'.
19:42: Unexpected character '→'.
19:116: Unexpected character '→'.
19:130: Unexpected character '→'.
25:92: Unexpected character '—'.
25:191: Unexpected character '∪'.
27:2: Char literals must contain exactly one character and close with '''.
29:36: Char literals must contain exactly one character and close with '''.
31:2: Char literals must contain exactly one character and close with '''.
33:65: Unexpected character '∅'.
33:77: Unexpected character '→'.
33:80: Unexpected character '✓'.
33:105: Unexpected character '∅'.
33:124: Unexpected character '→'.
33:127: Unexpected character '✗'.
37:48: Char literals must contain exactly one character and close with '''.
39:21: Char literals must contain exactly one character and close with '''.
39:101: Char literals must contain exactly one character and close with '''.
43:103: Unexpected character '→'.
43:116: Unexpected character '→'.
43:143: Char literals must contain exactly one character and close with '''.
45:32: Char literals must contain exactly one character and close with '''.
47:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 253
OUT1 857 |
| × |
LcbFullHouse2LiveCodeBench / atcoder / easy: abc386_a Full House 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FullHouse2 case1: expected [No] got [Yes] |
Failed |
IN9 202
OUT407 |
| × |
LcbMountainValleyFoldsLiveCodeBench / atcoder / hard: arc184_c Mountain and Valley Folds (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MountainValleyFolds case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 311
OUT311 |
| × |
LcbOperate1LiveCodeBench / atcoder / medium: abc386_c Operate 1 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Operate1 case3: expected [Yes] got [No] |
Failed |
IN9 211
OUT452 |
| × |
LcbOperateKLiveCodeBench / atcoder / hard: abc386_f Operate K (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OperateK case2: expected [Yes] got [No] |
Failed |
IN31 505
OUT518 |
| × |
LcbPlusOneAndLiveCodeBench / atcoder / hard: arc185_b +1 and (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PlusOneAnd case0: expected [Yes
No
Yes] got [Yes
Yes
Yes] |
Failed |
IN9 232
OUT226 |
| × |
LcbRandomWalkOnTreeLiveCodeBench / atcoder / hard: arc185_d Random Walk on Tree (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RandomWalkOnTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:57: Unexpected character '≤'.
5:61: Unexpected character '≤'.
12:30: Unexpected character '→'.
13:30: Unexpected character '→'.
28:60: Unexpected character '≤'.
32:6: Char literals must contain exactly one character and close with '''.
34:48: Char literals must contain exactly one character and close with '''.
44:99: Char literals must contain exactly one character and close with '''.
48:17: Unexpected character '#'.
50:34: Char literals must contain exactly one character and close with '''.
56:95: Char literals must contain exactly one character and close with '''.
58:93: Unexpected character '≥'.
58:196: Unexpected character '≥'.
60:23: Char literals must contain exactly one character and close with '''.
60:144: Char literals must contain exactly one character and close with '''.
81:163: Unexpected character '→'.
81:165: Unexpected character '→'.
82:94: Unexpected character '→'.
82:143: Char literals must contain exactly one character and close with '''.
83:89: Unexpected character '→'.
83:104: Char literals must contain exactly one character and close with '''.
86:7: Unexpected character '→'.
86:16: Unexpected character '→'.
87:7: Unexpected character '→'.
87:9: Unexpected character '→'.
87:11: Unexpected character '→'.
87:13: Unexpected character '→'.
88:7: Unexpected character '→'.
88:9: Unexpected character '→'.
88:11: Unexpected character '→'.
88:13: Unexpected character '→'.
88:15: Unexpected character '→'.
88:17: Unexpected character '→'.
89:11: Unexpected character '≥'.
89:130: Unexpected character '≥'.
102:66: Unexpected character '→'.
102:68: Unexpected character '→'.
102:70: Unexpected character '→'.
103:34: Unexpected character '#'.
105:11: Unexpected character '→'.
105:22: Unexpected character '→'.
105:33: Unexpected character '→'.
107:263: Unexpected character '≠'.
107:289: Unexpected character '≠'.
109:76: Unexpected character '∈'.
109:329: Unexpected character '∈'.
109:426: Unexpected character '≤'.
109:501: Unexpected character '±'.
109:528: Char literals must contain exactly one character and close with '''.
110:47: Unexpected character '≤'.
110:54: Unexpected character '∈'.
110:190: Unexpected character '—'.
110:252: Char literals must contain exactly one character and close with '''.
110:569: Char literals must contain exactly one character and close with '''.
110:655: Char literals must contain exactly one character and close with '''.
111:40: Char literals must contain exactly one character and close with '''.
116:14: Unexpected character '∈'.
116:27: Unexpected character '≤'.
116:146: Unexpected character '∈'.
118:37: Unexpected character '→'.
118:39: Char literals must contain exactly one character and close with '''.
119:29: Char literals must contain exactly one character and close with '''.
121:99: Unsupported numeric literal suffix 'E'. SC# supports 'm' for decimal literals and 'L' for long integer literals.
121:114: Unsupported numeric literal suffix 'E'. SC# supports 'm' for decimal literals and 'L' for long integer literals.
121:125: Unsupported numeric literal suffix 'E'. SC# supports 'm' for decimal literals and 'L' for long integer literals.
121:133: Unexpected character '⇒'.
121:134: Unsupported numeric literal suffix 'E'. SC# supports 'm' for decimal literals and 'L' for long integer literals.
121:144: Unexpected character '⇒'.
122:11: Unexpected character '∈'.
123:25: Unexpected character '→'.
123:27: Char literals must contain exactly one character and close with '''.
123:58: Char literals must contain exactly one character and close with '''.
123:261: Unexpected character '⇒'.
123:264: Char literals must contain exactly one character and close with '''.
125:21: Char literals must contain exactly one character and close with '''.
128:6: Unexpected character '∈'.
129:23: Unexpected character '→'.
129:25: Char literals must contain exactly one character and close with '''.
129:47: Unexpected character '⇒'.
129: ... (message truncated) |
Failed |
IN9 282
OUT5 608 |
| – |
LcbSumOfThreeIntegersLiveCodeBench / atcoder / hard: arc185_c Sum of Three Integers (Pattern B, stdin) NOTE: multiple valid triples may exist; exact-match oracle may reject a different valid answer.0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbUndercluedLiveCodeBench / atcoder / hard: arc186_a Underclued (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: Underclued case1 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN56 280
OUT14 760 |
Code Generation - LiveCodeBench v5 AtCoder (set 8)
2 passed10 failed1 skipped |
16.7% |
IN171 462
OUT30 843 |
| × |
LcbAbcSymmetryLiveCodeBench / atcoder / hard: arc188_a ABC Symmetry (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: AbcSymmetry case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:17: Expected ';' after the variable declaration, found ','.
Values:
run.Compiled = false |
Failed |
IN20 198
OUT2 920 |
| × |
LcbBallAndBoxLiveCodeBench / atcoder / hard: arc186_c Ball and Box (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BallAndBox case0: expected [2
0
28] got [10
299999999999999
1686] |
Failed |
IN9 288
OUT392 |
| ✓ |
LcbBallsAndBoxesLiveCodeBench / atcoder / hard: arc189_c Balls and Boxes (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN21 547
OUT3 600 |
| – |
LcbHonestLiarConfusedLiveCodeBench / atcoder / hard: arc188_c Honest or Liar or Confused (Pattern B, stdin) NOTE: multiple valid confused sets may exist; exact-match oracle may reject a different valid answer.0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbMinimizeSumLiveCodeBench / atcoder / hard: arc189_b Minimize Sum (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MinimizeSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN35 365
OUT3 366 |
| × |
LcbMirrorAndOrderLiveCodeBench / atcoder / hard: arc188_d Mirror and Order (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MirrorAndOrder case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:42: Char literals must contain exactly one character and close with '''.
3:100: Unexpected character '∈'.
3:148: Unexpected character '∈'.
3:229: Unexpected character '×'.
3:292: Char literals must contain exactly one character and close with '''.
11:114: Unexpected character '≠'.
11:437: Unexpected character '≠'.
11:508: Unexpected character '→'.
13:98: Unexpected character '≤'.
13:147: Char literals must contain exactly one character and close with '''.
15:89: Char literals must contain exactly one character and close with '''.
17:96: Char literals must contain exactly one character and close with '''.
19:39: Char literals must contain exactly one character and close with '''.
21:52: Unexpected character '≤'.
21:58: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 322
OUT1 867 |
| × |
LcbMissingSubsequenceLiveCodeBench / atcoder / hard: arc186_e Missing Subsequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MissingSubsequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:67: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:69: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:81: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:83: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:99: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:100: Unexpected character '\'.
1:104: Unexpected character '\'.
1:112: Unexpected character '\'.
1:114: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:158: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:160: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:167: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:168: Unexpected character '\'.
1:172: Unexpected character '\'.
1:180: Unexpected character '\'.
1:182: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:213: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:215: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:228: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
1:234: Unexpected character '\'.
1:245: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:29: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:31: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:40: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:42: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:64: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
4:66: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:4: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:6: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:39: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:41: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:88: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:90: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:102: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:104: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:133: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:135: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:201: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:203: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:78: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:80: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:114: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:116: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:140: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:142: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:193: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:195: Unexpected character '$'. Did you mean a string literal like "...", @"..." o ... (message truncated) |
Failed |
IN9 245
OUT4 772 |
| × |
LcbPolishManiaLiveCodeBench / atcoder / hard: arc186_d Polish Mania (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PolishMania case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 266
OUT263 |
| × |
LcbReversi2LiveCodeBench / atcoder / medium: arc189_a Reversi 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: Reversi2 case0 runtime error: List index 1 is out of range. The list has 0 element(s).
Values:
run.Success = false |
Failed |
IN9 279
OUT739 |
| × |
LcbSumOfCCLiveCodeBench / atcoder / hard: arc187_b Sum of CC (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SumOfCC case0: expected [6] got [998244344] |
Failed |
IN9 284
OUT812 |
| ✓ |
LcbSymmetricPaintingLiveCodeBench / atcoder / hard: arc188_b Symmetric Painting (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 241
OUT6 014 |
| × |
LcbTakahashiIsSlimeLiveCodeBench / atcoder / hard: arc189_d Takahashi is Slime (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TakahashiIsSlime case0: expected [4 30 2 13 2 13] got [12 17 15 3 32 30] |
Failed |
IN9 233
OUT623 |
| × |
LcbTypicalPermutationDescriptorLiveCodeBench / atcoder / hard: arc186_b Typical Permutation Descriptor (Pattern B, stdin)1 AI query
1 tool callcategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: TypicalPermutationDescriptor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:19: Expected ';' after the struct field, found '('.
Values:
run.Compiled = false |
Failed |
IN20 194
OUT5 475 |
Code Generation - LiveCodeBench v5 LeetCode (set 1)
2 passed9 failed |
18.2% |
IN101 058
OUT4 236 |
| × |
LcbCountWinningSequencesLiveCodeBench / leetcode / hard: count-the-number-of-winning-sequences1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countWinning case0: expected 3 got 24 |
Failed |
IN9 255
OUT419 |
| × |
LcbDfsPalindromesLiveCodeBench / leetcode / hard: check-if-dfs-strings-are-palindromes1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: dfsPalindromes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'findAnswer' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, FindSubtreePalindromes.
Values:
run.Compiled = false |
Failed |
IN9 259
OUT552 |
| × |
LcbFindXSumLiveCodeBench / leetcode / easy: find-x-sum-of-all-k-long-subarrays-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: findXSum case0: expected [6, 10, 12] got [6,7,5] |
Failed |
IN9 231
OUT536 |
| ✓ |
LcbIsBalancedStringLiveCodeBench / leetcode / easy: check-balanced-string1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 121
OUT101 |
| × |
LcbLengthAfterTransformationsLiveCodeBench / leetcode / medium: total-characters-in-string-after-transformations-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: lengthAfter case0 runtime error: Pow with int, long or decimal base expects a non-negative integer exponent. Use double values for fractional or negative exponents.
Values:
run.Success = false |
Failed |
IN9 173
OUT205 |
| × |
LcbMaxFactorScoreLiveCodeBench / leetcode / medium: find-the-maximum-factor-score-of-array1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxScore case2: expected 9 got 0 |
Failed |
IN9 166
OUT427 |
| × |
LcbMinDivisionOperationsLiveCodeBench / leetcode / medium: minimum-division-operations-to-make-array-non-decreasing1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOperations case0: expected 1 got -1 |
Failed |
IN9 155
OUT382 |
| × |
LcbMinTimeToReachRoomLiveCodeBench / leetcode / medium: find-minimum-time-to-reach-last-room-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minTimeToReach case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:13: Variable 'dr' has type 'int' but its initializer has type 'List<int>'.
22:13: Variable 'dc' has type 'int' but its initializer has type 'List<int>'.
25:28: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
26:28: Type 'int' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
Values:
run.Compiled = false |
Failed |
IN9 225
OUT426 |
| ✓ |
LcbNumberOfSubstringsKFreqLiveCodeBench / leetcode / medium: count-substrings-with-k-frequency-characters-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 111
OUT181 |
| × |
LcbStringSequenceLiveCodeBench / leetcode / medium: find-the-sequence-of-strings-appeared-on-the-screen1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<string>>(run.ResultJson) == FromJson<List<string>>(expected[i])
Context: stringSequence case0: expected ["a", "aa", "ab", "aba", "abb", "abc"] got ["a","ab","abc"] |
Failed |
IN9 174
OUT216 |
| × |
LcbSubsequencePairCountGcdLiveCodeBench / leetcode / hard: find-the-number-of-subsequences-with-equal-gcd1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: subseqPair case0: expected 10 got 192 |
Failed |
IN9 188
OUT791 |
Code Generation - LiveCodeBench v5 LeetCode (set 2)
6 passed4 failed |
60.0% |
IN92 112
OUT6 550 |
| × |
LcbCountBalancedPermutationsLiveCodeBench / leetcode / hard: count-number-of-balanced-permutations1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: countBalPerm case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
260:7: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed |
IN9 158
OUT2 929 |
| × |
LcbCountKReducibleNumbersLiveCodeBench / leetcode / hard: count-k-reducible-numbers-less-than-n1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: countKReducible case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 185
OUT936 |
| ✓ |
LcbCountValidSelectionsLiveCodeBench / leetcode / easy: make-array-elements-equal-to-zero1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 241
OUT255 |
| ✓ |
LcbHasIncreasingSubarraysLiveCodeBench / leetcode / easy: adjacent-increasing-subarrays-detection-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 221
OUT197 |
| ✓ |
LcbIsZeroArrayLiveCodeBench / leetcode / medium: zero-array-transformation-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 213
OUT188 |
| × |
LcbMaximizeSumOfWeightsLiveCodeBench / leetcode / hard: maximize-sum-of-weights-after-edge-removals1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maximizeSumWeights case0: expected 22 got 0 |
Failed |
IN9 268
OUT766 |
| ✓ |
LcbMinAdjacentDifferenceLiveCodeBench / leetcode / hard: minimize-the-maximum-adjacent-element-difference1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 171
OUT363 |
| ✓ |
LcbMinArraySumLiveCodeBench / leetcode / medium: minimum-array-sum1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 282
OUT386 |
| ✓ |
LcbMinimumPositiveSumSubarrayLiveCodeBench / leetcode / easy: minimum-positive-sum-subarray1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 215
OUT193 |
| × |
LcbSumOfGoodSubsequencesLiveCodeBench / leetcode / hard: sum-of-good-subsequences1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: sumGood case0: expected 14 got 8 |
Failed |
IN9 158
OUT337 |
Code Generation - LiveCodeBench v5 LeetCode (set 3)
3 passed8 failed |
27.3% |
IN101 333
OUT4 103 |
| × |
LcbMaxCollectedFruitsLiveCodeBench / leetcode / hard: find-the-maximum-number-of-fruits-collected (id 3648)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxCollectedFruits case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
65:56: Cannot assign a value of type 'long' to variable 'bestNext' of type 'int'.
Values:
run.Compiled = false |
Failed |
IN9 358
OUT1 034 |
| × |
LcbMaxFrequencyOpsLiveCodeBench / leetcode / medium: maximum-frequency-of-an-element-after-performing-operations-i (id 3622)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxFrequency case1: expected 2 got 1 |
Failed |
IN9 177
OUT246 |
| × |
LcbMaxRemovalsSourceLiveCodeBench / leetcode / medium: find-maximum-removals-from-source-string (id 3487)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxRemovals case1: expected 2 got 1 |
Failed |
IN9 275
OUT619 |
| × |
LcbMaxScoreTouristLiveCodeBench / leetcode / medium: maximum-points-tourist-can-earn (id 3587)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: maxScore case0 runtime error: List index 1 is out of range. The list has 1 element(s).
Values:
run.Success = false |
Failed |
IN9 267
OUT298 |
| × |
LcbMinBitwiseArrayLiveCodeBench / leetcode / easy: construct-the-minimum-bitwise-array-i (id 3605)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: minBitwiseArray case0: expected [-1, 1, 4, 3] got [-1,-1,-1,-1] |
Failed |
IN9 189
OUT300 |
| × |
LcbNumberOfWaysEventLiveCodeBench / leetcode / hard: find-the-number-of-possible-ways-for-an-event (id 3604)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfWays case0: expected 6 got 27 |
Failed |
IN9 216
OUT201 |
| × |
LcbPossibleStringCountLiveCodeBench / leetcode / easy: find-the-original-typed-string-i (id 3617)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: possibleStringCount case0: expected 5 got 8 |
Failed |
IN9 135
OUT135 |
| ✓ |
LcbShiftDistanceLiveCodeBench / leetcode / medium: shift-distance-between-two-strings (id 3591)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 203
OUT369 |
| ✓ |
LcbSmallestDivisibleDigitProductLiveCodeBench / leetcode / easy: smallest-divisible-digit-product-i (id 3626)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 125
OUT131 |
| ✓ |
LcbStoneRemovalGameLiveCodeBench / leetcode / easy: stone-removal-game (id 3625)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 134
OUT175 |
| × |
LcbZeroArrayTransformIIILiveCodeBench / leetcode / medium: zero-array-transformation-iii (id 3647)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxRemoval case0: expected 1 got 0 |
Failed |
IN9 254
OUT595 |
Code Generation - LiveCodeBench v5 LeetCode (set 4)
4 passed6 failed |
40.0% |
IN91 869
OUT2 977 |
| × |
LcbButtonLongestTimeLiveCodeBench / leetcode / easy: button-with-longest-push-time (id 3632)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: buttonWithLongestTime case1: expected 10 got 1 |
Failed |
IN9 204
OUT156 |
| ✓ |
LcbCountComponentsLcmLiveCodeBench / leetcode / hard: count-connected-components-in-lcm-graph (id 3680)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 173
OUT318 |
| × |
LcbDigitOpsEqualLiveCodeBench / leetcode / medium: digit-operations-to-make-two-integers-equal (id 3655)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: digitOpsEqual case0: expected 85 got 22 |
Failed |
IN9 199
OUT579 |
| × |
LcbLargestOutlierLiveCodeBench / leetcode / medium: identify-the-largest-outlier-in-an-array (id 3594)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: getLargestOutlier case1: expected 4 got -2 |
Failed |
IN9 198
OUT231 |
| ✓ |
LcbMaxRectangleAreaLiveCodeBench / leetcode / medium: maximum-area-rectangle-with-point-constraints-i (id 3681)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 212
OUT944 |
| ✓ |
LcbMaxSubarraySumDivKLiveCodeBench / leetcode / medium: maximum-subarray-sum-with-length-divisible-by-k (id 3653)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 148
OUT175 |
| × |
LcbMinOpsEqualKLiveCodeBench / leetcode / easy: minimum-operations-to-make-array-values-equal-to-k (id 3621)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsEqualK case0: expected 2 got -1 |
Failed |
IN9 187
OUT194 |
| × |
LcbMinTimeBreakLocksLiveCodeBench / leetcode / medium: minimum-time-to-break-locks-i (id 3649)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: findMinimumTime case0: expected 4 got 5 |
Failed |
IN9 182
OUT125 |
| × |
LcbSmallestAllSetBitsLiveCodeBench / leetcode / easy: smallest-number-with-all-set-bits (id 3676)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: smallestAllBits case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 121
OUT91 |
| ✓ |
LcbTransformedArrayLiveCodeBench / leetcode / easy: transformed-array (id 3651)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 245
OUT164 |
Code Generation - LiveCodeBench v5 LeetCode (set 5)
5 passed5 failed |
50.0% |
IN91 765
OUT3 038 |
| ✓ |
LcbBeautifulSplitsLiveCodeBench / leetcode / medium: count beautiful splits in an array (LCB id 3686)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 146
OUT301 |
| ✓ |
LcbCountGoodArraysLiveCodeBench / leetcode / hard: count arrays with k matching adjacent elements (LCB id 3682)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 183
OUT474 |
| ✓ |
LcbCountLen3SubarraysLiveCodeBench / leetcode / easy: count length-3 subarrays with a condition (LCB id 3685)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 115
OUT114 |
| × |
LcbCountSpecialSubsequencesLiveCodeBench / leetcode / medium: count special subsequences (LCB id 3699)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfSubsequences case1: expected 3 got 2 |
Failed |
IN9 197
OUT175 |
| × |
LcbLargestBoxStringLiveCodeBench / leetcode / medium: lexicographically largest string from the box I (LCB id 3683)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: answerString case1: expected "g" got "ggg" |
Failed |
IN9 147
OUT212 |
| × |
LcbMakeStringGoodLiveCodeBench / leetcode / hard: minimum operations to make character frequencies equal (LCB id 3638)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: makeStringGood case0: expected 1 got 0 |
Failed |
IN9 151
OUT226 |
| ✓ |
LcbMaxCurrencyAmountLiveCodeBench / leetcode / medium: maximize amount after two days of conversions (LCB id 3613)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 327
OUT656 |
| × |
LcbMaxDistinctElementsLiveCodeBench / leetcode / medium: maximum number of distinct elements after operations (LCB id 3620)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxDistinctElements case1: expected 3 got 4 |
Failed |
IN9 154
OUT355 |
| × |
LcbMinIdenticalSubstringLiveCodeBench / leetcode / hard: smallest substring with identical characters I (LCB id 3690)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minLength case0: expected 2 got 4 |
Failed |
IN9 176
OUT375 |
| ✓ |
LcbMinOpsDistinctLiveCodeBench / leetcode / easy: minimum operations to make elements distinct (LCB id 3656)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 169
OUT150 |
Code Generation - LiveCodeBench v5 LeetCode (set 6)
2 passed8 failed |
20.0% |
IN92 060
OUT5 051 |
| × |
LcbCheckValidCutsLiveCodeBench / leetcode / medium: check if grid can be cut into sections (LCB id 3657)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: checkValidCuts case2: expected false got true |
Failed |
IN9 265
OUT532 |
| × |
LcbCountXorPathsLiveCodeBench / leetcode / medium: count paths with the given xor value (LCB id 3659)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countPathsWithXorValue case0: expected 3 got 2 |
Failed |
IN9 250
OUT552 |
| × |
LcbLongestDecreasingDiffLiveCodeBench / leetcode / medium: longest subsequence with decreasing adjacent difference (LCB id 3716)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: longestSubsequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:9: Unknown variable 'dpj'. Declare it first, e.g. 'int dpj = ...;'.
10:39: Unknown variable 'dpj'.
16:28: Unknown variable 'dpi'.
17:28: Unknown variable 'dpj'.
17:38: Unknown variable 'dpj'.
20:13: Unknown variable 'dpj'.
20:37: Unknown variable 'dpj'.
21:39: Unknown variable 'dpi'.
21:48: Unknown variable 'dpj'.
Values:
run.Compiled = false |
Failed |
IN9 206
OUT250 |
| × |
LcbMaxCoinsKBagsLiveCodeBench / leetcode / medium: maximum coins from k consecutive bags (LCB id 3715)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maximumCoins case0: expected 10 got 17 |
Failed |
IN9 195
OUT733 |
| × |
LcbMaxIntervalScoreLiveCodeBench / leetcode / hard: maximum score of non-overlapping intervals (LCB id 3562)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maximumWeight case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
18:19: Cannot add a value of type 'List<long>' to a 'List<long>'.
19:21: Cannot add a value of type 'List<List<int>>' to a 'List<List<int>>'.
23:17: Type 'long' cannot be assigned by index. Only List and Dictionary elements can be assigned.
24:19: Cannot assign a value of type 'List<int>' to an element of type 'int'.
42:33: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
44:41: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
45:57: List constructor expects a List or HashSet to copy, but got 'int'.
51:33: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
53:41: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
54:57: List constructor expects a List or HashSet to copy, but got 'int'.
60:38: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
61:27: Variable 'skipI' has type 'List<int>' but its initializer has type 'int'.
72:21: Type 'long' cannot be assigned by index. Only List and Dictionary elements can be assigned.
73:23: Cannot assign a value of type 'List<int>' to an element of type 'int'.
80:27: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
81:19: Variable 'ii' has type 'List<int>' but its initializer has type 'int'.
Values:
run.Compiled = false |
Failed |
IN9 259
OUT966 |
| × |
LcbMaxProductEquivalentLiveCodeBench / leetcode / easy: maximum subarray with equal products (LCB id 3702)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxLength case0: expected 5 got 1 |
Failed |
IN9 183
OUT278 |
| × |
LcbMaxSubarrayAfterRemovalLiveCodeBench / leetcode / hard: maximize subarray sum after removing all occurrences of one element (LCB id 3688)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxSubarraySum case0: expected 7 got 6 |
Failed |
IN9 154
OUT154 |
| × |
LcbMiddleModeSubsequencesLiveCodeBench / leetcode / hard: subsequences with a unique middle mode I (LCB id 3700)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: subsequencesWithMiddleMode case0: expected 6 got 0 |
Failed |
IN9 211
OUT1 029 |
| ✓ |
LcbMirrorScoreLiveCodeBench / leetcode / medium: find mirror score of a string (LCB id 3634)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 184
OUT276 |
| ✓ |
LcbSubstringPatternLiveCodeBench / leetcode / easy: substring matching pattern (LCB id 3684)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 153
OUT281 |
Code Generation - LiveCodeBench v6 AtCoder (set 1)
7 passed9 failed |
43.8% |
IN170 427
OUT11 803 |
| × |
LcbComingOfAgeLiveCodeBench / atcoder abc388_d / medium: Coming of Age Celebration1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ComingOfAge case0: expected [2 0 10 5] got [8 3 12 6] |
Failed |
IN9 208
OUT218 |
| × |
LcbCountArraysLiveCodeBench / atcoder abc387_f / hard: Count Arrays1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CountArrays case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
54:46: Cannot add a value of type 'null' to a 'List<List<long>>'.
Values:
run.Compiled = false |
Failed |
IN20 159
OUT4 896 |
| × |
LcbDangerousSugorokuLiveCodeBench / atcoder abc388_f / hard: Dangerous Sugoroku1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DangerousSugoroku case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:17: Unknown variable 'tt'. Declare it first, e.g. 'int tt = ...;'.
39:25: Unknown variable 'tt'.
Values:
run.Compiled = false |
Failed |
IN9 247
OUT726 |
| ✓ |
LcbDigitProductLiveCodeBench / atcoder abc389_a / easy: 9x91 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 148
OUT71 |
| ✓ |
LcbFactorialInverseLiveCodeBench / atcoder abc389_b / easy: tcaF1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 154
OUT69 |
| ✓ |
LcbHeavySnakeLiveCodeBench / atcoder abc388_b / easy: Heavy Snake1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 193
OUT256 |
| ✓ |
LcbKagamimochi2LiveCodeBench / atcoder abc388_g / hard: Simultaneous Kagamimochi 21 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 257
OUT398 |
| ✓ |
LcbNotXSumLiveCodeBench / atcoder abc387_b / easy: 9x9 Sum1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 209
OUT129 |
| × |
LcbOddEvenGraphLiveCodeBench / atcoder abc389_g / hard: Odd Even Graph1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: OddEvenGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 244
OUT1 470 |
| ✓ |
LcbQuestionUpcLiveCodeBench / atcoder abc388_a / easy: ?UPC1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 154
OUT37 |
| × |
LcbRatedRangeLiveCodeBench / atcoder abc389_f / hard: Rated Range1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RatedRange case0: expected [6
6
8] got [3
2
5] |
Failed |
IN21 460
OUT2 333 |
| × |
LcbSimultaneousKagamimochiLiveCodeBench / atcoder abc388_e / hard: Simultaneous Kagamimochi1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SimultaneousKagamimochi case0: expected [3] got [1] |
Failed |
IN9 196
OUT167 |
| × |
LcbSnakeNumbersLiveCodeBench / atcoder abc387_c / medium: Snake Numbers1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SnakeNumbers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:19: Function 'CountUpTo' must return 'string', but this return statement has type 'int'.
62:5: Function 'CountUpTo' must return 'string', but this return statement has type 'long'.
70:29: Operator '-' requires numeric operands, but got 'string' and 'string'.
Values:
run.Compiled = false |
Failed |
IN9 203
OUT527 |
| ✓ |
LcbSquareOfSumLiveCodeBench / atcoder abc387_a / easy: Happy New Year 20251 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 135
OUT65 |
| × |
LcbSquaresInCircleLiveCodeBench / atcoder abc389_d / medium: Squares in Circle1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SquaresInCircle case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 234
OUT242 |
| × |
LcbVariousKagamimochiLiveCodeBench / atcoder abc388_c / medium: Various Kagamimochi1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: VariousKagamimochi case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 226
OUT199 |
Code Generation - LiveCodeBench v6 AtCoder (set 2)
6 passed9 failed |
40.0% |
IN188 504
OUT17 143 |
| × |
LcbDoubleSum3LiveCodeBench / atcoder abc390_f / hard: Double Sum 31 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DoubleSum3 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 233
OUT674 |
| ✓ |
LcbGeometricSequenceLiveCodeBench / atcoder abc390_b / easy: Geometric Sequence1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 164
OUT254 |
| × |
LcbGravityLiveCodeBench / atcoder abc391_d / medium: Gravity1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Gravity case0: expected [Yes
Yes
No
Yes
No
Yes] got [No
Yes
No
Yes
No
Yes] |
Failed |
IN21 834
OUT2 162 |
| × |
LcbKthLargestLiveCodeBench / atcoder abc391_f / hard: K1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: KthLargest case0 runtime error: Integer overflow.
Values:
run.Success = false |
Failed |
IN9 236
OUT562 |
| × |
LcbMajorityVoteLiveCodeBench / atcoder abc391_e / hard: Hierarchical Majority Vote1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MajorityVote case0: expected [1] got [0] |
Failed |
IN21 063
OUT1 569 |
| × |
LcbManyLcsLiveCodeBench / atcoder abc391_g / hard: Many LCS1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ManyLcs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:15: Unexpected character '≤'.
5:21: Unexpected character '≤'.
7:9: Unexpected character '≤'.
9:101: Unexpected character '≤'.
11:26: Unexpected character '≤'.
11:128: Char literals must contain exactly one character and close with '''.
13:140: Unexpected character '≥'.
13:154: Unexpected character '≥'.
13:187: Unexpected character '≥'.
15:18: Unexpected character '≥'.
17:66: Unexpected character '≥'.
19:111: Char literals must contain exactly one character and close with '''.
21:77: Unexpected character '≥'.
21:183: Char literals must contain exactly one character and close with '''.
23:15: Unexpected character '≤'.
25:80: Unexpected character '⊆'.
25:246: Unexpected character '∈'.
25:258: Unexpected character '∈'.
25:279: Unexpected character '≥'.
25:305: Unexpected character '∈'.
25:583: Unexpected character '≠'.
25:626: Unexpected character '∈'.
25:629: Unexpected character '→'.
25:863: Unexpected character '→'.
25:919: Unexpected character '≠'.
25:922: Unexpected character '→'.
25:946: Unexpected character '≠'.
25:949: Unexpected character '→'.
27:182: Unexpected character '⊆'.
29:48: Unexpected character '≤'.
29:439: Unexpected character '→'.
31:16: Char literals must contain exactly one character and close with '''.
31:314: Unexpected character '∈'.
31:389: Unexpected character '→'.
31:436: Unexpected character '∈'.
31:553: Unexpected character '≤'.
33:76: Char literals must contain exactly one character and close with '''.
37:26: Unexpected character '≈'.
37:40: Unexpected character '≈'.
59:16: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 213
OUT2 743 |
| ✓ |
LcbOneSwapSortLiveCodeBench / atcoder abc390_a / easy: 124351 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 197
OUT153 |
| ✓ |
LcbOppositeDirectionLiveCodeBench / atcoder abc391_a / easy: Lucky Direction1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 174
OUT119 |
| ✓ |
LcbPaintRectangleLiveCodeBench / atcoder abc390_c / medium: Paint to make a rectangle1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 220
OUT346 |
| × |
LcbPermutationConcatLiveCodeBench / atcoder abc390_g / hard: Permutation Concatenation1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PermutationConcat case0: expected [1332] got [72000] |
Failed |
IN9 245
OUT372 |
| ✓ |
LcbSeekGridLiveCodeBench / atcoder abc391_b / easy: Seek Grid1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 212
OUT572 |
| ✓ |
LcbShuffledEquationLiveCodeBench / atcoder abc392_a / easy: Shuffled Equation1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 189
OUT144 |
| × |
LcbSquarePriceLiveCodeBench / atcoder abc389_e / hard: Square Price1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SquarePrice case1: expected [53] got [31] |
Failed |
IN9 196
OUT339 |
| × |
LcbStoneXorLiveCodeBench / atcoder abc390_d / medium: Stone XOR1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: StoneXor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:20: Expected '(' to declare a function or '=' to declare a global variable after 'List<long> g_blocks', found ';'. Global variables must have an initializer.
Values:
run.Compiled = false |
Failed |
IN35 118
OUT2 576 |
| × |
LcbVitaminBalanceLiveCodeBench / atcoder abc390_e / hard: Vitamin Balance1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: VitaminBalance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 210
OUT4 558 |
Code Generation - LiveCodeBench v6 AtCoder (set 3)
5 passed5 failed |
50.0% |
IN118 345
OUT7 827 |
| ✓ |
LcbABCIntervalsLiveCodeBench / atcoder / abc393_b A..B..C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT149 |
| × |
LcbBibStaringLiveCodeBench / atcoder / abc392_c Bib (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BibStaring case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:29: Argument 2 of 'Join' must be 'List<string>', but it has type 'List<int>'.
Values:
run.Compiled = false |
Failed |
IN9 235
OUT236 |
| × |
LcbDoublesProbabilityLiveCodeBench / atcoder / abc392_d Doubles (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DoublesProbability case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
32:16: Type 'double' has no supported method named 'ToString'.
32:26: Unknown variable 'System'.
Values:
run.Compiled = false |
Failed |
IN9 245
OUT301 |
| ✓ |
LcbFineTripletsLiveCodeBench / atcoder / abc392_g Fine Triplets (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 215
OUT198 |
| × |
LcbGcdOfSubsetLiveCodeBench / atcoder / abc393_e GCD of Subset (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: GcdOfSubset case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN35 272
OUT4 199 |
| × |
LcbInsertSequenceLiveCodeBench / atcoder / abc392_f Insert (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: InsertSequence case0: expected [4 2 3 1] got [1 3 3 4] |
Failed |
IN9 230
OUT250 |
| ✓ |
LcbPoisonousOysterLiveCodeBench / atcoder / abc393_a Poisonous Oyster (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 250
OUT150 |
| × |
LcbPrefixLisQueryLiveCodeBench / atcoder / abc393_f Prefix LIS Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PrefixLisQuery case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
5:74: Unexpected character '≤'.
7:256: Unexpected character '≤'.
7:350: Char literals must contain exactly one character and close with '''.
9:163: Unexpected character '≤'.
9:254: Char literals must contain exactly one character and close with '''.
13:100: Unexpected character '≤'.
13:174: Unexpected character '→'.
15:19: Unexpected character '≥'.
17:16: Unexpected character '#'.
21:28: Unexpected character '→'.
21:273: Char literals must contain exactly one character and close with '''.
23:24: Unexpected character '→'.
27:31: Unexpected character '→'.
27:53: Unexpected character '→'.
Values:
run.Compiled = false |
Failed |
IN9 260
OUT1 853 |
| ✓ |
LcbSwapToGatherLiveCodeBench / atcoder / abc393_d Swap to Gather (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 213
OUT215 |
| ✓ |
LcbWhoIsMissingLiveCodeBench / atcoder / abc392_b Who is Missing? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 233
OUT276 |
Code Generation - LiveCodeBench v6 AtCoder (set 4)
4 passed6 failed |
40.0% |
IN142 736
OUT10 926 |
| × |
LcbAlkaneSubtreeLiveCodeBench / atcoder / abc394_f Alkane (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AlkaneSubtree case0: expected [8] got [-1] |
Failed |
IN9 248
OUT454 |
| × |
LcbCatConcatLiveCodeBench / atcoder / abc394_b cat (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: CatConcat case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:8: Function 'Solve' is declared more than once. Overloads are not supported.
11:12: Type 'List<List<int>>' has no supported method named 'SortByLength' with 0 argument(s).
Values:
run.Compiled = false |
Failed |
IN9 175
OUT406 |
| ✓ |
LcbColorfulBracketsLiveCodeBench / atcoder / abc394_d Colorful Bracket Sequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 188
OUT255 |
| × |
LcbDebugWaAcLiveCodeBench / atcoder / abc394_c Debug (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DebugWaAc case1: expected [ACC] got [WAC] |
Failed |
IN9 159
OUT130 |
| × |
LcbDenseBuildingsLiveCodeBench / atcoder / abc394_g Dense Buildings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: DenseBuildings case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
42:21: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN9 380
OUT3 978 |
| ✓ |
LcbFilterTwosLiveCodeBench / atcoder / abc394_a 22222 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 151
OUT65 |
| ✓ |
LcbMakeTargetLiveCodeBench / atcoder / abc395_b Make Target (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 263
OUT375 |
| × |
LcbPalindromicShortestPathLiveCodeBench / atcoder / abc394_e Palindromic Shortest Path (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PalindromicShortestPath case0: expected [0 1 2 4
-1 0 1 -1
3 -1 0 1
1 -1 -1 0] got [1 1 2 4
-1 0 1 -1
3 -1 0 1
1 -1 -1 0] |
Failed |
IN37 085
OUT3 805 |
| ✓ |
LcbShortestDuplicateSubarrayLiveCodeBench / atcoder / abc395_c Shortest Duplicate Subarray (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 216
OUT168 |
| × |
LcbSmoothOcclusionLiveCodeBench / atcoder / abc395_f Smooth Occlusion (Pattern B, stdin)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SmoothOcclusion case0: expected [15] got [0] |
Failed |
IN31 871
OUT1 290 |
Code Generation - LiveCodeBench v6 AtCoder (set 5)
5 passed10 failed1 skipped |
33.3% |
IN204 860
OUT74 283 |
| × |
LcbBuyBallsLiveCodeBench / atcoder / medium: Buy Balls (abc396_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BuyBalls case0: expected [19] got [11] |
Failed |
IN9 231
OUT442 |
| × |
LcbCardPileLiveCodeBench / atcoder / easy: Card Pile (abc396_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: CardPile case0 runtime error: List index -1 is out of range. The list has 0 element(s).
Values:
run.Success = false |
Failed |
IN9 210
OUT187 |
| – |
LcbCubesLiveCodeBench / atcoder / hard: Cubes (abc397_d)0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbFlipEdgeLiveCodeBench / atcoder / hard: Flip Edge (abc395_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN9 224
OUT813 |
| ✓ |
LcbFlipRowOrColLiveCodeBench / atcoder / hard: Flip Row or Col (abc396_g)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN75 905
OUT58 832 |
| × |
LcbMaximizeDistanceLiveCodeBench / atcoder / hard: Maximize Distance (abc397_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MaximizeDistance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:85: Char literals must contain exactly one character and close with '''.
5:156: Unexpected character '…'.
5:262: Char literals must contain exactly one character and close with '''.
9:260: Unexpected character '≤'.
15:320: Unexpected character '≥'.
15:384: Unexpected character '≥'.
19:82: Unexpected character '→'.
19:109: Unexpected character '→'.
19:137: Char literals must contain exactly one character and close with '''.
21:243: Char literals must contain exactly one character and close with '''.
23:105: Unexpected character '∩'.
25:113: Char literals must contain exactly one character and close with '''.
27:248: Char literals must contain exactly one character and close with '''.
29:360: Unexpected character '→'.
29:468: Unexpected character '→'.
31:85: Unexpected character '→'.
31:398: Char literals must contain exactly one character and close with '''.
35:42: Char literals must contain exactly one character and close with '''.
37:44: Char literals must contain exactly one character and close with '''.
39:13: Unexpected character '⇔'.
41:2: Char literals must contain exactly one character and close with '''.
45:2: Char literals must contain exactly one character and close with '''.
47:14: Char literals must contain exactly one character and close with '''.
49:43: Unexpected character '#'.
51:3: Unexpected character '—'.
51:5: Char literals must contain exactly one character and close with '''.
55:2: Char literals must contain exactly one character and close with '''.
57:20: Char literals must contain exactly one character and close with '''.
59:5: Char literals must contain exactly one character and close with '''.
132:31: Unexpected character '#'.
Values:
run.Compiled = false |
Failed |
IN9 204
OUT3 313 |
| × |
LcbMinRestrictedSumLiveCodeBench / atcoder / hard: Min of Restricted Sum (abc396_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MinRestrictedSum case0: expected [0 3 4] got [-1] |
Failed |
IN9 245
OUT610 |
| × |
LcbMinXorPathLiveCodeBench / atcoder / medium: Minimum XOR Path (abc396_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: MinXorPath case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed |
IN9 227
OUT569 |
| × |
LcbPathDecompositionLiveCodeBench / atcoder / hard: Path Decomposition of a Tree (abc397_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PathDecomposition case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
7:31: Char literals must contain exactly one character and close with '''.
9:15: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:17: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:54: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:58: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:101: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:108: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:23: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:30: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:68: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
11:112: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:39: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:41: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:76: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
13:78: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
15:57: Unexpected character '→'.
15:61: Char literals must contain exactly one character and close with '''.
17:25: Char literals must contain exactly one character and close with '''.
18:6: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
18:10: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:6: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:9: Unexpected character '\'.
19:15: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:84: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:86: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:100: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:105: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:174: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:176: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:214: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:216: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:263: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:265: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:305: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:307: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:322: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
32:329: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
38:51: Char literals must contain exactly one character and close with '''.
44:189: Unexpected character '≤'.
44:196: Unexpected character '—'.
46:84: Unexpected character '→'.
46:92: Char literals must contain exactly one character and close with '''.
48:45: Unexpected character 'ℓ'.
52:85: Char literals must contain exactly one character and close with '''.
56:24: Char literals must contain exactly one character and close with '''.
58:147: Unexpected character '≥'.
60:87: Unexpected character '≥'.
60:107: Unexpected character '⇒'.
60:113: Unexpected character '≤'. ... (message truncated) |
Failed |
IN9 197
OUT3 926 |
| × |
LcbRotatedInversionsLiveCodeBench / atcoder / hard: Rotated Inversions (abc396_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RotatedInversions case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
5:11: Unexpected character 'ᵢ'.
5:17: Unexpected character 'ᵢ'.
10:21: Char literals must contain exactly one character and close with '''.
12:2: Unexpected character 'ᵢ'.
12:11: Unexpected character 'ᵢ'.
15:7: Unexpected character 'ᵢ'.
15:19: Unexpected character 'ᵢ'.
15:32: Unexpected character 'ᵢ'.
15:40: Unexpected character 'ᵢ'.
16:7: Unexpected character 'ᵢ'.
16:13: Unexpected character '≥'.
16:19: Unexpected character 'ᵢ'.
16:47: Unexpected character 'ᵢ'.
16:57: Unexpected character 'ᵢ'.
16:95: Unexpected character 'ᵢ'.
16:114: Unexpected character 'ᵢ'.
16:122: Unexpected character '≤'.
18:11: Unexpected character 'ᵢ'.
18:20: Unexpected character 'ᵢ'.
18:55: Unexpected character 'ᵢ'.
20:49: Unexpected character 'ᵢ'.
22:97: Unexpected character 'ᵢ'.
22:102: Unexpected character 'ⱼ'.
24:2: Unexpected character 'ᵢ'.
24:11: Unexpected character 'ᵢ'.
24:24: Unexpected character 'ⱼ'.
24:33: Unexpected character 'ⱼ'.
26:3: Unexpected character 'ᵢ'.
26:18: Unexpected character 'ⱼ'.
28:10: Unexpected character 'ᵢ'.
28:15: Unexpected character 'ⱼ'.
31:22: Unexpected character 'ᵢ'.
31:35: Unexpected character 'ⱼ'.
31:43: Unexpected character '→'.
31:46: Unexpected character 'ᵢ'.
31:51: Unexpected character 'ⱼ'.
31:53: Unexpected character '→'.
32:7: Unexpected character 'ᵢ'.
32:11: Unexpected character '≥'.
32:20: Unexpected character 'ⱼ'.
32:31: Unexpected character 'ᵢ'.
32:42: Unexpected character 'ⱼ'.
32:47: Unexpected character '→'.
32:67: Unexpected character 'ᵢ'.
32:82: Unexpected character 'ⱼ'.
32:86: Unexpected character '≥'.
32:101: Unexpected character 'ᵢ'.
32:149: Unexpected character 'ᵢ'.
32:158: Unexpected character 'ⱼ'.
32:162: Unexpected character '→'.
32:165: Unexpected character 'ᵢ'.
32:174: Unexpected character 'ⱼ'.
32:176: Unexpected character '→'.
32:179: Unexpected character 'ᵢ'.
32:184: Unexpected character 'ⱼ'.
32:190: Unexpected character '→'.
32:198: Unexpected character 'ᵢ'.
32:201: Unexpected character 'ⱼ'.
32:238: Unexpected character 'ᵢ'.
32:247: Unexpected character 'ⱼ'.
32:261: Unexpected character 'ᵢ'.
32:270: Unexpected character 'ⱼ'.
32:280: Unexpected character 'ᵢ'.
32:285: Unexpected character 'ⱼ'.
36:5: Unexpected character 'ᵢ'.
36:9: Unexpected character '≥'.
36:15: Unexpected character 'ᵢ'.
36:20: Unexpected character 'ᵢ'.
36:38: Unexpected character 'ᵢ'.
36:43: Unexpected character 'ⱼ'.
36:48: Unexpected character 'ⱼ'.
36:63: Unexpected character 'ⱼ'.
36:72: Unexpected character 'ᵢ'.
36:81: Unexpected character 'ⱼ'.
36:85: Unexpected character '⟺'.
36:88: Unexpected character 'ᵢ'.
36:95: Unexpected character 'ⱼ'.
36:97: Unexpected character '⟺'.
36:100: Unexpected character 'ᵢ'.
36:105: Unexpected character 'ⱼ'.
36:117: Unexpected character 'ᵢ'.
36:128: Unexpected character 'ⱼ'.
36:130: Unexpected character '≥'.
36:136: Unexpected character 'ᵢ'.
36:141: Unexpected character 'ⱼ'.
36:145: Unexpected character '≥'.
38:7: Unexpected character 'ⱼ'.
38:11: Unexpected character '≥'.
38:20: Unexpected character 'ᵢ'.
38:30: Unexpected character 'ᵢ'.
38:35: Unexpected character 'ᵢ'.
38:41: Unexpected character 'ⱼ'.
38:46: Unexpected character 'ⱼ'.
38:65: Unexpected character 'ᵢ'.
38:72: Unexpected character 'ⱼ'.
38:78: Unexpected character '⟺'.
38:81: Unexpected character 'ᵢ'.
38:86: Unexpected character 'ⱼ'.
38:90: Unexpected character '⟺'.
38:93: Too many lexical errors; further errors are not reported.
Values:
run.Compiled = false |
Failed |
IN9 233
OUT4 290 |
| ✓ |
LcbStrictlyIncreasingLiveCodeBench / atcoder / easy: Strictly Increasing? (abc395_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 192
OUT125 |
| ✓ |
LcbThermometerLiveCodeBench / atcoder / easy: Thermometer (abc397_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 187
OUT65 |
| × |
LcbTicketGateLogLiveCodeBench / atcoder / medium: Ticket Gate Log (abc397_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TicketGateLog case0: expected [1] got [0] |
Failed |
IN9 190
OUT142 |
| ✓ |
LcbTripleFourLiveCodeBench / atcoder / easy: Triple Four (abc396_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 195
OUT148 |
| × |
LcbVarietySplitEasyLiveCodeBench / atcoder / medium: Variety Split Easy (abc397_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: VarietySplitEasy case0: expected [5] got [4] |
Failed |
IN9 203
OUT322 |
| × |
LcbVarietySplitHardLiveCodeBench / atcoder / hard: Variety Split Hard (abc397_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: VarietySplitHard case0: expected [5] got [11] |
Failed |
IN9 217
OUT499 |
Code Generation - LiveCodeBench v6 AtCoder (set 6)
6 passed9 failed |
40.0% |
IN200 447
OUT16 186 |
| ✓ |
LcbAbc400PartyLiveCodeBench / atcoder / easy: ABC400 Party (abc400_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 180
OUT61 |
| × |
LcbAbcbaLiveCodeBench / atcoder / hard: ABCBA (abc398_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Abcba case0: expected [ABCBA] got [ABCC] |
Failed |
IN9 145
OUT226 |
| × |
LcbBonfireLiveCodeBench / atcoder / hard: Bonfire (abc398_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Bonfire case0: expected [001010] got [0101010] |
Failed |
IN9 329
OUT410 |
| ✓ |
LcbDoorsInCenterLiveCodeBench / atcoder / easy: Doors in the Center (abc398_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 160
OUT137 |
| × |
LcbFullHouse3LiveCodeBench / atcoder / medium: Full House 3 (abc398_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FullHouse3 case2: expected [No] got [Yes] |
Failed |
IN9 202
OUT188 |
| × |
LcbGoodIntegersLiveCodeBench / atcoder / medium: 2^a b^2 good integers (abc400_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GoodIntegers case0: expected [5] got [0] |
Failed |
IN9 200
OUT145 |
| ✓ |
LcbHammingDistanceLiveCodeBench / atcoder / easy: Hamming Distance (abc399_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 184
OUT92 |
| ✓ |
LcbMakeItForestLiveCodeBench / atcoder / medium: Make it Forest (abc399_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 182
OUT255 |
| × |
LcbNotOnlyTreeGameLiveCodeBench / atcoder / hard: Not Only Tree Game (abc398_g)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NotOnlyTreeGame case2: expected [Aoki] got [Takahashi] |
Failed |
IN32 906
OUT2 739 |
| × |
LcbRangePowerSumLiveCodeBench / atcoder / hard: Range Power Sum (abc399_f)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RangePowerSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
41:5: Expected an expression, found '.'.
Values:
run.Compiled = false |
Failed |
IN20 113
OUT5 305 |
| ✓ |
LcbRankingWithTiesLiveCodeBench / atcoder / easy: Ranking with Ties (abc399_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed |
IN9 211
OUT397 |
| × |
LcbReplaceLiveCodeBench / atcoder / hard: Replace (abc399_e)1 AI query
2 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Replace case0: expected [4] got [6] |
Failed |
IN36 903
OUT4 641 |
| × |
LcbSwitchSeatsLiveCodeBench / atcoder / medium: Switch Seats (abc399_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SwitchSeats case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:12: Expected '(' to declare a function or '=' to declare a global variable after 'No visible', found 'content'. Global variables must have an initializer.
Values:
run.Compiled = false |
Failed |
IN9 258
OUT5 |
| ✓ |
LcbUniquenessLiveCodeBench / atcoder / medium: Uniqueness (abc398_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 186
OUT229 |
| × |
LcbWallBreakerLiveCodeBench / atcoder / medium: Takahashi the Wall Breaker (abc400_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: WallBreaker case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:17: Expected ';' after the variable declaration, found ','.
Values:
run.Compiled = false |
Failed |
IN9 288
OUT1 356 |
Code Generation - LiveCodeBench v6 AtCoder (set 7)
1 passed12 failed2 skipped |
7.7% |
IN154 692
OUT39 633 |
| – |
LcbAPowNLiveCodeBench / atcoder / hard: A^n (arc191_c) NOTE: special judge - any valid (A,M) pair is accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| ✓ |
LcbArcArcLiveCodeBench / atcoder / medium: ARC Arc (arc192_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 357
OUT354 |
| × |
LcbBasicGridUpdatesLiveCodeBench / atcoder / hard: Basic Grid Problem with Updates (arc190_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: BasicGridUpdates case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
30:21: Function 'ModInv' expects 2 argument(s), but the call has 1.
40:26: Function 'Fact' expects 2 argument(s), but the call has 1.
41:26: Function 'Fact' expects 2 argument(s), but the call has 1.
42:26: Function 'ModInv' expects 2 argument(s), but the call has 1.
42:33: Function 'Fact' expects 2 argument(s), but the call has 1.
55:26: Function 'Fact' expects 2 argument(s), but the call has 1.
56:26: Function 'Fact' expects 2 argument(s), but the call has 1.
57:26: Function 'ModInv' expects 2 argument(s), but the call has 1.
57:33: Function 'Fact' expects 2 argument(s), but the call has 1.
Values:
run.Compiled = false |
Failed |
IN9 420
OUT1 522 |
| × |
LcbBrokenWheelLiveCodeBench / atcoder / hard: Broken Wheel (arc193_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BrokenWheel case0: expected [14] got [6] |
Failed |
IN9 337
OUT179 |
| × |
LcbComplementIntervalGraphLiveCodeBench / atcoder / hard: Complement Interval Graph (arc193_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: ComplementIntervalGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:149: Char literals must contain exactly one character and close with '''.
7:191: Char literals must contain exactly one character and close with '''.
15:139: Unexpected character '—'.
15:235: Unexpected character '…'.
17:167: Unexpected character '→'.
17:189: Unexpected character '→'.
17:229: Char literals must contain exactly one character and close with '''.
21:192: Char literals must contain exactly one character and close with '''.
23:306: Unexpected character '≤'.
25:57: Unexpected character '≤'.
25:65: Char literals must contain exactly one character and close with '''.
27:80: Char literals must contain exactly one character and close with '''.
31:152: Unexpected character '→'.
33:42: Unexpected character '≥'.
33:186: Unexpected character '≥'.
33:306: Unexpected character '…'.
33:350: Char literals must contain exactly one character and close with '''.
35:15: Unexpected character '#'.
35:35: Unexpected character '≥'.
35:150: Unexpected character '—'.
35:155: Unexpected character '≥'.
35:418: Unexpected character '≥'.
37:210: Unexpected character '…'.
37:308: Unexpected character '→'.
39:111: Char literals must contain exactly one character and close with '''.
41:73: Unexpected character '≤'.
41:258: Unexpected character '…'.
43:20: Char literals must contain exactly one character and close with '''.
45:64: Unexpected character '→'.
45:177: Char literals must contain exactly one character and close with '''.
47:32: Unexpected character '→'.
49:82: Unexpected character '≤'.
51:306: Unexpected character '≤'.
51:452: Unexpected character '→'.
51:545: Char literals must contain exactly one character and close with '''.
53:114: Unexpected character '≤'.
53:203: Unexpected character '→'.
53:267: Char literals must contain exactly one character and close with '''.
56:48: Unexpected character '→'.
56:274: Unexpected character '→'.
56:347: Unexpected character '→'.
58:87: Unexpected character '…'.
58:144: Unexpected character '×'.
60:26: Char literals must contain exactly one character and close with '''.
62:2: Char literals must contain exactly one character and close with '''.
64:258: Unexpected character '≤'.
64:261: Unexpected character '×'.
66:2: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed |
IN9 384
OUT3 164 |
| × |
LcbFennecVsSnuke2LiveCodeBench / atcoder / hard: Fennec VS. Snuke 2 (arc192_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FennecVsSnuke2 case1: expected [Snuke] got [Fennec] |
Failed |
IN9 277
OUT141 |
| × |
LcbFractionLineLiveCodeBench / atcoder / hard: Fraction Line (arc192_d)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: FractionLine case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
17:5: Expected an expression, found '.'.
Values:
run.Compiled = false |
Failed |
IN20 336
OUT9 890 |
| – |
LcbInsideOrOutsideLiveCodeBench / atcoder / hard: Inside or Outside (arc190_a) NOTE: special judge - multiple minimum-cost operation sequences are accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbMatrixPowSumLiveCodeBench / atcoder / hard: Matrix Pow Sum (arc190_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MatrixPowSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:3: Char literals must contain exactly one character and close with '''.
5:9: Char literals must contain exactly one character and close with '''.
5:157: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:161: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
5:213: Unexpected character '—'.
11:14: Char literals must contain exactly one character and close with '''.
15:9: Unexpected character '—'.
19:26: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:30: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:52: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:57: Unexpected character '\'.
19:63: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:76: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:80: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:104: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:111: Unexpected character '\'.
19:118: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:140: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:143: Unexpected character '\'.
19:152: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:165: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:171: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:174: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
19:181: Unexpected character '\'.
19:193: Unexpected character '\'.
19:203: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
21:30: Char literals must contain exactly one character and close with '''.
23:25: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
23:26: Unexpected character '\'.
23:36: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
23:52: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
23:56: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
25:21: Char literals must contain exactly one character and close with '''.
27:20: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:24: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:37: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:40: Unexpected character '\'.
27:48: Unexpected character '\'.
27:56: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:59: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:61: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:77: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:83: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:88: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
27:89: Unexpected character '\'.
27:99: Unexpected character '\'.
27:132: Unexpected character '\'.
27:152: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
29:39: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
29:41: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
31:16: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
31:17: Unexpected character '\'.
31:34: ... (message truncated) |
Failed |
IN9 305
OUT7 182 |
| × |
LcbMovingPiecesOnGraphLiveCodeBench / atcoder / hard: Moving Pieces on Graph (arc191_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: MovingPiecesOnGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:35: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
run.Compiled = false |
Failed |
IN9 314
OUT691 |
| × |
LcbPatisserieAbc3LiveCodeBench / atcoder / hard: Patisserie ABC 3 (abc400_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: PatisserieAbc3 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:19: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed |
IN9 334
OUT3 670 |
| × |
LcbReplaceDigitsLiveCodeBench / atcoder / medium: Replace Digits (arc191_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ReplaceDigits case0: expected [593] got [325] |
Failed |
IN9 241
OUT311 |
| × |
LcbRingoFavoriteNumbers3LiveCodeBench / atcoder / hard: Ringo's Favorite Numbers 3 (abc400_e)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: RingoFavoriteNumbers3 case0 runtime error: A list of length 1000001 exceeded the list length limit of 1000000.
Values:
run.Success = false |
Failed |
IN20 859
OUT1 165 |
| × |
LcbSnukeKyotoTripLiveCodeBench / atcoder / hard: Snuke's Kyoto Trip (arc192_e)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: SnukeKyotoTrip case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed |
IN20 295
OUT11 198 |
| × |
LcbSumOfGeometricSeriesLiveCodeBench / atcoder / easy: Sum of Geometric Series (abc400_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SumOfGeometricSeries case2: expected [1000000000] got [inf] |
Failed |
IN9 233
OUT166 |
Code Generation - LiveCodeBench v6 AtCoder (set 8)
2 passed11 failed1 broken1 skipped |
14.3% |
IN206 835
OUT54 206 |
| × |
LcbAdjacentDeleteLiveCodeBench / atcoder / hard: Adjacent Delete (arc196_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AdjacentDelete case0: expected [5] got [6] |
Failed |
IN9 205
OUT153 |
| × |
LcbCostToFlipLiveCodeBench / atcoder / hard: Cost to Flip (arc194_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CostToFlip case0: expected [16] got [6] |
Failed |
IN9 278
OUT352 |
| – |
LcbHamiltonianPiecesLiveCodeBench / atcoder / hard: Hamiltonian Pieces (arc195_c) NOTE: special judge - any valid placement is accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped |
IN0
OUT0 |
| × |
LcbMagnetsLiveCodeBench / atcoder / hard: Magnets (arc193_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Magnets case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:602: Char literals must contain exactly one character and close with '''.
1:1775: Unexpected character '≠'.
1:2313: Unexpected character '→'.
1:2341: Unexpected character '→'.
1:2388: Unexpected character '→'.
1:2401: Unexpected character '→'.
1:2426: Unexpected character '↔'.
1:2717: Unexpected character '–'.
1:2767: Char literals must contain exactly one character and close with '''.
1:3162: Char literals must contain exactly one character and close with '''.
1:3273: Unexpected character '#'.
1:3282: Unexpected character '#'.
1:3687: Unexpected character '≤'.
1:3706: Unexpected character '≥'.
1:3815: Unexpected character '≤'.
1:3858: Unexpected character '≠'.
1:3988: Unexpected character '…'.
1:3993: Char literals must contain exactly one character and close with '''.
1:4971: Char literals must contain exactly one character and close with '''.
1:7492: Unexpected character '≥'.
1:7523: Unexpected character '≤'.
1:7733: Unexpected character '≤'.
1:7737: Unexpected character '≤'.
1:7741: Unexpected character '≤'.
1:7777: Unexpected character '←'.
1:7845: Unexpected character '…'.
1:7937: Unexpected character '≤'.
1:7941: Unexpected character '≤'.
1:8296: Unexpected character 'ℓ'.
1:8300: Unexpected character '#'.
1:8320: Unexpected character '#'.
1:8383: Unexpected character 'ℓ'.
1:8408: Unexpected character 'ℓ'.
1:8464: Unexpected character '∈'.
1:8522: Unexpected character 'ℓ'.
1:8570: Unexpected character '≤'.
1:8571: Unexpected character 'ℓ'.
1:8580: Unexpected character 'ℓ'.
1:8597: Unexpected character '≥'.
1:8758: Unexpected character 'Σ'.
1:8966: Unexpected character 'Φ'.
1:9002: Unexpected character 'Δ'.
1:9003: Unexpected character 'Φ'.
1:9005: Unexpected character '∈'.
1:9063: Unexpected character 'Δ'.
1:9064: Unexpected character 'Φ'.
1:9082: Unexpected character '±'.
1:9109: Unexpected character 'Φ'.
1:9590: Char literals must contain exactly one character and close with '''.
1:9643: Char literals must contain exactly one character and close with '''.
1:11512: Char literals must contain exactly one character and close with '''.
1:11638: Unexpected character '∈'.
1:11947: Unexpected character '≤'.
1:11965: Char literals must contain exactly one character and close with '''.
1:14090: Char literals must contain exactly one character and close with '''.
1:14255: Unexpected character '−'.
1:14259: Unexpected character '−'.
1:14275: Unexpected character '−'.
1:14278: Unexpected character '≡'.
1:14291: Unexpected character '≡'.
1:14294: Unexpected character '−'.
1:14297: Unexpected character '≡'.
1:14312: Unexpected character 'Δ'.
1:14315: Unexpected character '≡'.
1:14334: Unexpected character '#'.
1:14785: Unexpected character '→'.
1:14789: Unexpected character '→'.
1:14803: Unexpected character '⇒'.
1:15128: Unexpected character '…'.
1:15403: Unexpected character '→'.
1:15405: Unexpected character '±'.
1:15578: Unexpected character '≠'.
1:15591: Unexpected character '∓'.
1:15625: Unexpected character '±'.
1:15646: Char literals must contain exactly one character and close with '''.
1:16003: Unexpected character '±'.
1:16273: Unexpected character '≤'.
1:16401: Unexpected character '≤'.
1:16551: Unexpected character '±'.
1:16729: Char literals must contain exactly one character and close with '''.
1:16741: Char literals must contain exactly one character and close with '''.
1:16745: Unexpected character '≤'.
1:16755: Char literals must contain exactly one character and close with '''.
1:16872: Char literals must contain exactly one character and close with '''.
1:16968: Unexpected character '≤'.
1:17018: Char literals must contain exactly one character and close with '''.
1:17316: Unexpected character '→'.
1:17369: Unexpected character '±'.
1:17375: Unexpected character '±'.
1:17576: Unexpected character '−'.
1:17622: Unexpected character '−'.
1:17742: Unexpected character '⇔'.
1:17745: Char literals must contain exactly one character and close with ' ... (message truncated) |
Failed |
IN9 355
OUT11 020 |
| × |
LcbMinimumCostSortLiveCodeBench / atcoder / hard: Minimum Cost Sort (arc194_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MinimumCostSort case0: expected [4] got [0] |
Failed |
IN9 218
OUT281 |
| × |
LcbOperationsOnStackLiveCodeBench / atcoder / medium: Operations on a Stack (arc194_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OperationsOnStack case0: expected [8] got [5] |
Failed |
IN9 237
OUT161 |
| × |
LcbRandomTreeDistanceLiveCodeBench / atcoder / hard: Random Tree Distance (arc195_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: RandomTreeDistance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:72: Char literals must contain exactly one character and close with '''.
7:9: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:11: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:43: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:46: Unexpected character '\'.
7:54: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:64: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:68: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:95: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:96: Unexpected character '\'.
7:109: Unexpected character '\'.
7:111: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:133: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:142: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:155: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:159: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:186: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:192: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:216: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:223: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:255: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:257: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:263: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
7:265: Unexpected character '$'. Did you mean a string literal like "...", @"..." or $"..."?
9:56: Char literals must contain exactly one character and close with '''.
15:32: Unexpected character '#'.
17:5: Unexpected character '#'.
21:12: Unexpected character '#'.
31:33: Char literals must contain exactly one character and close with '''.
39:426: Unexpected character '⊆'.
44:74: Unexpected character '→'.
45:49: Unexpected character '→'.
45:196: Char literals must contain exactly one character and close with '''.
52:22: Unexpected character '∈'.
64:23: Unexpected character '∈'.
66:104: Char literals must contain exactly one character and close with '''.
68:4: Char literals must contain exactly one character and close with '''.
70:113: Char literals must contain exactly one character and close with '''.
72:8: Unexpected character '∈'.
72:168: Unexpected character '→'.
74:69: Unexpected character '✓'.
74:84: Unexpected character '✓'.
74:99: Unexpected character '✗'.
76:40: Char literals must contain exactly one character and close with '''.
78:44: Unexpected character '∈'.
78:100: Unexpected character 'Σ'.
78:214: Unexpected character '≥'.
80:22: Unexpected character '∈'.
80:41: Unexpected character '→'.
80:176: Char literals must contain exactly one character and close with '''.
84:58: Unexpected character '∈'.
84:75: Unexpected character '∈'.
86:88: Char literals must contain exactly one character and close with '''.
88:58: Char literals must contain exactly one character and close with '''.
88:153: Unexpected character '→'.
88:166: Unexpected character '✓'.
88:255: Unexpected character '∈'.
88:306: Unexpected character '→'.
88:372: Unexpected character '✗'.
90:216: Char literals must contain exactly one character and close with '''.
92:75: Unexpected character '∈'.
92:163: Char literals must contain exactly one character and close with '''.
94:20: Char literals must contain exactly one character and close with '''.
98:53: Unexpected character '↔'.
98:200 ... (message truncated) |
Failed |
IN9 369
OUT5 167 |
| ! |
LcbReverseBracketsLiveCodeBench / atcoder / hard: Reverse Brackets (arc194_d)1 AI query
4 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchSkipped: AI output broken/partial |
Broken |
IN68 266
OUT15 705 |
| × |
LcbRoadwayLiveCodeBench / atcoder / hard: Roadway (arc196_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: Roadway case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:1: Expected a type (bool, byte, int, long, decimal, double, char, string, List<...>, Dictionary<...>, HashSet<...> or a struct name), found '<'.
Values:
run.Compiled = false |
Failed |
IN9 426
OUT921 |
| × |
LcbStronglyConnectedLiveCodeBench / atcoder / hard: Strongly Connected (arc196_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: StronglyConnected case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:66: Unexpected character '→'.
3:89: Char literals must contain exactly one character and close with '''.
3:141: Char literals must contain exactly one character and close with '''.
7:29: Unexpected character '→'.
7:33: Unexpected character '→'.
7:37: Unexpected character '→'.
7:43: Unexpected character '→'.
11:92: Unexpected character '∈'.
15:63: Char literals must contain exactly one character and close with '''.
17:163: Unexpected character '→'.
17:165: Unexpected character '→'.
17:169: Unexpected character '→'.
17:214: Unexpected character '≤'.
17:234: Char literals must contain exactly one character and close with '''.
23:100: Unexpected character '→'.
23:264: Unexpected character '≥'.
23:280: Unexpected character '≤'.
23:354: Char literals must contain exactly one character and close with '''.
25:77: Unexpected character '→'.
25:101: Unexpected character '→'.
25:111: Unexpected character '≤'.
25:155: Unexpected character '→'.
25:167: Unexpected character '≥'.
25:178: Unexpected character '≥'.
25:188: Unexpected character '≤'.
29:286: Unexpected character '≥'.
29:324: Unexpected character '≤'.
29:371: Char literals must contain exactly one character and close with '''.
31:93: Unexpected character '∈'.
31:149: Unexpected character '≥'.
31:161: Unexpected character '≤'.
34:166: Unexpected character '✓'.
35:36: Unexpected character '∃'.
35:69: Unexpected character '≥'.
35:80: Unexpected character '≤'.
37:137: Char literals must contain exactly one character and close with '''.
37:296: Unexpected character '≤'.
37:417: Unexpected character '≤'.
37:522: Char literals must contain exactly one character and close with '''.
37:707: Unexpected character '≤'.
37:741: Unexpected character '—'.
37:786: Char literals must contain exactly one character and close with '''.
39:42: Unexpected character '∈'.
39:98: Unexpected character '≥'.
39:108: Unexpected character '≤'.
39:178: Unexpected character '≤'.
39:269: Unexpected character '≤'.
39:282: Unexpected character '∈'.
41:26: Char literals must contain exactly one character and close with '''.
45:8: Char literals must contain exactly one character and close with '''.
49:62: Unexpected character '·'.
49:65: Unexpected character '⁵'.
51:129: Unexpected character '≤'.
53:25: Unexpected character '#'.
53:53: Unexpected character '#'.
55:17: Unexpected character '⟺'.
55:223: Unexpected character '≤'.
57:13: Unexpected character '#'.
57:34: Unexpected character '⇒'.
57:36: Unexpected character '#'.
57:45: Unexpected character '≤'.
57:68: Unexpected character '#'.
59:38: Unexpected character '#'.
59:53: Unexpected character '#'.
59:101: Unexpected character '#'.
59:116: Unexpected character '#'.
59:134: Unexpected character '#'.
59:138: Unexpected character '≤'.
59:150: Unexpected character '#'.
59:154: Unexpected character '≤'.
59:206: Unexpected character '≤'.
59:214: Unexpected character '⇔'.
59:222: Unexpected character '≤'.
59:230: Unexpected character '⇔'.
59:236: Unexpected character '≤'.
59:242: Unexpected character '⇔'.
59:247: Unexpected character '≥'.
61:48: Unexpected character '≥'.
61:74: Unexpected character '≥'.
61:237: Unexpected character '—'.
63:24: Unexpected character '≥'.
67:150: Unexpected character '→'.
72:11: Unexpected character '→'.
73:45: Unexpected character '↔'.
73:55: Unexpected character '→'.
73:139: Unexpected character '≤'.
73:144: Unexpected character '⇒'.
73:175: Unexpected character '≠'.
74:39: Unexpected character '↔'.
74:49: Unexpected character '→'.
74:69: Unexpected character '≤'.
76:34: Unexpected character '#'.
76:38: Unexpected character '≤'.
76:42: Unexpected character '#'.
76:46: Unexpected character '≤'.
76:117: Unexpected character '≥'.
78:27: Char literals must contain exactly one character and close with '''.
80:197: Unexpected character '≤'.
86:6: Unexpected character '→'.
86:10: Unexpected character '→'.
87:6: Too many lexical errors; further errors are not repor ... (message truncated) |
Failed |
IN9 301
OUT7 044 |
| × |
LcbSwap0X1YLiveCodeBench / atcoder / hard: Swap 0^X and 1^Y (arc194_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Swap0X1Y case0: expected [Yes] got [No] |
Failed |
IN9 388
OUT582 |
| × |
LcbSwapAndEraseLiveCodeBench / atcoder / hard: Swap and Erase (arc195_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SwapAndErase case0: expected [3
4
8] got [4
4
11] |
Failed |
IN9 316
OUT4 092 |
| × |
LcbTorusLoopLiveCodeBench / atcoder / hard: Torus Loop (arc196_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TorusLoop case0: expected [2
0
2] got [0
0
0] |
Failed |
IN9 493
OUT1 255 |
| ✓ |
LcbTwiceSubsequenceLiveCodeBench / atcoder / medium: Twice Subsequence (arc195_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed |
IN9 258
OUT394 |
| ✓ |
LcbUniformSumLiveCodeBench / atcoder / hard: Uniform Sum (arc195_b)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed |
IN26 725
OUT7 079 |
Code Generation - LiveCodeBench v6 LeetCode (set 1)
3 passed8 failed |
27.3% |
IN101 311
OUT3 829 |
| ✓ |
LcbCountEvenDiffPartitionsLiveCodeBench 3704 / leetcode / easy: count partitions with even sum difference (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 175
OUT153 |
| × |
LcbCountMentionsPerUserLiveCodeBench 3721 / leetcode / medium: count mentions per user (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: countMentionsPerUser case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
119:14: Unknown variable 'EventsTs'.
120:14: Unknown variable 'EventsTs'.
123:14: Unknown variable 'EventsK'.
124:14: Unknown variable 'EventsK'.
Values:
run.Compiled = false |
Failed |
IN9 295
OUT941 |
| × |
LcbCountNonDecreasingSubarraysLiveCodeBench 3674 / leetcode / hard: count non-decreasing subarrays after k operations (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: countNonDecreasingSubarrays case0 runtime error: List index -1 is out of range. The list has 6 element(s).
Values:
run.Success = false |
Failed |
IN9 214
OUT192 |
| × |
LcbMaxFrequencyAfterOpLiveCodeBench 3751 / leetcode / medium: maximum frequency after subarray operation (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxFrequencyAfterOp case0: expected 2 got 1 |
Failed |
IN9 180
OUT169 |
| × |
LcbMinimizeMaxEdgeWeightLiveCodeBench 3720 / leetcode / medium: minimize the maximum edge weight of graph (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimizeMaxEdgeWeight case2: expected 2 got 1 |
Failed |
IN9 238
OUT576 |
| × |
LcbMinMaxSubarraySumsLiveCodeBench 3725 / leetcode / hard: max and min sums of at most size k subarrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minMaxSubarraySums case0: expected 20 got 14 |
Failed |
IN9 124
OUT175 |
| × |
LcbMinMaxSubsequenceSumsLiveCodeBench 3714 / leetcode / medium: max and min sums of at most size k subsequences (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minMaxSubsequenceSums case0: expected 24 got 16 |
Failed |
IN9 162
OUT302 |
| × |
LcbPaintHouseIVLiveCodeBench 3737 / leetcode / medium: paint house IV (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: paintHouseIV case0: expected 9 got 1000000000000 |
Failed |
IN9 265
OUT315 |
| × |
LcbRobotMaxCoinsLiveCodeBench 3677 / leetcode / medium: maximum amount of money robot can earn (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: robotMaxCoins case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:55: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
25:76: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
30:47: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
30:64: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
35:49: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
35:66: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
60:21: Type 'long' has no supported method named 'Clear'.
61:21: Type 'long' has no supported method named 'Add'.
62:21: Type 'long' has no supported method named 'Add'.
65:41: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
65:62: Type 'long' cannot be indexed with [...]. Only List, Dictionary and string values support indexing.
Values:
run.Compiled = false |
Failed |
IN9 264
OUT666 |
| ✓ |
LcbVariableLengthSubarraySumLiveCodeBench 3731 / leetcode / easy: sum of variable length subarrays (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 143
OUT114 |
| ✓ |
LcbZigzagGridSkipLiveCodeBench 3708 / leetcode / easy: zigzag grid traversal with skip (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 251
OUT226 |
Code Generation - LiveCodeBench v6 LeetCode (set 2)
4 passed6 failed |
40.0% |
IN91 925
OUT2 720 |
| ✓ |
LcbAssignElementsToGroupsLiveCodeBench 3760 / leetcode / medium: assign elements to groups with constraints (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 256
OUT125 |
| × |
LcbCountSubstringsDivLastDigitLiveCodeBench 3696 / leetcode / hard: count substrings divisible by last digit (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: countSubstringsDivLastDigit case0: expected 11 got 8 |
Failed |
IN9 125
OUT154 |
| × |
LcbEatPizzasMaxWeightLiveCodeBench 3779 / leetcode / medium: eat pizzas (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: eatPizzasMaxWeight case0: expected 14 got 27 |
Failed |
IN9 227
OUT190 |
| ✓ |
LcbMaxEvenOddFreqDiffLiveCodeBench 3753 / leetcode / easy: maximum difference between even and odd frequency I (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT225 |
| × |
LcbMaximizeMinGameScoreLiveCodeBench 3762 / leetcode / hard: maximize the minimum game score (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maximizeMinGameScore case0: expected 4 got 6 |
Failed |
IN9 233
OUT731 |
| × |
LcbMaxManhattanAfterKLiveCodeBench 3754 / leetcode / medium: maximum manhattan distance after k changes (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxManhattanAfterK case0: expected 3 got 1 |
Failed |
IN9 233
OUT189 |
| × |
LcbMinIncrementsTargetMultiplesLiveCodeBench 3697 / leetcode / hard: minimum increments for target multiples in an array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minIncrementsTargetMultiples case0: expected 1 got 3 |
Failed |
IN9 159
OUT189 |
| × |
LcbSelectKDisjointSpecialLiveCodeBench 3771 / leetcode / medium: select k disjoint special substrings (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: selectKDisjointSpecial case0: expected true got false |
Failed |
IN9 178
OUT289 |
| ✓ |
LcbSortMatrixDiagonalsLiveCodeBench 3748 / leetcode / medium: sort matrix by diagonals (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 184
OUT348 |
| ✓ |
LcbSpecialSubstringLengthKLiveCodeBench 3709 / leetcode / easy: find special substring of length k (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 167
OUT280 |
Code Generation - LiveCodeBench v6 LeetCode (set 3)
4 passed7 failed |
36.4% |
IN101 405
OUT4 914 |
| × |
LcbChooseKMaxSumLiveCodeBench / leetcode / medium / id 3759: for each i, sum of k largest nums2[j] where nums1[j] < nums1[i]1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: findmaxsum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
41:14: Unknown variable 'nums1'.
42:14: Unknown variable 'nums1'.
Values:
run.Compiled = false |
Failed |
IN9 259
OUT421 |
| ✓ |
LcbFruitsIntoBasketsLiveCodeBench / leetcode / medium / id 3791: count fruit types that remain unplaced into baskets1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 233
OUT217 |
| ✓ |
LcbLargestAlmostMissingLiveCodeBench / leetcode / easy / id 3705: largest integer appearing in exactly one size-k subarray1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 182
OUT230 |
| × |
LcbLexSmallestGeneratedLiveCodeBench / leetcode / hard / id 3770: lexicographically smallest string generated from str1 and str21 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: gencase0 runtime error: String index 2 is out of range. The string has 2 character(s).
Values:
run.Success = false |
Failed |
IN9 238
OUT407 |
| × |
LcbLongestPalinSubseqKOpsLiveCodeBench / leetcode / medium / id 3786: longest palindromic subsequence after at most k letter shifts1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: palinsubseq case0: expected 3 got 10 |
Failed |
IN9 204
OUT622 |
| × |
LcbLongestVDiagonalLiveCodeBench / leetcode / hard / id 3733: length of longest V-shaped diagonal segment1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: vdiag case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 281
OUT758 |
| × |
LcbMaxDistanceSquareLiveCodeBench / leetcode / hard / id 3781: maximize the minimum Manhattan distance among k boundary points1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxdist case0: expected 2 got 0 |
Failed |
IN9 276
OUT777 |
| × |
LcbMaxSubarraysConflictLiveCodeBench / leetcode / hard / id 3789: max non-empty subarrays after removing one conflicting pair1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: maxsub case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
27:33: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed |
IN9 220
OUT570 |
| × |
LcbMaxSumKSubarraysLiveCodeBench / leetcode / medium / id 3722: max sum of k non-overlapping subarrays each length >= m1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxsumksub case0: expected 13 got 12 |
Failed |
IN9 157
OUT531 |
| ✓ |
LcbMaxSumWithLimitsLiveCodeBench / leetcode / medium / id 3764: maximum sum of at most k elements with per-row limits1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 192
OUT226 |
| ✓ |
LcbSameDigitsAfterOpsLiveCodeBench / leetcode / easy / id 3768: check if final two digits are equal after repeated pairwise sums1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT155 |
Code Generation - LiveCodeBench v6 LeetCode (set 4)
5 passed5 failed |
50.0% |
IN92 022
OUT2 946 |
| × |
LcbClosestEqualQueriesLiveCodeBench / leetcode / medium / id 3750: circular min distance to an equal-valued element per query1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: closesteq case0: expected [2, -1, 3] got [1,0,-4] |
Failed |
IN9 205
OUT427 |
| ✓ |
LcbCountBeautifulNumbersLiveCodeBench / leetcode / hard / id 3801: count numbers in [l,r] whose digit-product is divisible by digit-sum1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed |
IN9 157
OUT190 |
| × |
LcbLongestPalinConcatLiveCodeBench / leetcode / medium / id 3793: longest palindrome from a substring of s concatenated with one of t1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: palinconcat case3: expected 5 got 3 |
Failed |
IN9 167
OUT982 |
| ✓ |
LcbMaxContainersShipLiveCodeBench / leetcode / easy / id 3817: max containers of weight w on an n x n deck within maxWeight1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 173
OUT78 |
| ✓ |
LcbMaxUniqueSubarraySumLiveCodeBench / leetcode / easy / id 3788: max sum of a subarray with all-unique elements after deletions1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 176
OUT165 |
| × |
LcbMinCostReachPositionsLiveCodeBench / leetcode / easy / id 3832: min total cost to reach each position via prefix-min swaps1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: mincosts case0: expected [5, 3, 3, 1, 1, 1] got [1,1,1,1,2,2] |
Failed |
IN9 227
OUT101 |
| × |
LcbMinOpsElementsZeroLiveCodeBench / leetcode / hard / id 3744: sum over queries of min pair-divide operations to zero a range1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minops case0: expected 3 got 5 |
Failed |
IN9 180
OUT121 |
| × |
LcbMinTimeBrewPotionsLiveCodeBench / leetcode / medium / id 3794: min synchronized time for n wizards to brew m potions in order1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: mintime case0: expected 110 got 99 |
Failed |
IN9 249
OUT209 |
| ✓ |
LcbPropertiesGraphLiveCodeBench / leetcode / medium / id 3809: connected components where edges need >= k shared distinct values1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 236
OUT437 |
| ✓ |
LcbZeroArrayTransformIVLiveCodeBench / leetcode / medium / id 3795: min k queries to reduce nums to a zero array (subset decrements)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN9 252
OUT236 |
Code Generation - LiveCodeBench v6 LeetCode (set 5)
4 passed7 failed |
36.4% |
IN113 690
OUT6 485 |
| ✓ |
LcbFindValidDigitPairLiveCodeBench / leetcode / easy: find-valid-pair-of-adjacent-digits-in-string (id 3736)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 163
OUT222 |
| × |
LcbLongestSpecialPathLiveCodeBench / leetcode / hard: longest-special-path (id 3687)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: longestSpecialPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed |
IN9 283
OUT737 |
| × |
LcbManhattanArrangementsLiveCodeBench / leetcode / hard: manhattan-distances-of-all-arrangements-of-pieces (id 3739)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: manhattanArrange case1: expected 20 got 10 |
Failed |
IN9 179
OUT469 |
| ✓ |
LcbMaxAdjacentCircularLiveCodeBench / leetcode / easy: maximum-difference-between-adjacent-elements-in-a-circular-array (id 3747)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 106
OUT114 |
| ✓ |
LcbMaxFreeTimeLiveCodeBench / leetcode / medium: reschedule-meetings-for-maximum-free-time-i (id 3743)1 AI query
1 tool callcategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed |
IN21 859
OUT1 896 |
| × |
LcbMaxProductAlternatingSumLiveCodeBench / leetcode / hard: maximum-product-of-subsequences-with-an-alternating-sum-equal-to-k (id 3777)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxProductAlt case0: expected 6 got -1 |
Failed |
IN9 211
OUT719 |
| × |
LcbMinCostGoodCaptionLiveCodeBench / leetcode / hard: minimum-cost-good-caption (id 3701)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minCostGoodCaption case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
36:19: Cannot assign a value of type 'int' to an element of type 'char'.
48:21: Variable 'cnt' has type 'int' but its initializer has type 'char'.
Values:
run.Compiled = false |
Failed |
IN9 214
OUT846 |
| × |
LcbMinOpsKSubarraysEqualLiveCodeBench / leetcode / hard: minimum-operations-to-make-elements-within-k-subarrays-equal (id 3717)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: minOpsKSub case0 runtime error: List index -3 is out of range. The list has 1 element(s).
Values:
run.Success = false |
Failed |
IN9 187
OUT401 |
| × |
LcbMinPairRemovalSortLiveCodeBench / leetcode / easy: minimum-pair-removal-to-sort-array-i (id 3773)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: minPairRemoval case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:34: List constructor expects a List or HashSet to copy, but got 'int'.
Values:
run.Compiled = false |
Failed |
IN9 134
OUT278 |
| × |
LcbSeparateSquaresLiveCodeBench / leetcode / medium: separate-squares-i (id 3763)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: Math.Abs(got - want) < 1e-4
Context: separateSquares case0: expected 1.0 got 3 |
Failed |
IN9 198
OUT637 |
| ✓ |
LcbSumOfGoodNumbersLiveCodeBench / leetcode / easy: sum-of-good-numbers (id 3723)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 156
OUT166 |
Code Generation - LiveCodeBench v6 LeetCode (set 6)
2 passed8 failed |
20.0% |
IN91 912
OUT3 744 |
| × |
LcbCountCopyArraysLiveCodeBench / leetcode / medium: find-the-number-of-copy-arrays (id 3785)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countCopyArrays case0: expected 2 got 1 |
Failed |
IN9 257
OUT170 |
| × |
LcbLongestCommonPrefixRemovalLiveCodeBench / leetcode / hard: longest-common-prefix-of-k-strings-after-removal (id 3784)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: lcpRemoval case0: expected [3, 4, 4, 3, 4] got [0,1,2,3] |
Failed |
IN9 199
OUT539 |
| × |
LcbMaxActiveSectionsTradeLiveCodeBench / leetcode / medium: maximize-active-section-with-trade-i (id 3805)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxActiveTrade case1: expected 4 got 1 |
Failed |
IN9 243
OUT392 |
| × |
LcbMinCostDivideArrayLiveCodeBench / leetcode / hard: minimum-cost-to-divide-array-into-subarrays (id 3765)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: minCostDivide case0 runtime error: List index 1 is out of range. The list has 1 element(s).
Values:
run.Success = false |
Failed |
IN9 247
OUT445 |
| × |
LcbMinCostRemoveElementsLiveCodeBench / leetcode / medium: find-minimum-cost-to-remove-array-elements (id 3776)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minCostRemove case0: expected 12 got 8 |
Failed |
IN9 175
OUT282 |
| × |
LcbPermutationsIVLiveCodeBench / leetcode / hard: permutations-iv (id 3783)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Compiled
Context: permutationsIV case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:50: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
8:39: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
9:20: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
13:38: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
14:20: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
16:78: Operator '+' cannot be applied to 'List<int>' and 'List<int>'. It supports numbers and string/char concatenation.
22:38: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
23:20: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
27:39: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
28:20: Function 'CountAlternatingRemainder' must return 'List<int>', but this return statement has type 'long'.
30:78: Operator '+' cannot be applied to 'List<int>' and 'List<int>'. It supports numbers and string/char concatenation.
44:18: Variable 'countAfter' has type 'long' but its initializer has type 'List<int>'.
62:18: Variable 'countAfter' has type 'long' but its initializer has type 'List<int>'.
97:10: Variable 'totalStartParity0' has type 'long' but its initializer has type 'List<int>'.
98:10: Variable 'totalStartParity1' has type 'long' but its initializer has type 'List<int>'.
Values:
run.Compiled = false |
Failed |
IN9 164
OUT1 090 |
| ✓ |
LcbReverseDegreeLiveCodeBench / leetcode / easy: reverse-degree-of-a-string (id 3811)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 131
OUT81 |
| × |
LcbShortestMatchingSubstringLiveCodeBench / leetcode / hard: shortest-matching-substring (id 3692)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBenchAssertion expression evaluated to false: run.Success
Context: shortestMatching case0 runtime error: Substring(start: 12, length: 2) is out of range for a string of length 13.
Values:
run.Success = false |
Failed |
IN9 192
OUT500 |
| ✓ |
LcbTransformArrayParityLiveCodeBench / leetcode / easy: transform-array-by-parity (id 3778)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed |
IN9 149
OUT78 |
| × |
LcbUnique3DigitEvenLiveCodeBench / leetcode / easy: unique-3-digit-even-numbers (id 3799)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBenchAssertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: unique3Digit case1: expected 2 got 4 |
Failed |
IN9 155
OUT167 |