Benchmark summary
LiveCodeBench adapted to SC#, adding an extra challenge: models must solve newer coding problems while generating correct code in a custom sandboxed language from its provided specification.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Code Generation - LiveCodeBench v1 AtCoder (set 1)
1 passed7 failed |
12.5% |
IN75 096 OUT57 302 | |
| × | LcbAntiDdosCountLiveCodeBench / atcoder abc301_f / hard: Anti (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AntiDdosCount case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
23:28: Expected ';' after the statement, found '++'.
Values:
run.Compiled = false |
Failed | IN9 412 OUT52 273 |
| × | LcbAtCoderCardsLiveCodeBench / atcoder abc301_c / medium: AtCoder Cards (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AtCoderCards case1: expected [Yes] got [No] |
Failed | IN9 344 OUT494 |
| ✓ | LcbAttackCountLiveCodeBench / atcoder abc302_a / easy: Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 298 OUT70 |
| × | LcbBitmaskLeqNLiveCodeBench / atcoder abc301_d / medium: Bitmask (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: BitmaskLeqN case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
80:17: Expected an expression, found ';'.
Values:
run.Compiled = false |
Failed | IN9 372 OUT1 770 |
| × | LcbFillTheGapsLiveCodeBench / atcoder abc301_b / easy: Fill the Gaps (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FillTheGaps case0: expected [2 3 4 5 4 3 2 1 2] got [2 3 4 3 4 5 4 3 4 3 2 3 4 3 2 3 2 1 2] |
Failed | IN9 407 OUT318 |
| × | LcbFindSnukeLiveCodeBench / atcoder abc302_b / medium: Find snuke (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: FindSnuke case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
60:6: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed | IN9 448 OUT556 |
| × | LcbOverallWinnerLiveCodeBench / atcoder abc301_a / easy: Overall Winner (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OverallWinner case1: expected [T] got [A] |
Failed | IN9 358 OUT252 |
| × | LcbPacCandyPathLiveCodeBench / atcoder abc301_e / hard: Pac (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PacCandyPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:24: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
run.Compiled = false |
Failed | IN9 457 OUT1 569 |
Code Generation - LiveCodeBench v1 AtCoder (set 10)
2 passed4 failed2 skipped |
33.3% |
IN56 063 OUT2 780 | |
| × | LcbBlueSpringPassesLiveCodeBench / atcoder / medium: abc318_c Blue Spring (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BlueSpringPasses case0: expected [20] got [30] |
Failed | IN9 347 OUT350 |
| – | LcbBookPrerequisitesLiveCodeBench / atcoder / hard: abc315_e Prerequisites (Pattern B, stdin) FLAG: non-unique output (any valid reading order accepted); exact-string driver can reject a correct answer -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: non-unique output: multiple valid reading orders accepted; exact-string driver would reject correct answers |
Skipped | IN0 OUT0 |
| – | LcbCheckpointShortcutsLiveCodeBench / atcoder / hard: abc315_f Shortcuts (Pattern B, stdin) FLAG: float output (Euclidean distance sum); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: SC# stdin driver: floating-point distance output needs tolerance comparison; exact string compare unsafe |
Skipped | IN0 OUT0 |
| ✓ | LcbFullMoonDaysLiveCodeBench / atcoder / easy: abc318_a Full Moon (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 317 OUT140 |
| × | LcbIceCreamFlavorsLiveCodeBench / atcoder / medium: abc315_c Flavors (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: IceCreamFlavors case0: expected [16] got [18] |
Failed | IN9 341 OUT492 |
| × | LcbMagicalCookiesLiveCodeBench / atcoder / medium: abc315_d Magical Cookies (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MagicalCookies case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:22: Variable 'alive' has type 'List<List<bool>>' but its initializer has type 'List<bool>'.
28:26: Variable 'marked' has type 'List<List<bool>>' but its initializer has type 'List<bool>'.
Values:
run.Compiled = false |
Failed | IN9 381 OUT867 |
| × | LcbMiddleDayLiveCodeBench / atcoder / easy: abc315_b The Middle Day (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MiddleDay case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 359 OUT217 |
| ✓ | LcbOverlappingSheetsLiveCodeBench / atcoder / easy: abc318_b Overlapping sheets (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 318 OUT714 |
Code Generation - LiveCodeBench v1 AtCoder (set 11)
5 passed2 failed1 skipped |
71.4% |
IN65 373 OUT2 871 | |
| ✓ | LcbBusStopsQueriesLiveCodeBench / atcoder / hard: abc319_e Bus Stops (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 436 OUT346 |
| – | LcbFalseHopeProbabilityLiveCodeBench / atcoder / medium: abc319_c False Hope (Pattern B, stdin) FLAG: float output (probability); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Skipped: SC# stdin driver: probability (float) output needs tolerance comparison; exact string compare unsafe |
Skipped | IN0 OUT0 |
| ✓ | LcbLeylandNumberLiveCodeBench / atcoder / easy: abc320_a Leyland Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 282 OUT87 |
| ✓ | LcbLongestPalindromeSubLiveCodeBench / atcoder / easy: abc320_b Longest Palindrome (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 291 OUT233 |
| ✓ | LcbMeasureDivisorsLiveCodeBench / atcoder / easy: abc319_b Measure (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 344 OUT189 |
| × | LcbMinimumWindowWidthLiveCodeBench / atcoder / medium: abc319_d Minimum Width (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MinimumWindowWidth case1: expected [10000000009] got [10000000001] |
Failed | IN9 363 OUT413 |
| ✓ | LcbSandwichTriplesLiveCodeBench / atcoder / hard: abc318_e Sandwiches (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 310 OUT1 375 |
| × | LcbWeightedMaxMatchingLiveCodeBench / atcoder / medium: abc318_d General Weighted Max Matching (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WeightedMaxMatching case0: expected [13] got [15] |
Failed | IN9 347 OUT228 |
Code Generation - LiveCodeBench v1 AtCoder (set 12)
4 passed5 failed |
44.4% |
IN84 299 OUT3 531 | |
| ✓ | Lcb321LikeLiveCodeBench / atcoder / easy: abc321_a 321-like Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 375 OUT112 |
| × | LcbCompleteBinaryTreeLiveCodeBench / atcoder / hard: abc321_e Complete Binary Tree (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CompleteBinaryTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
62:44: The shift count of '<<' must be 'int', but got 'long'.
63:51: The shift count of '<<' must be 'int', but got 'long'.
85:31: The shift count of '<<' must be 'int', but got 'long'.
86:36: The shift count of '<<' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed | IN9 365 OUT1 003 |
| ✓ | LcbCutoffLiveCodeBench / atcoder / easy: abc321_b Cutoff (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 393 OUT246 |
| ✓ | LcbFirstAbc2LiveCodeBench / atcoder / easy: abc322_a First ABC 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 316 OUT117 |
| × | LcbKth321LikeLiveCodeBench / atcoder / medium: abc321_c K-th 321-like Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Kth321Like case1: expected [9610] got [85410] |
Failed | IN9 352 OUT215 |
| × | LcbRelativePositionLiveCodeBench / atcoder / medium: abc320_d Relative Position (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RelativePosition case1: expected [0 0
2 1
-1 -2] got [-2 -1
undecidable
undecidable] |
Failed | IN9 373 OUT513 |
| ✓ | LcbSetMenuLiveCodeBench / atcoder / medium: abc321_d Set Menu (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 351 OUT431 |
| × | LcbSlotStrategy2LiveCodeBench / atcoder / medium: abc320_c Slot Strategy 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SlotStrategy2 case0: expected [6] got [8] |
Failed | IN9 390 OUT420 |
| × | LcbSomenNagashiLiveCodeBench / atcoder / hard: abc320_e Somen Nagashi (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SomenNagashi case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:26: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 384 OUT474 |
Code Generation - LiveCodeBench v1 AtCoder (set 13)
5 passed4 failed |
55.6% |
IN84 329 OUT4 420 | |
| ✓ | LcbFestivalLiveCodeBench / atcoder / medium: abc322_c Festival (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 350 OUT243 |
| × | LcbMergeSlimesLiveCodeBench / atcoder / hard: abc323_d Merge Slimes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MergeSlimes case0: expected [3] got [2] |
Failed | IN9 332 OUT302 |
| × | LcbPlaylistProbLiveCodeBench / atcoder / hard: abc323_e Playlist (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PlaylistProb case0: expected [369720131] got [332748123] |
Failed | IN9 427 OUT673 |
| × | LcbPolyominoLiveCodeBench / atcoder / medium: abc322_d Polyomino (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: Polyomino case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:36: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
run.Compiled = false |
Failed | IN9 404 OUT1 597 |
| ✓ | LcbPrefixSuffixLiveCodeBench / atcoder / easy: abc322_b Prefix and Suffix (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 362 OUT132 |
| ✓ | LcbProductDevelopmentLiveCodeBench / atcoder / hard: abc322_e Product Development (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 365 OUT471 |
| × | LcbRoundRobinRankLiveCodeBench / atcoder / easy: abc323_b Round (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: RoundRobinRank case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
44:18: Type 'int' has no supported method named 'CompareTo'.
46:13: Type 'int' has no supported method named 'CompareTo'.
Values:
run.Compiled = false |
Failed | IN9 373 OUT330 |
| ✓ | LcbWeakBeatsLiveCodeBench / atcoder / easy: abc323_a Weak Beats (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 318 OUT104 |
| ✓ | LcbWorldTourFinalsLiveCodeBench / atcoder / medium: abc323_c World Tour Finals (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 398 OUT568 |
Code Generation - LiveCodeBench v1 AtCoder (set 14)
4 passed2 failed1 skipped |
66.7% |
IN55 860 OUT2 048 | |
| ✓ | Lcb2x3yLiveCodeBench / atcoder / easy: abc324_b 2^x 3^y (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 282 OUT123 |
| ✓ | LcbAllValuesEqualLiveCodeBench / atcoder / easy: abc324_a Same (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 279 OUT121 |
| – | LcbBeautifulPathLiveCodeBench / atcoder / hard: abc324_f Beautiful Path (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: float stdout: stdin driver compares stdout by exact trimmed string, cannot tolerance-match a floating-point ratio like 0.7500000000000000 |
Skipped | IN0 OUT0 |
| ✓ | LcbErrorCorrectionLiveCodeBench / atcoder / medium: abc324_c Error Correction (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 398 OUT692 |
| × | LcbJointTwoStringsLiveCodeBench / atcoder / hard: abc324_e Joint Two Strings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: JointTwoStrings case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:43: Expected '(' after the List type. Lists are created with 'new List<T>()' or 'new List<T>(items)', found '['.
Values:
run.Compiled = false |
Failed | IN9 332 OUT641 |
| × | LcbSquarePermutationLiveCodeBench / atcoder / hard: abc324_d Square Permutation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: SquarePermutation case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 295 OUT430 |
| ✓ | LcbSurnameSanLiveCodeBench / atcoder / easy: abc325_a Takahashi san (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 274 OUT41 |
Code Generation - LiveCodeBench v1 AtCoder (set 15)
5 passed3 failed1 skipped |
62.5% |
IN75 333 OUT3 766 | |
| ✓ | Lcb326LikeNumberLiveCodeBench / atcoder / abc326_b: 3261 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 355 OUT115 |
| – | LcbAbcPuzzleLiveCodeBench / atcoder / abc326_d: ABC Puzzle NOTE: output is NON-UNIQUE (any valid grid is accepted); exact string compare is unsafe -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: non-unique output: any valid grid is accepted, exact string compare unsafe |
Skipped | IN0 OUT0 |
| ✓ | LcbCarTrainTravelLiveCodeBench / atcoder / abc325_e: Our clients, please wait a moment1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 487 OUT859 |
| ✓ | LcbGiftIntervalLiveCodeBench / atcoder / abc326_c: Peak1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 379 OUT212 |
| × | LcbPrintingMachineLiveCodeBench / atcoder / abc325_d: Printing Machine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PrintingMachine case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:23: Expected ')' to close the parenthesized expression, found 'a'.
Values:
run.Compiled = false |
Failed | IN9 416 OUT456 |
| × | LcbSensorClustersLiveCodeBench / atcoder / abc325_c: Sensors1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SensorClusters case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 440 OUT563 |
| × | LcbSensorCostDilemmaLiveCodeBench / atcoder / abc325_f: Sensor Optimization Dilemma1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SensorCostDilemma case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 510 OUT1 117 |
| ✓ | LcbStairsOrElevatorLiveCodeBench / atcoder / abc326_a: 2UP3DOWN1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 331 OUT113 |
| ✓ | LcbWorldMeetingLiveCodeBench / atcoder / abc325_b: World Meeting1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 415 OUT331 |
Code Generation - LiveCodeBench v1 AtCoder (set 16)
4 passed3 failed1 skipped |
57.1% |
IN65 621 OUT2 091 | |
| ✓ | LcbAdjacentABLiveCodeBench / atcoder / abc327_a: ab1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 328 OUT167 |
| × | LcbGoodTupleLiveCodeBench / atcoder / abc327_d: Good Tuple Problem1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: GoodTuple case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:21: Variable 'adj' has type 'List<List<int>>' but its initializer has type 'List<int>'.
Values:
run.Compiled = false |
Failed | IN9 409 OUT418 |
| – | LcbMaximizeRatingLiveCodeBench / atcoder / abc327_e: Maximize Rating NOTE: output is a FLOAT with tolerance 1e-6; exact trimmed string compare is unsafe -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: floating-point output with 1e-6 tolerance; exact string compare unsafe |
Skipped | IN0 OUT0 |
| ✓ | LcbNotTooHardLiveCodeBench / atcoder / abc328_a: Not Too Hard1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 329 OUT124 |
| × | LcbNumberPlaceLiveCodeBench / atcoder / abc327_c: Number Place1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: NumberPlace case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
21:17: Operator '!' requires 'bool', but the operand has type 'void'.
33:17: Operator '!' requires 'bool', but the operand has type 'void'.
51:25: Operator '!' requires 'bool', but the operand has type 'void'.
Values:
run.Compiled = false |
Failed | IN9 427 OUT434 |
| ✓ | LcbPerfectPowerAALiveCodeBench / atcoder / abc327_b: A^A1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 309 OUT149 |
| ✓ | LcbRepdigitDatesLiveCodeBench / atcoder / abc328_b: 11/111 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 346 OUT321 |
| × | LcbSalaryExpectationLiveCodeBench / atcoder / abc326_e: Revenge of "The Salary of AtCoder Inc."1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SalaryExpectation case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:20: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 473 OUT478 |
Code Generation - LiveCodeBench v1 AtCoder (set 17)
4 passed4 failed |
50.0% |
IN74 894 OUT2 486 | |
| ✓ | LcbConsecutivePairsLiveCodeBench / atcoder / abc328_c: Consecutive1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 413 OUT361 |
| × | LcbCountRepeatsLiveCodeBench / atcoder / abc329_c: Count xxx1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountRepeats case0: expected [4] got [10] |
Failed | IN9 328 OUT136 |
| × | LcbElectionReportLiveCodeBench / atcoder / abc329_d: Election Quick Report1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 392 OUT290 |
| × | LcbModuloMstLiveCodeBench / atcoder / abc328_e: Modulo MST1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ModuloMst case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:12: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 424 OUT584 |
| ✓ | LcbSecondLargestLiveCodeBench / atcoder / abc329_b: Next1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 317 OUT231 |
| ✓ | LcbSpreadCharsLiveCodeBench / atcoder / abc329_a: Spread1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 305 OUT82 |
| × | LcbStampMatchLiveCodeBench / atcoder / abc329_e: Stamp1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: StampMatch case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
57:13: Unknown variable 'positionstack'.
Values:
run.Compiled = false |
Failed | IN9 384 OUT604 |
| ✓ | LcbTakeAbcLiveCodeBench / atcoder / abc328_d: Take ABC1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 331 OUT198 |
Code Generation - LiveCodeBench v1 AtCoder (set 18)
4 passed5 failed |
44.4% |
IN84 066 OUT2 838 | |
| × | LcbColoredBallLiveCodeBench / atcoder / hard: Colored Ball [LCB id: abc329_f]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 377 OUT314 |
| × | LcbCountingLsLiveCodeBench / atcoder / medium: Counting Ls [LCB id: abc330_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountingLs case0: expected [4] got [0] |
Failed | IN9 426 OUT482 |
| ✓ | LcbCountingPassesLiveCodeBench / atcoder / easy: Counting Passes [LCB id: abc330_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 296 OUT122 |
| ✓ | LcbEggPacksLiveCodeBench / atcoder / easy: Buy One Carton of Milk (egg packs) [LCB id: abc331_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 308 OUT388 |
| × | LcbMexAndUpdateLiveCodeBench / atcoder / hard: Mex and Update [LCB id: abc330_e]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 382 OUT494 |
| ✓ | LcbMinimizeAbs1LiveCodeBench / atcoder / easy: Minimize Abs 1 [LCB id: abc330_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 370 OUT206 |
| ✓ | LcbMinimizeAbs2LiveCodeBench / atcoder / medium: Minimize Abs 2 [LCB id: abc330_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 276 OUT280 |
| × | LcbSumGreaterThanMeLiveCodeBench / atcoder / medium: Sum of Numbers Greater Than Me [LCB id: abc331_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SumGreaterThanMe case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 318 OUT381 |
| × | LcbTomorrowCalendarLiveCodeBench / atcoder / easy: Tomorrow [LCB id: abc331_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: TomorrowCalendar case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:14: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed | IN9 313 OUT171 |
Code Generation - LiveCodeBench v1 AtCoder (set 19)
4 passed4 failed |
50.0% |
IN75 351 OUT8 912 | |
| ✓ | LcbGlassAndMugLiveCodeBench / atcoder / easy: Glass and Mug [LCB id: abc332_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 399 OUT204 |
| ✓ | LcbOnlineShoppingLiveCodeBench / atcoder / easy: Online Shopping [LCB id: abc332_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 368 OUT185 |
| ✓ | LcbPentagonDiagonalLiveCodeBench / atcoder / easy: Pentagon [LCB id: abc333_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 380 OUT369 |
| × | LcbSetMealLiveCodeBench / atcoder / hard: Set Meal [LCB id: abc331_e]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SetMeal case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
28:45: Expected ';' after the variable declaration, found 'di'.
Values:
run.Compiled = false |
Failed | IN9 436 OUT512 |
| × | LcbSwappingPuzzleLiveCodeBench / atcoder / hard: Swapping Puzzle [LCB id: abc332_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SwappingPuzzle case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
359:5: Expected an expression, found 'struct'.
Values:
run.Compiled = false |
Failed | IN9 494 OUT5 187 |
| ✓ | LcbThreeThreesLiveCodeBench / atcoder / easy: Three Threes [LCB id: abc333_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 272 OUT65 |
| × | LcbTilePatternLiveCodeBench / atcoder / hard: Tile Pattern [LCB id: abc331_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: TilePattern case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
103:23: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 476 OUT2 078 |
| × | LcbTShirtsLiveCodeBench / atcoder / medium: T-Shirts [LCB id: abc332_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TShirts case0: expected [2] got [4] |
Failed | IN9 526 OUT312 |
Code Generation - LiveCodeBench v1 AtCoder (set 2)
3 passed5 failed |
37.5% |
IN75 147 OUT4 097 | |
| × | LcbAlmostEqualRearrangeLiveCodeBench / atcoder abc302_c / medium: Almost Equal (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AlmostEqualRearrange case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
87:22: Type 'List<int>' has no supported method named 'Push' with 1 argument(s).
Values:
run.Compiled = false |
Failed | IN9 350 OUT691 |
| × | LcbDashHealthLiveCodeBench / atcoder abc303_c / medium: Dash (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: DashHealth case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
23:25: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
59:27: Operator '+' cannot be applied to 'int' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed | IN9 442 OUT443 |
| × | LcbDiscordBadMoodLiveCodeBench / atcoder abc303_b / easy: Discord (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: DiscordBadMood case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:48: Expected ';' after the variable declaration, found 'v'.
Values:
run.Compiled = false |
Failed | IN9 367 OUT248 |
| ✓ | LcbImpartialGiftLiveCodeBench / atcoder abc302_d / medium: Impartial Gift (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 392 OUT412 |
| × | LcbIsolationQueriesLiveCodeBench / atcoder abc302_e / hard: Isolation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 379 OUT421 |
| × | LcbMergeSetLiveCodeBench / atcoder abc302_f / hard: Merge Set (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MergeSet case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
78:29: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 434 OUT980 |
| ✓ | LcbShiftVsCapsLockLiveCodeBench / atcoder abc303_d / medium: Shift vs. CapsLock (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 402 OUT673 |
| ✓ | LcbSimilarStringLiveCodeBench / atcoder abc303_a / easy: Similar String (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 381 OUT229 |
Code Generation - LiveCodeBench v1 AtCoder (set 20)
2 passed5 failed1 skipped |
28.6% |
IN65 607 OUT3 150 | |
| × | LcbChristmasColorGridLiveCodeBench / atcoder / hard: Christmas Color Grid 1 [LCB id: abc334_e]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ChristmasColorGrid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
30:26: Expected a parameter name, found 'List'.
Values:
run.Compiled = false |
Failed | IN9 505 OUT1 339 |
| ✓ | LcbChristmasPresentLiveCodeBench / atcoder / easy: Christmas Present [LCB id: abc334_a]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 289 OUT84 |
| ✓ | LcbChristmasTreesLiveCodeBench / atcoder / medium: Christmas Trees [LCB id: abc334_b]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 366 OUT430 |
| × | LcbEraseLeavesLiveCodeBench / atcoder / medium: Erase Leaves [LCB id: abc333_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: EraseLeaves case0: expected [5] got [4] |
Failed | IN9 368 OUT383 |
| × | LcbReindeerSleighLiveCodeBench / atcoder / medium: Reindeer and Sleigh [LCB id: abc334_d]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 384 OUT341 |
| × | LcbRepunitTrioLiveCodeBench / atcoder / medium: Repunit Trio [LCB id: abc333_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: RepunitTrio case2 runtime error: List index 332 is out of range. The list has 210 element(s).
Values:
run.Success = false |
Failed | IN9 310 OUT273 |
| × | LcbSocks2LiveCodeBench / atcoder / medium: Socks 2 [LCB id: abc334_c]1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Socks2 case2: expected [2] got [4] |
Failed | IN9 385 OUT300 |
| – | LcbTakahashiQuestLiveCodeBench / atcoder / hard: Takahashi Quest [LCB id: abc333_e]0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: non-unique output: any valid pick-up sequence achieving K_min is accepted, cannot value-compare against a single oracle line |
Skipped | IN0 OUT0 |
Code Generation - LiveCodeBench v1 AtCoder (set 21)
6 passed2 failed1 skipped |
75.0% |
IN74 648 OUT1 601 | |
| ✓ | Lcb2023To2024LiveCodeBench / atcoder / abc335_a: 2023 -> 2024 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 279 OUT83 |
| ✓ | LcbCtzLiveCodeBench / atcoder / abc336_b: CTZ (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 306 OUT69 |
| ✓ | LcbEvenDigitsLiveCodeBench / atcoder / abc336_c: Even Digits (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 363 OUT138 |
| ✓ | LcbLongLoongLiveCodeBench / atcoder / abc336_a: Long Loong (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 306 OUT87 |
| – | LcbLoongAndTakahashiLiveCodeBench / atcoder / abc335_d: Loong and Takahashi (Pattern B, stdin) [NON-UNIQUE output]0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| × | LcbLoongTrackingLiveCodeBench / atcoder / abc335_c: Loong Tracking (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: LoongTracking case0: expected [3 0
2 0
1 1
1 0
1 0] got [3 0
1 1
2 0
1 0
1 0] |
Failed | IN9 372 OUT483 |
| × | LcbPyramidSequenceLiveCodeBench / atcoder / abc336_d: Pyramid (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PyramidSequence case0: expected [2] got [3] |
Failed | IN9 360 OUT380 |
| ✓ | LcbScoreboardLiveCodeBench / atcoder / abc337_a: Scoreboard (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 325 OUT182 |
| ✓ | LcbTetrahedralNumberLiveCodeBench / atcoder / abc335_b: Tetrahedral Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 337 OUT179 |
Code Generation - LiveCodeBench v1 AtCoder (set 22)
5 passed2 failed1 skipped |
71.4% |
IN65 336 OUT2 417 | |
| – | LcbBadJuiceLiveCodeBench / atcoder / abc337_e: Bad Juice (INTERACTIVE — not expressible as batch stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: SC# subset: interactive problem (two-way judge interaction + non-unique valid output) |
Skipped | IN0 OUT0 |
| ✓ | LcbCapitalizedLiveCodeBench / atcoder / abc338_a: Capitalized? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 273 OUT132 |
| ✓ | LcbCheatingGomokuLiveCodeBench / atcoder / abc337_d: Cheating Gomoku Narabe (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 362 OUT483 |
| ✓ | LcbExtendedAbcLiveCodeBench / atcoder / abc337_b: Extended ABC (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 328 OUT147 |
| ✓ | LcbFrequencyLiveCodeBench / atcoder / abc338_b: Frequency (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 276 OUT215 |
| × | LcbIslandTourLiveCodeBench / atcoder / abc338_d: Island Tour (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: IslandTour case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 414 OUT651 |
| × | LcbLeftoverRecipesLiveCodeBench / atcoder / abc338_c: Leftover Recipes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: LeftoverRecipes case0: expected [5] got [4] |
Failed | IN9 356 OUT491 |
| ✓ | LcbLiningUp2LiveCodeBench / atcoder / abc337_c: Lining Up 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 327 OUT298 |
Code Generation - LiveCodeBench v1 AtCoder (set 23)
5 passed3 failed |
62.5% |
IN74 660 OUT3 248 | |
| ✓ | LcbAppendQueryLiveCodeBench / atcoder / abc340_b: Append (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 315 OUT204 |
| ✓ | LcbArithmeticProgressionLiveCodeBench / atcoder / abc340_a: Arithmetic Progression (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 280 OUT249 |
| ✓ | LcbChordsLiveCodeBench / atcoder / abc338_e: Chords (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 323 OUT316 |
| ✓ | LcbLangtonsTakahashiLiveCodeBench / atcoder / abc339_b: Langton's Takahashi (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 385 OUT607 |
| × | LcbNegativeTspLiveCodeBench / atcoder / abc338_f: Negative Traveling Salesman (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: NegativeTsp case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:22: Variable 'dist' has type 'List<List<long>>' but its initializer has type 'List<long>'.
Values:
run.Compiled = false |
Failed | IN9 374 OUT587 |
| × | LcbPerfectBusLiveCodeBench / atcoder / abc339_c: Perfect Bus (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PerfectBus case0: expected [3] got [2] |
Failed | IN9 322 OUT173 |
| × | LcbSynchronizedPlayersLiveCodeBench / atcoder / abc339_d: Synchronized Players (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SynchronizedPlayers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
56:38: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 372 OUT1 067 |
| ✓ | LcbTldLiveCodeBench / atcoder / abc339_a: TLD (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 289 OUT45 |
Code Generation - LiveCodeBench v1 AtCoder (set 24)
5 passed5 failed |
50.0% |
IN93 808 OUT8 921 | |
| × | LcbAlternatingStringLiveCodeBench / atcoder abc341_e / hard: Alternating String (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AlternatingString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:5: Expected an expression, found 'void'.
Values:
run.Compiled = false |
Failed | IN9 393 OUT2 841 |
| ✓ | LcbBreakdownLiveCodeBench / atcoder abc341_f / hard: Breakdown (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 456 OUT3 808 |
| × | LcbDivideAndDivideLiveCodeBench / atcoder abc340_c / medium: Divide and Divide (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: DivideAndDivide case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 378 OUT150 |
| ✓ | LcbForeignExchangeLiveCodeBench / atcoder abc341_b / easy: Foreign Exchange (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 431 OUT234 |
| × | LcbMancala2LiveCodeBench / atcoder abc340_e / hard: Mancala 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Mancala2 case0: expected [0 4 2 7 2] got [2 4 1 6 2] |
Failed | IN9 433 OUT438 |
| × | LcbOnlyOneOfTwoLiveCodeBench / atcoder abc341_d / medium: Only one of two (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OnlyOneOfTwo case1: expected [5] got [3] |
Failed | IN9 317 OUT311 |
| ✓ | LcbPrint341LiveCodeBench / atcoder abc341_a / easy: Print 341 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 278 OUT110 |
| × | LcbSuperTakahashiBrosLiveCodeBench / atcoder abc340_d / hard: Super Takahashi Bros. (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SuperTakahashiBros case1: expected [90] got [1010] |
Failed | IN9 422 OUT248 |
| ✓ | LcbTakahashiGetsLostLiveCodeBench / atcoder abc341_c / medium: Takahashi Gets Lost (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 407 OUT593 |
| ✓ | LcbYayLiveCodeBench / atcoder abc342_a / easy: Yay! (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 293 OUT188 |
Code Generation - LiveCodeBench v1 AtCoder (set 25)
2 passed5 failed2 skipped |
28.6% |
IN65 737 OUT2 918 | |
| ✓ | LcbAdjacencyMatrixLiveCodeBench / atcoder abc343_b / easy: Adjacency Matrix (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 384 OUT282 |
| × | LcbDiversityOfScoresLiveCodeBench / atcoder abc343_d / medium: Diversity of Scores (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 404 OUT385 |
| × | LcbLastTrainLiveCodeBench / atcoder abc342_e / hard: Last Train (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: LastTrain case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:5: Expected an expression, found 'struct'.
Values:
run.Compiled = false |
Failed | IN9 541 OUT932 |
| × | LcbManyReplacementLiveCodeBench / atcoder abc342_c / medium: Many Replacement (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ManyReplacement case0: expected [recover] got [aecovea] |
Failed | IN9 370 OUT261 |
| ✓ | LcbPalindromicCubeLiveCodeBench / atcoder abc343_c / medium: 343 (palindromic cube) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 315 OUT194 |
| – | LcbSevenCubedLiveCodeBench / atcoder abc343_e / hard: 7x7x7 (Pattern B, stdin) Non-unique output: any valid 9-integer placement is accepted; exact-match oracle cannot judge the Yes case.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: Non-unique output: any valid tuple is accepted for the Yes case; exact-match oracle cannot judge |
Skipped | IN0 OUT0 |
| × | LcbSquarePairLiveCodeBench / atcoder abc342_d / medium: Square Pair (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SquarePair case0: expected [6] got [2] |
Failed | IN9 354 OUT574 |
| × | LcbWhichIsAheadLiveCodeBench / atcoder abc342_b / easy: Which is ahead? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 369 OUT290 |
| – | LcbWrongAnswerLiveCodeBench / atcoder abc343_a / easy: Wrong Answer (Pattern B, stdin) Non-unique output: ANY integer 0..9 not equal to A+B is accepted; exact-match oracle cannot judge it.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Skipped: Non-unique output: any integer 0..9 != A+B is a valid answer; exact-match oracle cannot judge |
Skipped | IN0 OUT0 |
Code Generation - LiveCodeBench v1 AtCoder (set 3)
5 passed4 failed |
55.6% |
IN84 329 OUT4 028 | |
| ✓ | LcbAbcdefgDistLiveCodeBench / atcoder abc305_b / easy: ABCDEFG (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 342 OUT257 |
| × | LcbCookiePickerLiveCodeBench / atcoder abc305_c / medium: Snuke the Cookie Picker (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CookiePicker case0: expected [2 4] got [5 6] |
Failed | IN9 366 OUT402 |
| ✓ | LcbFirstPlayerLiveCodeBench / atcoder abc304_a / easy: First Player (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 338 OUT232 |
| × | LcbGiftFromStarsLiveCodeBench / atcoder abc303_e / hard: A Gift From the Stars (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GiftFromStars case0: expected [2 2] got [5] |
Failed | IN9 394 OUT851 |
| × | LcbGoodGraphLiveCodeBench / atcoder abc304_e / hard: Good Graph (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: GoodGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 390 OUT632 |
| ✓ | LcbPieceOfCakeLiveCodeBench / atcoder abc304_d / medium: A Piece of Cake (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 388 OUT852 |
| ✓ | LcbSubscribersLiveCodeBench / atcoder abc304_b / easy: Subscribers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 446 OUT274 |
| × | LcbVirusSpreadLiveCodeBench / atcoder abc304_c / medium: Virus (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: VirusSpread case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:33: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 337 OUT427 |
| ✓ | LcbWaterStationLiveCodeBench / atcoder abc305_a / easy: Water Station (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 328 OUT101 |
Code Generation - LiveCodeBench v1 AtCoder (set 4)
3 passed5 failed |
37.5% |
IN74 833 OUT5 245 | |
| ✓ | LcbArtGalleryLiveCodeBench / atcoder abc305_e / hard: Art Gallery on Graph (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 384 OUT648 |
| × | LcbBase2LiveCodeBench / atcoder abc306_b / easy: Base 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: Base2 case0 runtime error: Integer overflow.
Values:
run.Success = false |
Failed | IN9 327 OUT115 |
| × | LcbBestPerformancesLiveCodeBench / atcoder abc306_e / hard: Best Performances (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: BestPerformances case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
36:54: Expected ')' to close the argument list, found 'sumTopK'.
Values:
run.Compiled = false |
Failed | IN9 362 OUT649 |
| × | LcbCentersLiveCodeBench / atcoder abc306_c / medium: Centers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: Centers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
32:19: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed | IN9 338 OUT364 |
| ✓ | LcbEchoStringLiveCodeBench / atcoder abc306_a / easy: Echo (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 302 OUT78 |
| × | LcbPoisonousFullLiveCodeBench / atcoder abc306_d / medium: Poisonous Full (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PoisonousFull case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 389 OUT312 |
| × | LcbSleepLogLiveCodeBench / atcoder abc305_d / hard: Sleep Log (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SleepLog case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
145:31: Expected ')' to close the argument list, found ':'.
Values:
run.Compiled = false |
Failed | IN9 391 OUT2 881 |
| ✓ | LcbWeeklyRecordsLiveCodeBench / atcoder abc307_a / easy: Weekly Records (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 340 OUT198 |
Code Generation - LiveCodeBench v1 AtCoder (set 5)
4 passed4 failed |
50.0% |
IN74 827 OUT3 731 | |
| ✓ | LcbDefaultPriceLiveCodeBench / atcoder abc308_b / easy: Default Price (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 343 OUT241 |
| × | LcbDistinctAdjacentLiveCodeBench / atcoder abc307_e / hard: Distinct Adjacent (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DistinctAdjacent case0: expected [6] got [9] |
Failed | IN9 344 OUT304 |
| × | LcbIdealSheetLiveCodeBench / atcoder abc307_c / medium: Ideal Sheet (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: IdealSheet case0 runtime error: Cannot parse '3 5' as int.
Values:
run.Success = false |
Failed | IN9 433 OUT1 548 |
| × | LcbMismatchedParensLiveCodeBench / atcoder abc307_d / medium: Mismatched Parentheses (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MismatchedParens case0: expected [ac] got [a(b))c] |
Failed | IN9 316 OUT244 |
| ✓ | LcbNewSchemeLiveCodeBench / atcoder abc308_a / easy: New Scheme (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 335 OUT198 |
| ✓ | LcbRacecarPalindromeLiveCodeBench / atcoder abc307_b / easy: racecar (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 342 OUT233 |
| × | LcbSnukeMazeLiveCodeBench / atcoder abc308_d / medium: Snuke Maze (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SnukeMaze case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:34: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 384 OUT536 |
| ✓ | LcbStandingsLiveCodeBench / atcoder abc308_c / medium: Standings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 330 OUT427 |
Code Generation - LiveCodeBench v1 AtCoder (set 6)
4 passed5 failed |
44.4% |
IN84 610 OUT4 466 | |
| × | LcbAddEdgeMaxDistanceLiveCodeBench / atcoder / medium: abc309_d Add One Edge1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AddEdgeMaxDistance case0: expected [5] got [1] |
Failed | IN9 487 OUT590 |
| ✓ | LcbDrinkCouponLiveCodeBench / atcoder / easy: abc310_a Order Something Else1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 357 OUT156 |
| ✓ | LcbFamilyInsuranceLiveCodeBench / atcoder / hard: abc309_e Family and Insurance1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 347 OUT495 |
| ✓ | LcbMedicineDayLiveCodeBench / atcoder / medium: abc309_c Medicine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 368 OUT288 |
| × | LcbMexTriplesLiveCodeBench / atcoder / hard: abc308_e MEX1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MexTriples case2: expected [13] got [166] |
Failed | IN9 387 OUT1 198 |
| ✓ | LcbNineAdjacentLiveCodeBench / atcoder / easy: abc309_a Nine1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 356 OUT189 |
| × | LcbRotateOuterLiveCodeBench / atcoder / easy: abc309_b Rotate1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RotateOuter case0: expected [1010
1101
0111
0001] got [0111
1100
0111
0001] |
Failed | IN9 446 OUT711 |
| × | LcbStrictlySuperiorLiveCodeBench / atcoder / easy: abc310_b Strictly Superior1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: StrictlySuperior case1: expected [No] got [Yes] |
Failed | IN9 486 OUT403 |
| × | LcbVouchersDiscountLiveCodeBench / atcoder / hard: abc308_f Vouchers1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: VouchersDiscount case0: expected [4] got [2] |
Failed | IN9 376 OUT436 |
Code Generation - LiveCodeBench v1 AtCoder (set 7)
4 passed3 failed1 skipped |
57.1% |
IN65 867 OUT2 634 | |
| – | LcbFindCycleLiveCodeBench / atcoder / medium: abc311_c Find it! (NON-UNIQUE output: any valid directed cycle is accepted; exact-match unreliable)0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Skipped: non-unique output: any valid directed cycle is accepted, exact string match unreliable |
Skipped | IN0 OUT0 |
| ✓ | LcbFirstAbcLiveCodeBench / atcoder / easy: abc311_a First ABC1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 321 OUT169 |
| × | LcbGridIceFloorLiveCodeBench / atcoder / medium: abc311_d Grid Ice Floor1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: GridIceFloor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:22: Variable 'visited' has type 'List<List<bool>>' but its initializer has type 'List<bool>'.
Values:
run.Compiled = false |
Failed | IN9 479 OUT495 |
| × | LcbMake10AgainLiveCodeBench / atcoder / hard: abc310_f Make 10 Again1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: Make10Again case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 460 OUT555 |
| ✓ | LcbNandRepeatedlyLiveCodeBench / atcoder / hard: abc310_e NAND repeatedly1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 426 OUT208 |
| × | LcbPeacefulTeamsLiveCodeBench / atcoder / medium: abc310_d Peaceful Teams1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PeacefulTeams case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
62:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 403 OUT743 |
| ✓ | LcbReversibleSticksLiveCodeBench / atcoder / medium: abc310_c Reversible1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 397 OUT151 |
| ✓ | LcbVacationTogetherLiveCodeBench / atcoder / easy: abc311_b Vacation Together1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 381 OUT313 |
Code Generation - LiveCodeBench v1 AtCoder (set 8)
5 passed3 failed |
62.5% |
IN75 187 OUT6 464 | |
| ✓ | LcbBecomeStrongestLiveCodeBench / atcoder / easy: abc313_a To Be Saikyo1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 334 OUT208 |
| × | LcbCansOpenersLiveCodeBench / atcoder / hard: abc312_f Cans and Openers1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CansOpeners case0: expected [27] got [12] |
Failed | IN9 425 OUT701 |
| ✓ | LcbChordCheckLiveCodeBench / atcoder / easy: abc312_a Chord1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 286 OUT90 |
| × | LcbCountBracketsLiveCodeBench / atcoder / medium: abc312_d Count Bracket Sequences1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CountBrackets case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 383 OUT515 |
| ✓ | LcbCuboidTangencyLiveCodeBench / atcoder / hard: abc312_e Tangency of Cuboids1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 473 OUT998 |
| × | LcbHolelessSquaresLiveCodeBench / atcoder / hard: abc311_e Defect1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: HolelessSquares case3 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 454 OUT499 |
| ✓ | LcbInvisibleHandLiveCodeBench / atcoder / medium: abc312_c Invisible Hand1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 342 OUT387 |
| ✓ | LcbTakCodeLiveCodeBench / atcoder / easy: abc312_b TaK Code1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 490 OUT3 066 |
Code Generation - LiveCodeBench v1 AtCoder (set 9)
5 passed3 failed1 skipped |
62.5% |
IN74 962 OUT4 661 | |
| × | LcbApproxEqualizationLiveCodeBench / atcoder / medium: abc313_c Approximate Equalization 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ApproxEqualization case0: expected [3] got [4] |
Failed | IN9 321 OUT238 |
| × | LcbCertainGameExpectedLiveCodeBench / atcoder / hard: abc314_f A Certain Game (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CertainGameExpected case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
33:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 436 OUT2 582 |
| ✓ | LcbLowerUpperOpsLiveCodeBench / atcoder / medium: abc314_d LOWER (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 355 OUT403 |
| ✓ | LcbPiTruncateLiveCodeBench / atcoder / easy: abc314_a 3.14 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 400 OUT164 |
| ✓ | LcbRemoveVowelsLiveCodeBench / atcoder / easy: abc315_a tcdr (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 285 OUT167 |
| ✓ | LcbRotateColoredSubsequenceLiveCodeBench / atcoder / medium: abc314_c Rotate Colored Subsequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 384 OUT350 |
| ✓ | LcbRouletteFewestBetsLiveCodeBench / atcoder / easy: abc314_b Roulette (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 401 OUT499 |
| – | LcbRoulettesExpectedLiveCodeBench / atcoder / hard: abc314_e Roulettes (Pattern B, stdin) FLAG: float output (expected value); exact-string driver cannot do tolerance -> SkipTest.0 AI queries
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Hardsource:LiveCodeBench Skipped: SC# stdin driver: float/expected-value output needs tolerance comparison; exact string compare unsafe |
Skipped | IN0 OUT0 |
| × | LcbStrongestProgrammerLiveCodeBench / atcoder / medium: abc313_b Who is Saikyo? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: StrongestProgrammer case0: expected [1] got [-1] |
Failed | IN9 380 OUT258 |
Code Generation - LiveCodeBench v1 Codeforces (set 1)
5 passed4 failed |
55.6% |
IN84 506 OUT3 058 | |
| × | LcbChemistryPalindromeLiveCodeBench / codeforces 1883_B / medium: Chemistry (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ChemistryPalindrome case0: expected [YES
NO
YES
YES
YES
YES
NO
NO
YES
YES
YES
YES
NO
YES] got [YES
YES
NO
NO
NO
YES
YES
YES
YES
NO
NO
NO
NO
NO] |
Failed | IN9 382 OUT365 |
| ✓ | LcbGameWithIntegersLiveCodeBench / codeforces 1899_A / easy: Game with Integers (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBench |
Passed | IN9 387 OUT186 |
| ✓ | LcbGoodKidLiveCodeBench / codeforces 1873_B / easy: Good Kid (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBench |
Passed | IN9 376 OUT291 |
| ✓ | LcbOneDEraserLiveCodeBench / codeforces 1873_D / easy: 1D Eraser (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBench |
Passed | IN9 372 OUT220 |
| × | LcbRaspberriesLiveCodeBench / codeforces 1883_C / medium: Raspberries (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Raspberries case0: expected [2
2
1
0
2
0
1
2
0
1
1
4
0
4
3] got [1
2
1
0
2
1
1
2
0
2
2
2
0
2
2] |
Failed | IN9 380 OUT627 |
| ✓ | LcbShortSortLiveCodeBench / codeforces 1873_A / easy: Short Sort (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Easysource:LiveCodeBench |
Passed | IN9 359 OUT213 |
| × | LcbTruckWeightDifferenceLiveCodeBench / codeforces 1899_B / hard: 250 Thousand Tons of TNT (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 438 OUT481 |
| × | LcbYarikArrayAltParityLiveCodeBench / codeforces 1899_C / hard: Yarik and Array (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: YarikArrayAltParity case0: expected [15
17
8
4
-1000
101
10] got [7
25
5
5
-1000
101
11] |
Failed | IN9 400 OUT337 |
| ✓ | LcbYarikMusicalNotesLiveCodeBench / codeforces 1899_D / hard: Yarik and Musical Notes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@Codeforceslevel:Hardsource:LiveCodeBench |
Passed | IN9 412 OUT338 |
Code Generation - LiveCodeBench v1 LeetCode (set 1)
7 passed2 failed |
77.8% |
IN83 806 OUT4 271 | |
| ✓ | LcbBuyChocoLiveCodeBench / leetcode / easy: buy-two-chocolates (LCB 2756)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 288 OUT164 |
| ✓ | LcbColorTheArrayLiveCodeBench / leetcode / medium: number-of-adjacent-elements-with-the-same-color (LCB 2779)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 365 OUT276 |
| × | LcbCountOfIntegersLiveCodeBench / leetcode / hard: count-of-integers (LCB 2757)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: count case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:18: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 332 OUT2 443 |
| ✓ | LcbCountSeniorsLiveCodeBench / leetcode / easy: number-of-senior-citizens (LCB 2727)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 368 OUT87 |
| ✓ | LcbDistinctDifferenceArrayLiveCodeBench / leetcode / easy: find-the-distinct-difference-array (LCB 2777)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 309 OUT253 |
| ✓ | LcbMatrixSumLiveCodeBench / leetcode / medium: sum-in-a-matrix (LCB 2728)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 283 OUT227 |
| ✓ | LcbMaximumOrLiveCodeBench / leetcode / medium: maximum-or (LCB 2730)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 295 OUT201 |
| × | LcbMaxStrengthLiveCodeBench / leetcode / medium: maximum-strength-of-a-group (LCB 2754)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxStrength case1: expected 20 got -5 |
Failed | IN9 274 OUT435 |
| ✓ | LcbMinExtraCharLiveCodeBench / leetcode / medium: extra-characters-in-a-string (LCB 2755)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 292 OUT185 |
Code Generation - LiveCodeBench v1 LeetCode (set 10)
6 passed3 failed |
66.7% |
IN84 190 OUT5 403 | |
| ✓ | LcbCanBeEqualSwapLiveCodeBench / leetcode / easy: check-if-strings-can-be-made-equal-with-operations-i (id 2999)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 314 OUT149 |
| ✓ | LcbCanMakeSubsequenceCyclicLiveCodeBench / leetcode / medium: make-string-a-subsequence-using-cyclic-increments (id 3018)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 379 OUT209 |
| × | LcbConstructProductMatrixLiveCodeBench / leetcode / medium: construct-product-matrix (id 3031)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<List<List<int>>>(run.ResultJson) == FromJson<List<List<int>>>(expected[i])
Context: constructProduct case0: expected [[24, 12], [8, 6]] got [[24,12],[24,6]] |
Failed | IN9 328 OUT599 |
| ✓ | LcbFurthestDistanceFromOriginLiveCodeBench / leetcode / easy: furthest-point-from-origin (id 3019)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 326 OUT154 |
| ✓ | LcbMaxBallPassingValueLiveCodeBench / leetcode / hard: maximize-value-of-function-in-a-ball-passing-game (id 3032)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 440 OUT357 |
| × | LcbMinAbsoluteDifferenceApartLiveCodeBench / leetcode / medium: minimum-absolute-difference-between-elements-with-constraint (id 3000)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minAbsDiff case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:19: Expected ';' after the statement, found ','.
Values:
run.Compiled = false |
Failed | IN9 336 OUT703 |
| × | LcbMinOperationsSubsequenceSumLiveCodeBench / leetcode / hard: minimum-operations-to-form-subsequence-with-target-sum (id 3025)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minOpsSubseqSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
73:27: Cannot assign a value of type 'long' to an element of type 'int'.
77:27: Cannot assign a value of type 'long' to an element of type 'int'.
Values:
run.Compiled = false |
Failed | IN9 393 OUT519 |
| ✓ | LcbNumberOfBeautifulIntegersLiveCodeBench / leetcode / hard: number-of-beautiful-integers-in-the-range (id 3017)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 314 OUT220 |
| ✓ | LcbStringTransformationWaysLiveCodeBench / leetcode / hard: string-transformation (id 3024)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 360 OUT2 493 |
Code Generation - LiveCodeBench v1 LeetCode (set 11)
3 passed6 failed |
33.3% |
IN83 917 OUT2 049 | |
| × | LcbMaximumCompleteSubsetSumLiveCodeBench / leetcode / hard: maximum-element-sum-of-a-complete-subset-of-indices (id 3047)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxCompleteSubset case0: expected 16 got 14 |
Failed | IN9 366 OUT319 |
| ✓ | LcbMaximumOddBinaryNumberLiveCodeBench / leetcode / easy: maximum-odd-binary-number (id 3055)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 282 OUT178 |
| × | LcbMinArrayLengthAfterRemovalsLiveCodeBench / leetcode / medium: minimum-array-length-after-pair-removals (id 3081)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minLengthRemovals case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minLengthAfterRemovals' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinLengthAfterRemovals.
Values:
run.Compiled = false |
Failed | IN9 340 OUT160 |
| × | LcbMinCostMakeStringsEqualLiveCodeBench / leetcode / medium: apply-operations-to-make-two-strings-equal (id 3033)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minCostEqual case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minOperations' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinOperations.
Values:
run.Compiled = false |
Failed | IN9 413 OUT349 |
| × | LcbMinimumRightShiftsLiveCodeBench / leetcode / easy: minimum-right-shifts-to-sort-the-array (id 3045)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minRightShifts case0: expected 2 got -1 |
Failed | IN9 291 OUT234 |
| ✓ | LcbMinOperationsCollectElementsLiveCodeBench / leetcode / easy: minimum-operations-to-collect-elements (id 3044)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 302 OUT113 |
| × | LcbMinOperationsSpecialNumberLiveCodeBench / leetcode / medium: minimum-operations-to-make-a-special-number (id 3046)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minOpsSpecial case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumOperations' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumOperations.
Values:
run.Compiled = false |
Failed | IN9 290 OUT350 |
| × | LcbNumberOfPointsCarsLiveCodeBench / leetcode / easy: points-that-intersect-with-cars (id 3034)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfPoints case0: expected 7 got 13 |
Failed | IN9 301 OUT85 |
| ✓ | LcbSplitArrayMaxSubarraysLiveCodeBench / leetcode / medium: split-array-into-maximum-number-of-subarrays (id 3080)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 332 OUT261 |
Code Generation - LiveCodeBench v1 LeetCode (set 12)
3 passed5 failed |
37.5% |
IN74 780 OUT2 747 | |
| × | LcbBeautifulTowersMaxSumLiveCodeBench / leetcode / medium: beautiful-towers-i (id 3114)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: beautifulTowers case0 runtime error: List index -1 is out of range. The list has 5 element(s).
Values:
run.Success = false |
Failed | IN9 396 OUT393 |
| × | LcbCountSubMultisetsBoundedSumLiveCodeBench / leetcode / hard: count-of-sub-multisets-with-bounded-sum (id 3091)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: countSubMultisets case0 runtime error: List index 1 is out of range. The list has 1 element(s).
Values:
run.Success = false |
Failed | IN9 423 OUT682 |
| × | LcbCountWaysHappyStudentsLiveCodeBench / leetcode / medium: happy-students (id 3104)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countWaysHappy case0: expected 2 got 1 |
Failed | IN9 332 OUT232 |
| ✓ | LcbLongestSubsequenceSumTargetLiveCodeBench / leetcode / medium: length-of-the-longest-subsequence-that-sums-to-target (id 3106)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 332 OUT234 |
| ✓ | LcbMinOperationsMakeArrayEmptyLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-array-empty (id 3094)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 307 OUT150 |
| × | LcbMinSizeSubarrayInfiniteLiveCodeBench / leetcode / medium: minimum-size-subarray-in-infinite-array (id 3141)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minSizeSubarray case2: expected -1 got 1 |
Failed | IN9 313 OUT612 |
| × | LcbShortestBeautifulSubstringLiveCodeBench / leetcode / medium: shortest-and-lexicographically-smallest-beautiful-string (id 3150)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: shortestBeautiful case0: expected "11001" got "100011" |
Failed | IN9 357 OUT300 |
| ✓ | LcbSumIndicesKSetBitsLiveCodeBench / leetcode / easy: sum-of-values-at-indices-with-k-set-bits (id 3093)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 320 OUT144 |
Code Generation - LiveCodeBench v1 LeetCode (set 13)
8 passed1 failed |
88.9% |
IN83 522 OUT3 054 | |
| ✓ | LcbDivisibleSumsDifferenceLiveCodeBench / leetcode / easy: 3172 divisible-and-non-divisible-sums-difference (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 248 OUT121 |
| ✓ | LcbLastVisitedIntegersLiveCodeBench / leetcode / easy: 3164 last-visited-integers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 332 OUT185 |
| ✓ | LcbMaxOrderedTripletValueLiveCodeBench / leetcode / easy: 3154 maximum-value-of-an-ordered-triplet-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 261 OUT205 |
| ✓ | LcbMaxSumOfSquaresLiveCodeBench / leetcode / hard: 3153 apply-operations-on-array-to-maximize-sum-of-squares (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 318 OUT906 |
| ✓ | LcbMinChangesBinaryBeautifulLiveCodeBench / leetcode / medium: 3174 minimum-number-of-changes-to-make-binary-string-beautiful (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 254 OUT81 |
| ✓ | LcbMinEqualSumReplaceZerosLiveCodeBench / leetcode / medium: 3171 minimum-equal-sum-of-two-arrays-after-replacing-zeros (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 251 OUT449 |
| × | LcbMinGroupsValidAssignmentLiveCodeBench / leetcode / medium: 3166 minimum-number-of-groups-to-create-a-valid-assignment (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinGroupsValidAssignment case1: expected 4 got -1 |
Failed | IN9 283 OUT658 |
| ✓ | LcbMinProcessingTimeLiveCodeBench / leetcode / medium: 3151 minimum-processing-time (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 323 OUT333 |
| ✓ | LcbSumDistinctSquaresLiveCodeBench / leetcode / easy: 3163 subarrays-distinct-element-sum-of-squares-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 252 OUT116 |
Code Generation - LiveCodeBench v1 LeetCode (set 14)
5 passed4 failed |
55.6% |
IN83 395 OUT2 864 | |
| ✓ | LcbFindChampionILiveCodeBench / leetcode / easy: 3188 find-champion-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 280 OUT84 |
| ✓ | LcbFindKOrLiveCodeBench / leetcode / easy: 3183 find-the-k-or-of-an-array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 271 OUT129 |
| ✓ | LcbFindWordsContainingLiveCodeBench / leetcode / easy: 3194 find-words-containing-character (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 228 OUT84 |
| × | LcbMaxBalancedSubseqSumLiveCodeBench / leetcode / hard: 3184 maximum-balanced-subsequence-sum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxBalancedSubseqSum case0: expected 14 got 9 |
Failed | IN9 289 OUT955 |
| × | LcbMaximumXorProductLiveCodeBench / leetcode / medium: 3192 maximum-xor-product (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaximumXorProduct case0: expected 98 got 52 |
Failed | IN9 251 OUT262 |
| × | LcbMaxStrongPairXorLiveCodeBench / leetcode / easy: 3193 maximum-strong-pair-xor-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MaxStrongPairXor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'maximumStrongPairXor' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MaximumStrongPairXor.
Values:
run.Compiled = false |
Failed | IN9 279 OUT586 |
| ✓ | LcbMinMountainTripletSumLiveCodeBench / leetcode / easy: 3176 minimum-sum-of-mountain-triplets-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 248 OUT259 |
| ✓ | LcbMinOpsMaximizeLastLiveCodeBench / leetcode / medium: 3190 minimum-operations-to-maximize-last-elements-in-arrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 294 OUT402 |
| × | LcbSeparateBlackWhiteBallsLiveCodeBench / leetcode / medium: 3195 separate-black-and-white-balls (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: SeparateBlackWhiteBalls case1: expected 2 got 0 |
Failed | IN9 255 OUT103 |
Code Generation - LiveCodeBench v1 LeetCode (set 15)
5 passed3 failed |
62.5% |
IN74 210 OUT3 897 | |
| ✓ | LcbCountBeautifulSubstringsLiveCodeBench / leetcode / medium: 3210 count-beautiful-substrings-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 275 OUT245 |
| ✓ | LcbDistributeCandiesILiveCodeBench / leetcode / easy: 3199 distribute-candies-among-children-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 216 OUT371 |
| ✓ | LcbFindCommonElementsLiveCodeBench / leetcode / easy: 3206 find-common-elements-between-two-arrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 291 OUT198 |
| × | LcbGoodStringsLeetLiveCodeBench / leetcode / medium: 3200 number-of-strings-which-can-be-rearranged-to-contain-substring (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: GoodStringsLeet case0: expected 12 got 1 |
Failed | IN9 247 OUT1 896 |
| ✓ | LcbMakeThreeStringsEqualLiveCodeBench / leetcode / easy: 3207 make-three-strings-equal (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 278 OUT241 |
| × | LcbMaxFrequencyScoreLiveCodeBench / leetcode / hard: 3196 apply-operations-to-maximize-frequency-score (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxFrequencyScore case0: expected 3 got 2 |
Failed | IN9 259 OUT234 |
| × | LcbMinCoinsForFruitsLiveCodeBench / leetcode / medium: 3209 minimum-number-of-coins-for-fruits (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinCoinsForFruits case1: expected 2 got 11 |
Failed | IN9 280 OUT230 |
| ✓ | LcbPalindromeRearrangeQueriesLiveCodeBench / leetcode / hard: 3203 palindrome-rearrangement-queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 364 OUT482 |
Code Generation - LiveCodeBench v1 LeetCode (set 16)
6 passed3 failed |
66.7% |
IN83 524 OUT3 274 | |
| ✓ | LcbCompleteSubstringsLiveCodeBench / leetcode / hard: count-complete-substrings (id 3223)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 275 OUT480 |
| ✓ | LcbCountTestedDevicesLiveCodeBench / leetcode / easy: count-tested-devices-after-test-operations (id 3220)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 322 OUT108 |
| ✓ | LcbFindPeaksLiveCodeBench / leetcode / easy: find-the-peaks (id 3221)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 247 OUT107 |
| ✓ | LcbGoodPartitionsCountLiveCodeBench / leetcode / hard: count-the-number-of-good-partitions (id 3212)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 248 OUT335 |
| × | LcbInfectionSequencesLiveCodeBench / leetcode / hard: count-the-number-of-infection-sequences (id 3224)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: infectionSeq case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
25:14: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 347 OUT891 |
| × | LcbLexSmallestSwapLiveCodeBench / leetcode / medium: make-lexicographically-smallest-array-by-swapping-elements (id 3219)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: lexSwap case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:39: Expected '(' after the List type. Lists are created with 'new List<T>()' or 'new List<T>(items)', found '{'.
Values:
run.Compiled = false |
Failed | IN9 304 OUT587 |
| ✓ | LcbLongestSubarrayKFreqLiveCodeBench / leetcode / medium: length-of-longest-subarray-with-at-most-k-frequency (id 3225)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 263 OUT247 |
| × | LcbMaxNonDecreasingLengthLiveCodeBench / leetcode / hard: find-maximum-non-decreasing-array-length (id 3211)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxNonDec case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'findMaximumLength' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, FindMaximumLength.
Values:
run.Compiled = false |
Failed | IN9 287 OUT346 |
| ✓ | LcbSubarraysMaxKTimesLiveCodeBench / leetcode / medium: count-subarrays-where-max-element-appears-at-least-k-times (id 3213)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 231 OUT173 |
Code Generation - LiveCodeBench v1 LeetCode (set 17)
5 passed4 failed |
55.6% |
IN83 626 OUT3 372 | |
| ✓ | LcbDoubleModExpLiveCodeBench / leetcode / medium: double-modular-exponentiation (id 3234)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 280 OUT312 |
| × | LcbEqualindromicCostLiveCodeBench / leetcode / medium: minimum-cost-to-make-array-equalindromic (id 3229)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: equalindromic case2: expected 22 got 30 |
Failed | IN9 332 OUT1 189 |
| × | LcbMaxPartitionsOpsLiveCodeBench / leetcode / hard: maximize-the-number-of-partitions-after-operations (id 3233)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxPartOps case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:34: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 296 OUT419 |
| × | LcbMaxSetSizeRemovalsLiveCodeBench / leetcode / medium: maximum-size-of-a-set-after-removals (id 3228)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxSetSize case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'maximumSetSize' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MaximumSetSize.
Values:
run.Compiled = false |
Failed | IN9 261 OUT278 |
| × | LcbMinCoinsToAddLiveCodeBench / leetcode / medium: minimum-number-of-coins-to-be-added (id 3231)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minCoins case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumAddedCoins' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumAddedCoins.
Values:
run.Compiled = false |
Failed | IN9 293 OUT120 |
| ✓ | LcbMinCostConvertStringLiveCodeBench / leetcode / medium: minimum-cost-to-convert-string-i (id 3235)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 364 OUT597 |
| ✓ | LcbMinimumNumberGameLiveCodeBench / leetcode / easy: minimum-number-game (id 3226)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 269 OUT83 |
| ✓ | LcbMissingAndRepeatedLiveCodeBench / leetcode / easy: find-missing-and-repeated-values (id 3227)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 283 OUT220 |
| ✓ | LcbRemoveAlmostEqualLiveCodeBench / leetcode / medium: remove-adjacent-almost-equal-characters (id 3230)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 248 OUT154 |
Code Generation - LiveCodeBench v1 LeetCode (set 18)
5 passed3 failed |
62.5% |
IN74 286 OUT3 962 | |
| ✓ | LcbBeautifulIndicesLiveCodeBench / leetcode / medium: find-beautiful-indices-in-the-given-array-i (id 3245)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 304 OUT455 |
| ✓ | LcbBitwiseOrTrailingZerosLiveCodeBench / leetcode / easy: check-if-bitwise-or-has-trailing-zeros (id 3246)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 289 OUT87 |
| ✓ | LcbMaxFrequencyElementsLiveCodeBench / leetcode / easy: count-elements-with-maximum-frequency (id 3242)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 223 OUT194 |
| × | LcbMaxNumberPricesSumLiveCodeBench / leetcode / medium: maximum-number-that-sum-of-the-prices-is-less-than-or-equal-to-k (id 3240)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxNumberPrices case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
32:16: The shift count of '<<' must be 'int', but got 'long'.
46:25: The shift count of '<<' must be 'int', but got 'long'.
49:28: The shift count of '<<' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed | IN9 323 OUT376 |
| × | LcbMinArrayLengthModLiveCodeBench / leetcode / medium: minimize-length-of-array-using-operations (id 3244)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minArrayLen case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumArrayLength' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumArrayLength.
Values:
run.Compiled = false |
Failed | IN9 286 OUT256 |
| × | LcbMinOpsXYEqualLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-x-and-y-equal (id 3239)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minOpsXY case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumOperationsToMakeEqual' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumOperationsToMakeEqual.
Values:
run.Compiled = false |
Failed | IN9 263 OUT517 |
| ✓ | LcbMissingSeqPrefixSumLiveCodeBench / leetcode / easy: smallest-missing-integer-greater-than-sequential-prefix-sum (id 3236)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 262 OUT201 |
| ✓ | LcbPowerfulIntegersLiveCodeBench / leetcode / hard: count-the-number-of-powerful-integers (id 3243)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 336 OUT1 876 |
Code Generation - LiveCodeBench v1 LeetCode (set 19)
5 passed4 failed |
55.6% |
IN83 457 OUT2 244 | |
| ✓ | LcbAreaOfMaxDiagonalLiveCodeBench / leetcode / easy: maximum-area-of-longest-diagonal-rectangle (id 3251)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 260 OUT194 |
| ✓ | LcbCountMatchingSubarraysLiveCodeBench / leetcode / medium: number-of-subarrays-that-match-a-pattern-i (id 3269)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 333 OUT249 |
| × | LcbIncremovableSubarrayCountLiveCodeBench / leetcode / easy: count-the-number-of-incremovable-subarrays-i (id 3252)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: incremovable case0: expected 10 got 4 |
Failed | IN9 253 OUT310 |
| × | LcbLargestPerimeterLiveCodeBench / leetcode / medium: find-polygon-with-the-largest-perimeter (id 3262)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: largestPerimeter case1: expected 12 got -1 |
Failed | IN9 319 OUT134 |
| ✓ | LcbLongestSpecialThriceLiveCodeBench / leetcode / medium: find-longest-special-substring-that-occurs-thrice-i (id 3267)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 247 OUT433 |
| × | LcbMaximumGoodSubarraySumLiveCodeBench / leetcode / medium: maximum-good-subarray-sum (id 3265)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxGoodSub case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'maximumSubarraySum' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MaximumSubarraySum.
Values:
run.Compiled = false |
Failed | IN9 264 OUT210 |
| ✓ | LcbMinimumCostThreeSubarraysLiveCodeBench / leetcode / easy: divide-an-array-into-subarrays-with-minimum-cost-i (id 3263)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 228 OUT138 |
| ✓ | LcbMinOpsXorEqualKLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-array-xor-equal-to-k (id 3249)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 271 OUT115 |
| × | LcbMinOrAfterOperationsLiveCodeBench / leetcode / hard: minimize-or-of-remaining-elements-using-operations (id 3261)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minOr case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minOrAfterOperations' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinOrAfterOperations.
Values:
run.Compiled = false |
Failed | IN9 282 OUT461 |
Code Generation - LiveCodeBench v1 LeetCode (set 2)
7 passed2 failed |
77.8% |
IN83 682 OUT1 666 | |
| ✓ | LcbCircularGameLosersLiveCodeBench / leetcode / easy: find-the-losers-of-the-circular-game (LCB 2791)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 316 OUT157 |
| ✓ | LcbCollectingChocolatesLiveCodeBench / leetcode / medium: collecting-chocolates (LCB 2810)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 314 OUT248 |
| ✓ | LcbLongestSemiRepetitiveLiveCodeBench / leetcode / medium: find-the-longest-semi-repetitive-substring (LCB 2786)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 291 OUT176 |
| ✓ | LcbMinLengthRemoveSubstrLiveCodeBench / leetcode / easy: minimum-string-length-after-removing-substrings (LCB 2800)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 259 OUT148 |
| × | LcbNeighboringXorLiveCodeBench / leetcode / medium: neighboring-bitwise-xor (LCB 2792)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: doesValidArrayExist case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'doesValidArrayExist' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, DoesValidArrayExist.
Values:
run.Compiled = false |
Failed | IN9 319 OUT78 |
| ✓ | LcbPaintWallsLiveCodeBench / leetcode / hard: painting-the-walls (LCB 2808)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 325 OUT257 |
| × | LcbPowerOfHeroesLiveCodeBench / leetcode / hard: power-of-heroes (LCB 2784)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: sumOfPower case0: expected 141 got 73 |
Failed | IN9 286 OUT167 |
| ✓ | LcbPunishmentNumberLiveCodeBench / leetcode / medium: find-the-punishment-number-of-an-integer (LCB 2802)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 279 OUT276 |
| ✓ | LcbSemiOrderedPermutationLiveCodeBench / leetcode / easy: semi-ordered-permutation (LCB 2785)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 293 OUT159 |
Code Generation - LiveCodeBench v1 LeetCode (set 20)
4 passed4 failed |
50.0% |
IN74 205 OUT3 107 | |
| ✓ | LcbCanSortArrayBySetBitsLiveCodeBench / leetcode / medium: find-if-array-can-be-sorted (id 3291)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 247 OUT399 |
| ✓ | LcbCountPrefixSuffixPairsLiveCodeBench / leetcode / easy: count-prefix-and-suffix-pairs-i (id 3309)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 263 OUT135 |
| × | LcbEarliestSecondToMarkLiveCodeBench / leetcode / medium: earliest-second-to-mark-indices-i (id 3292)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: earliestSecond case0: expected 8 got 7 |
Failed | IN9 324 OUT517 |
| ✓ | LcbFlowerGameLiveCodeBench / leetcode / medium: alice-and-bob-playing-flower-game (id 3279)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 279 OUT102 |
| ✓ | LcbLastNonEmptyStringLiveCodeBench / leetcode / medium: apply-operations-to-make-string-empty (id 3308)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 231 OUT278 |
| × | LcbMaxSelectedConsecutiveLiveCodeBench / leetcode / hard: maximize-consecutive-elements-in-an-array-after-modification (id 3298)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSelected case0: expected 3 got 2 |
Failed | IN9 278 OUT365 |
| × | LcbMaxSubsetPowerPatternLiveCodeBench / leetcode / medium: find-the-maximum-number-of-elements-in-subset (id 3299)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxSubsetPower case0: expected 3 got 1 |
Failed | IN9 325 OUT401 |
| × | LcbMinTimeRevertWordLiveCodeBench / leetcode / medium: minimum-time-to-revert-word-to-initial-state-i (id 3297)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minTimeRevert case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumTimeToInitialState' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumTimeToInitialState.
Values:
run.Compiled = false |
Failed | IN9 258 OUT910 |
Code Generation - LiveCodeBench v1 LeetCode (set 21)
8 passed |
100.0% |
IN74 222 OUT1 585 | |
| ✓ | LcbAntOnBoundaryLiveCodeBench / leetcode / easy: ant-on-the-boundary (id 3311)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 286 OUT76 |
| ✓ | LcbCountKeyChangesLiveCodeBench / leetcode / easy: number-of-changing-keys (id 3312)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 256 OUT167 |
| ✓ | LcbDistributeTwoArraysLiveCodeBench / leetcode / easy: distribute-elements-into-two-arrays-i (id 3347)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 310 OUT185 |
| ✓ | LcbLongestCommonPrefixDigitsLiveCodeBench / leetcode / medium: find-the-length-of-the-longest-common-prefix (id 3329)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 322 OUT203 |
| ✓ | LcbMaxOperationsSameScoreLiveCodeBench / leetcode / easy: maximum-number-of-operations-with-the-same-score-i (id 3320)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 256 OUT148 |
| ✓ | LcbMaxPalindromesAfterOpsLiveCodeBench / leetcode / medium: maximum-palindromes-after-operations (id 3317)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 299 OUT588 |
| ✓ | LcbMinOpsExceedThresholdLiveCodeBench / leetcode / easy: minimum-operations-to-exceed-threshold-value-i (id 3331)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 239 OUT62 |
| ✓ | LcbSplitArrayDistinctLiveCodeBench / leetcode / easy: split-the-array (id 3324)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 254 OUT156 |
Code Generation - LiveCodeBench v1 LeetCode (set 3)
5 passed3 failed |
62.5% |
IN74 191 OUT1 704 | |
| × | LcbGcdTraversalLiveCodeBench / leetcode / hard: greatest-common-divisor-traversal (LCB 2827)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: canTraverseAllPairs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
35:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 301 OUT651 |
| × | LcbIsFascinatingLiveCodeBench / leetcode / easy: check-if-the-number-is-fascinating (LCB 2824)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: isFascinating case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:13: Operator '!' requires 'bool', but the operand has type 'void'.
Values:
run.Compiled = false |
Failed | IN9 268 OUT134 |
| ✓ | LcbKAvoidingArrayLiveCodeBench / leetcode / medium: determine-the-minimum-sum-of-a-k-avoiding-array (LCB 2811)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 255 OUT154 |
| ✓ | LcbMaxAchievableNumberLiveCodeBench / leetcode / easy: find-the-maximum-achievable-number (LCB 2812)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 269 OUT27 |
| ✓ | LcbMinCostEqualCharsLiveCodeBench / leetcode / medium: minimum-cost-to-make-all-characters-equal (LCB 2817)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 297 OUT282 |
| × | LcbMinimizeStringLengthLiveCodeBench / leetcode / easy: minimize-string-length (LCB 2825)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimizedStringLength case1: expected 3 got 4 |
Failed | IN9 286 OUT168 |
| ✓ | LcbRemoveTrailingZerosLiveCodeBench / leetcode / easy: remove-trailing-zeros-from-a-string (LCB 2819)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 239 OUT64 |
| ✓ | LcbSmallestPalindromeLiveCodeBench / leetcode / easy: lexicographically-smallest-palindrome (LCB 2816)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 276 OUT224 |
Code Generation - LiveCodeBench v1 LeetCode (set 4)
5 passed4 failed |
55.6% |
IN84 177 OUT2 094 | |
| ✓ | LcbBeautifulPairsLiveCodeBench / leetcode / easy: number-of-beautiful-pairs (id 2831)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 321 OUT269 |
| × | LcbCountZeroRequestServersLiveCodeBench / leetcode / medium: count-zero-request-servers (id 2833)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countServers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:20: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed | IN9 420 OUT482 |
| ✓ | LcbLongestEqualSubarrayLiveCodeBench / leetcode / medium: find-the-longest-equal-subarray (id 2832)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 325 OUT320 |
| × | LcbMakeIntegerZeroLiveCodeBench / leetcode / medium: minimum-operations-to-make-the-integer-zero (id 2837)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: makeTheIntegerZero case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'makeTheIntegerZero' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MakeTheIntegerZero.
Values:
run.Compiled = false |
Failed | IN9 312 OUT170 |
| × | LcbMaximumSumQueriesLiveCodeBench / leetcode / hard: maximum-sum-queries (id 2839)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maximumSumQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'maximumSumQueries' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MaximumSumQueries.
Values:
run.Compiled = false |
Failed | IN9 472 OUT223 |
| ✓ | LcbRelocateMarblesLiveCodeBench / leetcode / medium: relocate-marbles (id 2834)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 398 OUT161 |
| × | LcbSmallestStringOpLiveCodeBench / leetcode / medium: lexicographically-smallest-string-after-substring-operation (id 2828)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: smallestString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:29: Unknown variable 'Cast'.
11:34: Unknown variable 'int'.
Values:
run.Compiled = false |
Failed | IN9 337 OUT273 |
| ✓ | LcbSumOfSquaresSpecialLiveCodeBench / leetcode / easy: sum-of-squares-of-special-elements (id 2844)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 286 OUT94 |
| ✓ | LcbValueOfPartitionLiveCodeBench / leetcode / medium: find-the-value-of-the-partition (id 2845)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 306 OUT102 |
Code Generation - LiveCodeBench v1 LeetCode (set 5)
7 passed2 failed |
77.8% |
IN84 086 OUT2 355 | |
| ✓ | LcbCountCompleteSubarraysLiveCodeBench / leetcode / medium: count-complete-subarrays-in-an-array (id 2856)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 291 OUT264 |
| × | LcbDecrementalConcatLiveCodeBench / leetcode / medium: decremental-string-concatenation (id 2854)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minimizeConcatenatedLength case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimizeConcatenatedLength' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, minimizeConcatenedLength.
Values:
run.Compiled = false |
Failed | IN9 403 OUT614 |
| ✓ | LcbLongestEvenOddThresholdLiveCodeBench / leetcode / easy: longest-even-odd-subarray-with-threshold (id 2866)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 375 OUT190 |
| ✓ | LcbLongestNewStringLiveCodeBench / leetcode / medium: construct-the-longest-new-string (id 2850)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 327 OUT111 |
| ✓ | LcbMaximumJumpsLiveCodeBench / leetcode / medium: maximum-number-of-jumps-to-reach-the-last-index (id 2855)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 374 OUT224 |
| ✓ | LcbMaxStringPairsLiveCodeBench / leetcode / easy: find-maximum-number-of-string-pairs (id 2847)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 325 OUT180 |
| × | LcbSpecialPermutationsLiveCodeBench / leetcode / medium: special-permutations (id 2848)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: specialPerm case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:26: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 308 OUT495 |
| ✓ | LcbSumImbalanceNumbersLiveCodeBench / leetcode / hard: sum-of-imbalance-numbers-of-all-subarrays (id 2849)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 330 OUT170 |
| ✓ | LcbTotalDistanceTraveledLiveCodeBench / leetcode / easy: total-distance-traveled (id 2857)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 353 OUT107 |
Code Generation - LiveCodeBench v1 LeetCode (set 6)
6 passed2 failed |
75.0% |
IN74 724 OUT2 204 | |
| ✓ | LcbContinuousSubarraysLiveCodeBench / leetcode / medium: continuous-subarrays (id 2868)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 316 OUT338 |
| ✓ | LcbEmployeesMetTargetLiveCodeBench / leetcode / easy: number-of-employees-who-met-the-target (id 2876)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 330 OUT65 |
| ✓ | LcbGoodSubarraySplitsLiveCodeBench / leetcode / medium: ways-to-split-array-into-good-subarrays (id 2867)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 311 OUT239 |
| ✓ | LcbLongestAlternatingLiveCodeBench / leetcode / easy: longest-alternating-subarray (id 2870)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 395 OUT226 |
| ✓ | LcbLongestNonDecreasingLiveCodeBench / leetcode / medium: longest-non-decreasing-subarray-from-two-arrays (id 2869)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 394 OUT348 |
| ✓ | LcbMaxArrayValueMergeLiveCodeBench / leetcode / medium: largest-element-in-an-array-after-merge-operations (id 2872)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 323 OUT129 |
| × | LcbPrimePairsSumLiveCodeBench / leetcode / medium: prime-pairs-with-target-sum (id 2873)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: findPrimePairs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:45: Type 'List<int>' has no supported method named 'ToList' with 0 argument(s).
4:9: Variable 'result' is already declared in this scope or an enclosing one.
Values:
run.Compiled = false |
Failed | IN9 325 OUT263 |
| × | LcbShortestThreeStringsLiveCodeBench / leetcode / medium: shortest-string-that-contains-three-strings (id 2877)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minimumString case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumString' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumString, MergeTwo, MergeThree, NextPermutation, Swap, Reverse.
Values:
run.Compiled = false |
Failed | IN9 330 OUT596 |
Code Generation - LiveCodeBench v1 LeetCode (set 7)
7 passed2 failed |
77.8% |
IN83 614 OUT3 476 | |
| × | LcbCheckArrayZeroableLiveCodeBench / leetcode / medium: apply-operations-to-make-all-array-elements-equal-to-zero (id 2878)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<bool>(run.ResultJson) == FromJson<bool>(expected[i])
Context: CheckArrayZeroable case0: expected true got false |
Failed | IN9 285 OUT273 |
| ✓ | LcbFaultyKeyboardLiveCodeBench / leetcode / easy: faulty-keyboard (id 2886)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 245 OUT116 |
| ✓ | LcbLongestValidSubstringForbiddenLiveCodeBench / leetcode / hard: length-of-the-longest-valid-substring (id 2884)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 269 OUT323 |
| ✓ | LcbMinBeautifulSubstringsLiveCodeBench / leetcode / medium: partition-string-into-minimum-beautiful-substrings (id 2883)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 279 OUT365 |
| × | LcbMinChangesKSemiPalindromesLiveCodeBench / leetcode / hard: minimum-changes-to-make-k-semi-palindromes (id 2879)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MinChangesKSemiPalindromes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
79:29: Variable 'c1' has type 'int' but its initializer has type 'char'.
80:29: Variable 'c2' has type 'int' but its initializer has type 'char'.
Values:
run.Compiled = false |
Failed | IN9 308 OUT1 326 |
| ✓ | LcbMinIndexValidSplitLiveCodeBench / leetcode / medium: minimum-index-of-a-valid-split (id 2888)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 343 OUT408 |
| ✓ | LcbSortVowelsInStringLiveCodeBench / leetcode / medium: sort-vowels-in-a-string (id 2887)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 271 OUT281 |
| ✓ | LcbSplitWordsBySeparatorLiveCodeBench / leetcode / easy: split-strings-by-separator (id 2881)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 309 OUT99 |
| ✓ | LcbWaysSumOfPowersLiveCodeBench / leetcode / medium: ways-to-express-an-integer-as-sum-of-powers (id 2882)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 305 OUT285 |
Code Generation - LiveCodeBench v1 LeetCode (set 8)
8 passed1 failed |
88.9% |
IN83 738 OUT2 129 | |
| ✓ | LcbCanSplitArrayMLiveCodeBench / leetcode / medium: check-if-it-is-possible-to-split-array (id 2916)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 329 OUT214 |
| ✓ | LcbCountInterestingSubarraysLiveCodeBench / leetcode / medium: count-of-interesting-subarrays (id 2915)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 316 OUT222 |
| ✓ | LcbCountPairsSumLessTargetLiveCodeBench / leetcode / easy: count-pairs-whose-sum-is-less-than-target (id 2917)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 276 OUT107 |
| ✓ | LcbIsArrayGoodLiveCodeBench / leetcode / easy: check-if-array-is-good (id 2892)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 324 OUT230 |
| ✓ | LcbMaxBeautyAfterOpLiveCodeBench / leetcode / medium: maximum-beauty-of-an-array-after-applying-operation (id 2891)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 309 OUT134 |
| ✓ | LcbMaxIncreasingGroupsLiveCodeBench / leetcode / hard: maximum-number-of-groups-with-increasing-length (id 2919)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 303 OUT108 |
| ✓ | LcbMaxPairSumEqualDigitLiveCodeBench / leetcode / easy: max-pair-sum-in-an-array (id 2902)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 269 OUT220 |
| ✓ | LcbMaxScoreVisitPositionsLiveCodeBench / leetcode / medium: visit-array-positions-to-maximize-score (id 2893)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 316 OUT556 |
| × | LcbMinSecondsCircularEqualLiveCodeBench / leetcode / medium: minimum-seconds-to-equalize-a-circular-array (id 2920)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MinSecondsCircularEqual case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumSeconds' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumSeconds.
Values:
run.Compiled = false |
Failed | IN9 296 OUT338 |
Code Generation - LiveCodeBench v1 LeetCode (set 9)
6 passed2 failed |
75.0% |
IN74 523 OUT2 682 | |
| ✓ | LcbAccountBalanceRoundedLiveCodeBench / leetcode / easy: account-balance-after-rounded-purchase (id 2955)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 282 OUT101 |
| ✓ | LcbCountPairsDistanceKLiveCodeBench / leetcode / medium: count-pairs-of-points-with-distance-k (id 2953)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 336 OUT162 |
| × | LcbCountSteppingNumbersLiveCodeBench / leetcode / hard: count-stepping-numbers-in-range (id 2921)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CountSteppingNumbers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
73:38: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
run.Compiled = false |
Failed | IN9 288 OUT1 139 |
| ✓ | LcbCountSymmetricIntegersLiveCodeBench / leetcode / easy: count-symmetric-integers (id 2998)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 281 OUT205 |
| ✓ | LcbIsAcronymOfWordsLiveCodeBench / leetcode / easy: check-if-a-string-is-an-acronym-of-words (id 2977)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 295 OUT90 |
| ✓ | LcbMaximizeSalesmanProfitLiveCodeBench / leetcode / medium: maximize-the-profit-as-the-salesman (id 2979)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 345 OUT335 |
| ✓ | LcbMaxSumAlmostUniqueLiveCodeBench / leetcode / medium: maximum-sum-of-almost-unique-subarray (id 2954)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 340 OUT395 |
| × | LcbMinTimeSumAtMostXLiveCodeBench / leetcode / hard: minimum-time-to-make-array-sum-at-most-x (id 2952)1 AI query
0 tool callscategory:CodeGenerationcategory:V1@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MinTimeSumAtMostX case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumTime' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumTime.
Values:
run.Compiled = false |
Failed | IN9 356 OUT255 |
Code Generation - LiveCodeBench v2 AtCoder (set 1)
5 passed3 failed |
62.5% |
IN74 856 OUT3 406 | |
| × | LcbABCSumQueryLiveCodeBench / atcoder abc344_c / medium: A+B+C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 393 OUT515 |
| ✓ | LcbCeilDivTenLiveCodeBench / atcoder abc345_b / easy: Integer Division Returns (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 279 OUT111 |
| × | LcbDelimiterReverseLiveCodeBench / atcoder abc344_b / easy: Delimiter (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 326 OUT130 |
| ✓ | LcbGomamayoSequenceLiveCodeBench / atcoder abc346_d / medium: Gomamayo Sequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 389 OUT1 144 |
| ✓ | LcbInsertOrEraseLiveCodeBench / atcoder abc344_e / hard: Insert or Erase (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 381 OUT440 |
| ✓ | LcbSpoilerRemoveLiveCodeBench / atcoder abc344_a / easy: Spoiler (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 281 OUT69 |
| ✓ | LcbStringBagsLiveCodeBench / atcoder abc344_d / hard: String Bags (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 429 OUT530 |
| × | LcbTilingGridLiveCodeBench / atcoder abc345_d / hard: Tiling (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TilingGrid case3: expected [No] got [Yes] |
Failed | IN9 378 OUT467 |
Code Generation - LiveCodeBench v2 AtCoder (set 2)
4 passed3 failed |
57.1% |
IN65 399 OUT2 631 | |
| ✓ | LcbAdjacentProductLiveCodeBench / atcoder abc346_a / easy: Adjacent Product (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 304 OUT150 |
| ✓ | LcbDivisibleQuotientsLiveCodeBench / atcoder abc347_a / easy: Divisible (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 314 OUT205 |
| ✓ | LcbIdealHolidaysLiveCodeBench / atcoder abc347_c / medium: Ideal Holidays (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 350 OUT296 |
| × | LcbPaintGridLiveCodeBench / atcoder abc346_e / hard: Paint (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PaintGrid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:16: Variable 'xOps' has type 'List<long>' but its initializer has type 'List<int>'.
Values:
run.Compiled = false |
Failed | IN9 435 OUT824 |
| ✓ | LcbPianoSubstringLiveCodeBench / atcoder abc346_b / easy: Piano (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 320 OUT573 |
| × | LcbSetAddQueryLiveCodeBench / atcoder abc347_e / hard: Set Add Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SetAddQuery case0: expected [6 2 2] got [0 0 -2] |
Failed | IN9 379 OUT417 |
| × | LcbSigmaMissingSumLiveCodeBench / atcoder abc346_c / medium: Sigma (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: SigmaMissingSum case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 297 OUT166 |
Code Generation - LiveCodeBench v2 AtCoder (set 3)
6 passed2 failed |
75.0% |
IN75 471 OUT5 121 | |
| ✓ | LcbAirportCodeLiveCodeBench / atcoder / abc349_c: Airport Code (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 422 OUT369 |
| ✓ | LcbColorfulBeansLiveCodeBench / atcoder / abc348_c: Colorful Beans (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 411 OUT259 |
| ✓ | LcbDistinctSubstringsLiveCodeBench / atcoder / abc347_b: Substring (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 322 OUT108 |
| × | LcbFarthestPointLiveCodeBench / atcoder / abc348_b: Farthest Point (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 463 OUT319 |
| × | LcbMedicinesOnGridLiveCodeBench / atcoder / abc348_d: Medicines on Grid (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: MedicinesOnGrid case0 runtime error: List index 9 is out of range. The list has 6 element(s).
Values:
run.Success = false |
Failed | IN9 642 OUT3 043 |
| ✓ | LcbMinimizeSumOfDistancesLiveCodeBench / atcoder / abc348_e: Minimize Sum of Distances (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 453 OUT776 |
| ✓ | LcbPenaltyKickLiveCodeBench / atcoder / abc348_a: Penalty Kick (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 356 OUT97 |
| ✓ | LcbZeroSumGameLiveCodeBench / atcoder / abc349_a: Zero Sum Game (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 402 OUT150 |
Code Generation - LiveCodeBench v2 AtCoder (set 4)
3 passed2 failed2 skipped |
60.0% |
IN47 309 OUT2 117 | |
| ✓ | LcbCommencementLiveCodeBench / atcoder / abc349_b: Commencement (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 345 OUT270 |
| ✓ | LcbDentistAokiLiveCodeBench / atcoder / abc350_b: Dentist Aoki (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 409 OUT178 |
| × | LcbDivideIntervalLiveCodeBench / atcoder / abc349_d: Divide Interval (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: DivideInterval case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 543 OUT377 |
| ✓ | LcbPastABCsLiveCodeBench / atcoder / abc350_a: Past ABCs (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 427 OUT95 |
| – | LcbSortPermutationLiveCodeBench / atcoder / abc350_c: Sort (Pattern B, stdin) TRIAGE: 0 public tests in dump + non-unique output (any valid swap sequence accepted) — cannot oracle-compare.0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench Skipped: no public tests in dump; output non-unique (any valid swap sequence is accepted) — exact-compare oracle inapplicable, coordinator triage |
Skipped | IN0 OUT0 |
| – | LcbTowardZeroLiveCodeBench / atcoder / abc350_e: Toward 0 (Pattern B, stdin) TRIAGE: float output (absolute/relative error <= 1e-6) — RunLcbStdin does exact string compare; needs tolerance.0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Skipped: float output requires error tolerance (<=1e-6); RunLcbStdin does exact string compare — coordinator triage |
Skipped | IN0 OUT0 |
| × | LcbWeightedTicLiveCodeBench / atcoder / abc349_e: Weighted Tic (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: WeightedTic case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:5: Expected ';' after the variable declaration, found '{'.
Values:
run.Compiled = false |
Failed | IN9 585 OUT1 197 |
Code Generation - LiveCodeBench v2 AtCoder (set 5)
4 passed4 failed |
50.0% |
IN75 300 OUT2 817 | |
| ✓ | LcbBottomNinthLiveCodeBench / atcoder / abc351_a: The bottom of the ninth (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 460 OUT126 |
| × | LcbCliqueConnectLiveCodeBench / atcoder / abc352_e: Clique Connect (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CliqueConnect case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 471 OUT729 |
| ✓ | LcbDoubleSumMaxDiffLiveCodeBench / atcoder / abc351_f: Double Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 323 OUT188 |
| × | LcbGridAndMagnetLiveCodeBench / atcoder / abc351_d: Grid and Magnet (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: GridAndMagnet case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 426 OUT728 |
| × | LcbJumpDistanceSumLiveCodeBench / atcoder / abc351_e: Jump Distance Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: JumpDistanceSum case1: expected [11] got [7] |
Failed | IN9 427 OUT249 |
| ✓ | LcbMergeTheBallsLiveCodeBench / atcoder / abc351_c: Merge the balls (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 405 OUT191 |
| ✓ | LcbNewFriendsLiveCodeBench / atcoder / abc350_d: New Friends (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 354 OUT436 |
| × | LcbSpotTheDifferenceLiveCodeBench / atcoder / abc351_b: Spot the Difference (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SpotTheDifference case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
20:5: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed | IN9 434 OUT170 |
Code Generation - LiveCodeBench v2 AtCoder (set 6)
5 passed2 failed |
71.4% |
IN65 751 OUT1 745 | |
| ✓ | LcbAmusementParkLiveCodeBench / atcoder / abc353_b: AtCoder Amusement Park (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 416 OUT203 |
| ✓ | LcbAtcoderLineLiveCodeBench / atcoder / abc352_a: AtCoder Line (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 369 OUT172 |
| ✓ | LcbBuildingsTallerLiveCodeBench / atcoder / abc353_a: Buildings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 343 OUT145 |
| ✓ | LcbPermutationSubsequenceLiveCodeBench / atcoder / abc352_d: Permutation Subsequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 431 OUT312 |
| × | LcbSigmaRemainderSumLiveCodeBench / atcoder / abc353_c: Sigma Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SigmaRemainderSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 349 OUT396 |
| × | LcbStandingOnShouldersLiveCodeBench / atcoder / abc352_c: Standing On The Shoulders (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: StandingOnShoulders case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:23: Expected ')' to close the parenthesized expression, found 'x'.
Values:
run.Compiled = false |
Failed | IN9 442 OUT341 |
| ✓ | LcbTypingBackspaceLiveCodeBench / atcoder / abc352_b: Typing (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 401 OUT176 |
Code Generation - LiveCodeBench v2 AtCoder (set 7)
3 passed3 failed |
50.0% |
IN56 638 OUT2 535 | |
| ✓ | LcbAnotherSigmaProblemLiveCodeBench / atcoder / abc353_d: Another Sigma Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 450 OUT1 098 |
| ✓ | LcbAtcoderJanken2LiveCodeBench / atcoder / abc354_b: AtCoder Janken 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 434 OUT132 |
| × | LcbAtcoderMagicsLiveCodeBench / atcoder / abc354_c: AtCoder Magics (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AtcoderMagics case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:5: Expected an expression, found 'struct'.
Values:
run.Compiled = false |
Failed | IN9 503 OUT461 |
| ✓ | LcbExponentialPlantLiveCodeBench / atcoder / abc354_a: Exponential Plant (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 387 OUT82 |
| × | LcbRemovePairsLiveCodeBench / atcoder / abc354_e: Remove Pairs (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: RemovePairs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 480 OUT587 |
| × | LcbYetAnotherSigmaProblemLiveCodeBench / atcoder / abc353_e: Yet Another Sigma Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: YetAnotherSigmaProblem case0 runtime error: List index 2 is out of range. The list has 2 element(s).
Values:
run.Success = false |
Failed | IN9 384 OUT175 |
Code Generation - LiveCodeBench v2 AtCoder (set 8)
3 passed2 failed1 skipped |
60.0% |
IN47 183 OUT3 084 | |
| × | LcbAtcoderWallpaperLiveCodeBench / atcoder / abc354_d: AtCoder Wallpaper (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AtcoderWallpaper case0: expected [10] got [5] |
Failed | IN9 495 OUT1 990 |
| ✓ | LcbBingo2LiveCodeBench / atcoder / abc355_c: Bingo 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 508 OUT426 |
| – | LcbGuessTheSumLiveCodeBench / atcoder / abc355_e: Guess the Sum (Pattern B, stdin) — INTERACTIVE, cannot run in the batch harness0 AI queries
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Hardsource:LiveCodeBench Skipped: SC# subset: interactive problem (adaptive query/response dialogue; not expressible as a single stdin->stdout Solve) |
Skipped | IN0 OUT0 |
| × | LcbIntersectingIntervalsLiveCodeBench / atcoder / abc355_d: Intersecting Intervals (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: IntersectingIntervals case0: expected [2] got [1] |
Failed | IN9 365 OUT211 |
| ✓ | LcbPiano2LiveCodeBench / atcoder / abc355_b: Piano 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 433 OUT313 |
| ✓ | LcbWhoAteTheCakeLiveCodeBench / atcoder / abc355_a: Who Ate the Cake? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 382 OUT144 |
Code Generation - LiveCodeBench v2 LeetCode (set 1)
7 passed2 failed |
77.8% |
IN83 755 OUT3 008 | |
| ✓ | LcbCountSubstringsStartEndCharLiveCodeBench / leetcode / medium: count-substrings-starting-and-ending-with-given-character (LCB 3337)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 235 OUT106 |
| ✓ | LcbMaximumHappinessSumLiveCodeBench / leetcode / medium: maximize-happiness-of-selected-children (LCB 3351)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 330 OUT113 |
| × | LcbMaximumStrengthSubarraysLiveCodeBench / leetcode / hard: maximum-strength-of-k-disjoint-subarrays (LCB 3313)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maximumStrength case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
51:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 385 OUT1 207 |
| ✓ | LcbMaxLengthTwoOccurrencesLiveCodeBench / leetcode / easy: maximum-length-substring-with-two-occurrences (LCB 3349)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 217 OUT251 |
| ✓ | LcbMinDeletionsKSpecialLiveCodeBench / leetcode / medium: minimum-deletions-to-make-string-k-special (LCB 3360)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 290 OUT298 |
| ✓ | LcbMinimumBoxesApplesLiveCodeBench / leetcode / easy: apple-redistribution-into-boxes (LCB 3334)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 320 OUT134 |
| × | LcbMinMovesPickKOnesLiveCodeBench / leetcode / hard: minimum-moves-to-pick-k-ones (LCB 3327)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumMoves case0: expected 3 got 5 |
Failed | IN9 444 OUT382 |
| ✓ | LcbShortestUncommonSubstringsLiveCodeBench / leetcode / medium: shortest-uncommon-substring-in-an-array (LCB 3356)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 298 OUT313 |
| ✓ | LcbSubstringInReverseLiveCodeBench / leetcode / easy: existence-of-a-substring-in-a-string-and-its-reverse (LCB 3353)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 236 OUT204 |
Code Generation - LiveCodeBench v2 LeetCode (set 2)
6 passed3 failed |
66.7% |
IN83 652 OUT2 462 | |
| ✓ | LcbCountAlternatingSubarraysLiveCodeBench / leetcode / medium: count-alternating-subarrays (LCB 3374)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 248 OUT128 |
| ✓ | LcbHarshadDigitSumLiveCodeBench / leetcode / easy: harshad-number (LCB 3371)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 247 OUT132 |
| × | LcbLexSmallestWithinDistanceLiveCodeBench / leetcode / medium: lexicographically-smallest-string-after-operations-with-constraint (LCB 3346)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: getSmallestString case1: expected "aawcd" got "aaxcd" |
Failed | IN9 343 OUT156 |
| ✓ | LcbLongestCommonSuffixQueriesLiveCodeBench / leetcode / hard: longest-common-suffix-queries (LCB 3376)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 342 OUT547 |
| ✓ | LcbLongestMonotonicSubarrayLiveCodeBench / leetcode / easy: longest-strictly-increasing-or-decreasing-subarray (LCB 3372)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 256 OUT222 |
| × | LcbMinimizeManhattanDistanceLiveCodeBench / leetcode / hard: minimize-manhattan-distances (LCB 3344)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minimumDistance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumDistance' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumDistance.
Values:
run.Compiled = false |
Failed | IN9 279 OUT550 |
| ✓ | LcbMinOpsMedianKLiveCodeBench / leetcode / medium: minimum-operations-to-make-median-of-array-equal-to-k (LCB 3387)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 319 OUT223 |
| × | LcbMinOpsSumAtLeastKLiveCodeBench / leetcode / medium: apply-operations-to-make-sum-of-array-ge-k (LCB 3328)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOperations case0: expected 5 got 4 |
Failed | IN9 274 OUT113 |
| ✓ | LcbMostFrequentIdsLiveCodeBench / leetcode / medium: most-frequent-ids (LCB 3363)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 344 OUT391 |
Code Generation - LiveCodeBench v2 LeetCode (set 3)
3 passed6 failed |
33.3% |
IN84 002 OUT4 431 | |
| × | LcbKthSmallestAmountLiveCodeBench 3375 / leetcode / hard: kth-smallest-amount-with-single-denomination-combination (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: kthSmallestAmount case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'findKthSmallest' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, FindKthSmallest.
Values:
run.Compiled = false |
Failed | IN9 319 OUT233 |
| × | LcbLatestTimeReplaceLiveCodeBench 3361 / leetcode / easy: latest-time-you-can-obtain-after-replacing-characters (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: latestTimeReplace case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'findLatestTime' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, FindLatestTime.
Values:
run.Compiled = false |
Failed | IN9 343 OUT360 |
| ✓ | LcbMaximumPrimeDifferenceLiveCodeBench 3373 / leetcode / medium: maximum-prime-difference (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 288 OUT289 |
| × | LcbMinimizeStringValueLiveCodeBench 3354 / leetcode / medium: replace-question-marks-in-string-to-minimize-its-value (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minimizeStringValue case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimizeStringValue' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimizeStringValue, FindAssignedChar.
Values:
run.Compiled = false |
Failed | IN9 355 OUT1 100 |
| × | LcbMinSumValuesDividingLiveCodeBench 3364 / leetcode / hard: minimum-sum-of-values-by-dividing-array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minSumValuesDividing case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumValueSum' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumValueSum.
Values:
run.Compiled = false |
Failed | IN9 407 OUT501 |
| × | LcbShortestSubarrayOrKLiveCodeBench 3381 / leetcode / easy: shortest-subarray-with-or-at-least-k-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: shortestSubarrayOrK case0: expected 1 got 2 |
Failed | IN9 311 OUT205 |
| ✓ | LcbSumOfEncryptedIntsLiveCodeBench 3367 / leetcode / easy: find-the-sum-of-encrypted-integers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 289 OUT248 |
| ✓ | LcbSumOfPowerSubsequencesLiveCodeBench 3345 / leetcode / hard: find-the-sum-of-the-power-of-all-subsequences (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 304 OUT1 013 |
| × | LcbUnmarkedSumArrayLiveCodeBench 3306 / leetcode / medium: mark-elements-on-array-by-performing-queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: unmarkedSumArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 386 OUT482 |
Code Generation - LiveCodeBench v2 LeetCode (set 4)
5 passed4 failed |
55.6% |
IN83 757 OUT3 127 | |
| ✓ | LcbAddedIntegerLiveCodeBench 3397 / leetcode / easy: find-the-integer-added-to-array-i (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 332 OUT153 |
| × | LcbMedianUniquenessArrayLiveCodeBench 3362 / leetcode / hard: find-the-median-of-the-uniqueness-array (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: medianUniquenessArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'medianOfUniquenessArray' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MedianOfUniquenessArray, CountSubarraysWithDistinct.
Values:
run.Compiled = false |
Failed | IN9 352 OUT441 |
| × | LcbMinimumArrayEndLiveCodeBench 3394 / leetcode / medium: minimum-array-end (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumArrayEnd case0: expected 6 got 13 |
Failed | IN9 296 OUT270 |
| ✓ | LcbMinimumLevelsGainPointsLiveCodeBench 3355 / leetcode / medium: minimum-levels-to-gain-more-points (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 355 OUT338 |
| ✓ | LcbNumberOfSpecialCharsLiveCodeBench 3408 / leetcode / easy: count-the-number-of-special-characters-i (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 251 OUT169 |
| ✓ | LcbScoreOfStringLiveCodeBench 3379 / leetcode / easy: score-of-a-string (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 302 OUT70 |
| ✓ | LcbSubarraysBoundaryMaxLiveCodeBench 3382 / leetcode / hard: find-the-number-of-subarrays-where-boundary-elements-are-maximum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 257 OUT829 |
| × | LcbSumOfSubsequencePowersLiveCodeBench 3316 / leetcode / hard: find-the-sum-of-subsequence-powers (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: sumOfSubsequencePowers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 312 OUT569 |
| × | LcbValidWordLiveCodeBench 3396 / leetcode / easy: valid-word (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: validWord case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'isValid' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, IsValid.
Values:
run.Compiled = false |
Failed | IN9 300 OUT288 |
Code Generation - LiveCodeBench v2 LeetCode (set 5)
5 passed4 failed |
55.6% |
IN83 893 OUT2 974 | |
| ✓ | LcbCanMakeSquareColorLiveCodeBench / leetcode / easy: make-a-square-with-the-same-color1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 347 OUT229 |
| ✓ | LcbIsArraySpecialILiveCodeBench / leetcode / easy: special-array-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 257 OUT105 |
| ✓ | LcbMakeWordKPeriodicLiveCodeBench / leetcode / medium: minimum-number-of-operations-to-make-word-k-periodic1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 327 OUT193 |
| × | LcbMinAnagramLengthLiveCodeBench / leetcode / medium: minimum-length-of-anagram-concatenation1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MinAnagramLength case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
26:40: Only function calls can be used as statements.
Values:
run.Compiled = false |
Failed | IN9 261 OUT417 |
| × | LcbMinCostEqualizeArrayLiveCodeBench / leetcode / hard: minimum-cost-to-equalize-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MinCostEqualizeArray case0: expected 15 got 291172003 |
Failed | IN9 391 OUT412 |
| ✓ | LcbMysticDungeonEnergyLiveCodeBench / leetcode / medium: taking-maximum-energy-from-the-mystic-dungeon1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 318 OUT125 |
| ✓ | LcbNumberOfRightTrianglesLiveCodeBench / leetcode / medium: right-triangles1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 353 OUT247 |
| × | LcbNumberOfStableArraysLiveCodeBench / leetcode / medium: find-all-possible-stable-binary-arrays-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: NumberOfStableArrays case0: expected 2 got 0 |
Failed | IN9 347 OUT1 100 |
| × | LcbPermutationDifferenceLiveCodeBench / leetcode / easy: permutation-difference-between-two-strings1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PermutationDifference case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:5: Unknown variable 'j'. Declare it first, e.g. 'int j = ...;'.
14:12: Unknown variable 'j'.
16:20: Unknown variable 'j'.
18:33: Unknown variable 'j'.
19:9: Unknown variable 'j'. Declare it first, e.g. 'int j = ...;'.
19:9: Unknown variable 'j'.
Values:
run.Compiled = false |
Failed | IN9 292 OUT146 |
Code Generation - LiveCodeBench v2 LeetCode (set 6)
4 passed5 failed |
44.4% |
IN83 887 OUT4 114 | |
| × | LcbMaxSumNonAdjacentQueriesLiveCodeBench / leetcode / hard: maximum-sum-of-subsequence-with-non-adjacent-elements1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MaxSumNonAdjacentQueries case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'maximumSumSubsequence' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MaximumSumSubsequence.
Values:
run.Compiled = false |
Failed | IN9 358 OUT265 |
| × | LcbMinBalancedPartitionLiveCodeBench / leetcode / medium: minimum-substring-partition-of-equal-character-frequency1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MinBalancedPartition case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumSubstringsInPartition' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumSubstringsInPartition.
Values:
run.Compiled = false |
Failed | IN9 255 OUT373 |
| ✓ | LcbNumberOfGoodPairsILiveCodeBench / leetcode / easy: find-the-number-of-good-pairs-i1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 337 OUT143 |
| ✓ | LcbOccurrencesOfElementLiveCodeBench / leetcode / medium: find-occurrences-of-an-element-in-an-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 335 OUT182 |
| × | LcbProductsBigArrayLiveCodeBench / leetcode / hard: find-products-of-elements-of-big-array1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ProductsBigArray case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
88:16: The shift count of '<<' must be 'int', but got 'long'.
90:26: The shift count of '<<' must be 'int', but got 'long'.
93:24: The shift count of '<<' must be 'int', but got 'long'.
96:17: The shift count of '<<' must be 'int', but got 'long'.
109:26: The shift count of '<<' must be 'int', but got 'long'.
112:24: The shift count of '<<' must be 'int', but got 'long'.
156:17: The shift count of '>>' must be 'int', but got 'long'.
Values:
run.Compiled = false |
Failed | IN9 440 OUT2 137 |
| ✓ | LcbStringCompressionIIILiveCodeBench / leetcode / medium: string-compression-iii1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 279 OUT121 |
| × | LcbSumDigitDifferencesLiveCodeBench / leetcode / medium: sum-of-digit-differences-of-all-pairs1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: SumDigitDifferences case0: expected 4 got 8 |
Failed | IN9 277 OUT252 |
| × | LcbWaysToReachStairLiveCodeBench / leetcode / hard: find-number-of-ways-to-reach-the-k-th-stair1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: WaysToReachStair case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:35: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 334 OUT521 |
| ✓ | LcbXorAppearingTwiceLiveCodeBench / leetcode / easy: find-the-xor-of-numbers-which-appear-twice1 AI query
0 tool callscategory:CodeGenerationcategory:V2@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 272 OUT120 |
Code Generation - LiveCodeBench v3 AtCoder (set 1)
5 passed2 failed |
71.4% |
IN65 864 OUT2 156 | |
| ✓ | LcbKeysCombinationsLiveCodeBench / atcoder / medium: abc356_c Keys (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 522 OUT436 |
| × | LcbMaskedPopcountLiveCodeBench / atcoder / medium: abc356_d Masked Popcount (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MaskedPopcount case0: expected [4] got [6] |
Failed | IN9 359 OUT266 |
| × | LcbMaxMinFloorSumLiveCodeBench / atcoder / hard: abc356_e Max/Min (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MaxMinFloorSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
54:30: Math has no supported method named 'min'. Supported methods: Abs, Acos, Asin, Atan, Atan2, Ceiling, Cos, Exp, Floor, Log, Log10, Log2, Max, Min, Pow, Round, Sin, Sqrt, Tan, ToDegrees, ToRadians.
Values:
run.Compiled = false |
Failed | IN9 346 OUT487 |
| ✓ | LcbNutrientsLiveCodeBench / atcoder / easy: abc356_b Nutrients (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 423 OUT304 |
| ✓ | LcbSanitizeHandsLiveCodeBench / atcoder / easy: abc357_a Sanitize Hands (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 398 OUT149 |
| ✓ | LcbSierpinskiCarpetLiveCodeBench / atcoder / medium: abc357_c Sierpinski carpet (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 451 OUT279 |
| ✓ | LcbSubsegmentReverseLiveCodeBench / atcoder / easy: abc356_a Subsegment Reverse (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 365 OUT235 |
Code Generation - LiveCodeBench v3 AtCoder (set 2)
1 passed6 failed |
14.3% |
IN65 928 OUT3 400 | |
| × | LcbAlphabetTilesLiveCodeBench / atcoder / hard: abc358_e Alphabet Tiles (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AlphabetTiles case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 419 OUT1 246 |
| × | LcbConcatModuloLiveCodeBench / atcoder / medium: abc357_d 88888888 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ConcatModulo case0: expected [55555] got [5] |
Failed | IN9 410 OUT366 |
| × | LcbFunctionalGraphReachLiveCodeBench / atcoder / hard: abc357_e Reachability in Functional Graph (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FunctionalGraphReach case0: expected [8] got [5] |
Failed | IN9 398 OUT548 |
| × | LcbPopcornStandsLiveCodeBench / atcoder / medium: abc358_c Popcorn (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PopcornStands case2: expected [3] got [7] |
Failed | IN9 422 OUT374 |
| × | LcbSouvenirsLiveCodeBench / atcoder / medium: abc358_d Souvenirs (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Souvenirs case0: expected [7] got [9] |
Failed | IN9 465 OUT284 |
| × | LcbTicketCounterLiveCodeBench / atcoder / easy: abc358_b Ticket Counter (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 463 OUT200 |
| ✓ | LcbUppercaseLowercaseLiveCodeBench / atcoder / easy: abc357_b Uppercase and Lowercase (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 351 OUT382 |
Code Generation - LiveCodeBench v3 AtCoder (set 3)
3 passed3 failed1 broken |
42.9% |
IN65 545 OUT66 352 | |
| × | LcbAvoidKPalindromeLiveCodeBench / atcoder abc359_d / hard: Avoid K Palindrome (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AvoidKPalindrome case0: expected [1] got [0] |
Failed | IN9 353 OUT2 648 |
| ✓ | LcbCountTakahashiLiveCodeBench / atcoder abc359_a / easy: Count Takahashi (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 309 OUT82 |
| ✓ | LcbCouplesBetweenLiveCodeBench / atcoder abc359_b / easy: Couples (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 322 OUT157 |
| × | LcbMoveItBoxesLiveCodeBench / atcoder abc360_c / medium: Move It (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MoveItBoxes case0: expected [35] got [122] |
Failed | IN9 362 OUT813 |
| × | LcbTileDistance2TollLiveCodeBench / atcoder abc359_c / medium: Tile Distance 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TileDistance2Toll case0: expected [5] got [2] |
Failed | IN9 452 OUT432 |
| ! | LcbWaterTankFillLiveCodeBench / atcoder abc359_e / hard: Water Tank (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Skipped: AI output broken/partial |
Broken | IN9 449 OUT62 144 |
| ✓ | LcbWelcomeAtCoderLandLiveCodeBench / atcoder abc358_a / easy: Welcome to AtCoder Land (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 298 OUT76 |
Code Generation - LiveCodeBench v3 AtCoder (set 4)
4 passed3 failed |
57.1% |
IN65 623 OUT6 743 | |
| × | LcbGhostAntsPassLiveCodeBench / atcoder abc360_d / medium: Ghost Ants (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GhostAntsPass case0: expected [5] got [4] |
Failed | IN9 432 OUT417 |
| × | LcbGoStonePuzzleLiveCodeBench / atcoder abc361_d / hard: Go Stone Puzzle (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GoStonePuzzle case0: expected [4] got [-1] |
Failed | IN9 418 OUT1 796 |
| ✓ | LcbHealthyBreakfastLiveCodeBench / atcoder abc360_a / easy: A Healthy Breakfast (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 358 OUT138 |
| ✓ | LcbInsertAfterKLiveCodeBench / atcoder abc361_a / easy: Insert (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 307 OUT180 |
| ✓ | LcbIntersectCuboidsLiveCodeBench / atcoder abc361_b / medium: Intersection of Cuboids (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 339 OUT489 |
| × | LcbRandomSwapsBallsLiveCodeBench / atcoder abc360_e / hard: Random Swaps of Balls (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RandomSwapsBalls case0: expected [499122178] got [1] |
Failed | IN9 412 OUT3 484 |
| ✓ | LcbVerticalReadingPairLiveCodeBench / atcoder abc360_b / easy: Vertical Reading (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 357 OUT239 |
Code Generation - LiveCodeBench v3 AtCoder (set 5)
4 passed2 failed1 skipped |
66.7% |
IN56 150 OUT4 160 | |
| ✓ | LcbBuyAPenLiveCodeBench / atcoder / easy: abc362_a Buy a Pen (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 335 OUT200 |
| ✓ | LcbCountArithmeticSubsequencesLiveCodeBench / atcoder / hard: abc362_e Count Arithmetic Subsequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 393 OUT2 063 |
| ✓ | LcbMakeThemNarrowLiveCodeBench / atcoder / medium: abc361_c Make Them Narrow (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 345 OUT190 |
| × | LcbPerfectPowerCountLiveCodeBench / atcoder / hard: abc361_f x = a^b (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PerfectPowerCount case0: expected [12] got [11] |
Failed | IN9 288 OUT131 |
| ✓ | LcbShortestPath3LiveCodeBench / atcoder / hard: abc362_d Shortest Path 3 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 412 OUT673 |
| – | LcbSumZeroSequenceLiveCodeBench / atcoder / medium: abc362_c Sum = 0 (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| × | LcbTreeHamiltonPathLiveCodeBench / atcoder / hard: abc361_e Tree and Hamilton Path 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: TreeHamiltonPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:50: List constructor expects a List or HashSet to copy, but got 'int'.
11:52: List constructor expects a List or HashSet to copy, but got 'int'.
37:40: List constructor expects a List or HashSet to copy, but got 'int'.
38:48: List constructor expects a List or HashSet to copy, but got 'int'.
Values:
run.Compiled = false |
Failed | IN9 377 OUT903 |
Code Generation - LiveCodeBench v3 AtCoder (set 6)
3 passed3 failed1 skipped |
50.0% |
IN56 304 OUT2 648 | |
| × | LcbAvoidKPalindrome2LiveCodeBench / atcoder / medium: abc363_c Avoid K Palindrome 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AvoidKPalindrome2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
55:18: Expected ')' to close the argument list, found 'chars'.
Values:
run.Compiled = false |
Failed | IN9 401 OUT533 |
| ✓ | LcbCursedDollLiveCodeBench / atcoder / easy: abc363_b Japanese Cursed Doll (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 371 OUT282 |
| × | LcbNthPalindromeLiveCodeBench / atcoder / medium: abc363_d Palindromic Number (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: NthPalindrome case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
40:18: Variable 'i' has type 'int' but its initializer has type 'long'.
48:18: Variable 'i' has type 'int' but its initializer has type 'long'.
Values:
run.Compiled = false |
Failed | IN9 311 OUT441 |
| – | LcbPalindromicExpressionLiveCodeBench / atcoder / hard: abc363_f Palindromic Expression (Pattern B, stdin)0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| ✓ | LcbPilingUpLiveCodeBench / atcoder / easy: abc363_a Piling Up (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 391 OUT63 |
| × | LcbRightTriangleCheckLiveCodeBench / atcoder / easy: abc362_b Right Triangle (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: RightTriangleCheck case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
4:40: Type 'List<string>' has no supported method named 'ConvertAll' with 0 argument(s).
7:40: Type 'List<string>' has no supported method named 'ConvertAll' with 0 argument(s).
10:40: Type 'List<string>' has no supported method named 'ConvertAll' with 0 argument(s).
Values:
run.Compiled = false |
Failed | IN9 341 OUT294 |
| ✓ | LcbSinkingLandLiveCodeBench / atcoder / hard: abc363_e Sinking Land (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 489 OUT1 035 |
Code Generation - LiveCodeBench v3 AtCoder (set 7)
2 passed4 failed |
33.3% |
IN56 317 OUT5 216 | |
| × | LcbGluttonTakahashiLiveCodeBench / atcoder / easy: Glutton Takahashi (abc364_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GluttonTakahashi case1: expected [Yes] got [No] |
Failed | IN9 323 OUT138 |
| ✓ | LcbGridWalkLiveCodeBench / atcoder / easy: Grid Walk (abc364_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 503 OUT437 |
| ✓ | LcbKthClosestPointLiveCodeBench / atcoder / hard: K — k-th closest point (abc364_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 404 OUT406 |
| × | LcbMaximumGluttonLiveCodeBench / atcoder / hard: Maximum Glutton (abc364_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MaximumGlutton case0: expected [3] got [2] |
Failed | IN9 366 OUT321 |
| × | LcbMinimumGluttonLiveCodeBench / atcoder / medium: Minimum Glutton (abc364_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MinimumGlutton case0: expected [2] got [4] |
Failed | IN9 330 OUT375 |
| × | LcbRangeConnectMstLiveCodeBench / atcoder / hard: Range Connect MST (abc364_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: RangeConnectMst case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
194:8: Function 'Solve' is declared more than once. Overloads are not supported.
130:27: Unknown function 'SkipFind'.
134:24: Unknown function 'DsuFind'.
135:24: Unknown function 'DsuFind'.
144:13: Unknown function 'SkipUnion'.
158:17: Unknown function 'DsuFind'.
162:13: Unknown function 'DsuFind'.
Values:
run.Compiled = false |
Failed | IN9 391 OUT3 539 |
Code Generation - LiveCodeBench v3 AtCoder (set 8)
2 passed3 failed |
40.0% |
IN46 770 OUT1 616 | |
| × | LcbAtcoderJanken3LiveCodeBench / atcoder / medium: AtCoder Janken 3 (abc365_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AtcoderJanken3 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:21: Variable 'dp' has type 'List<List<int>>' but its initializer has type 'List<int>'.
Values:
run.Compiled = false |
Failed | IN9 399 OUT480 |
| ✓ | LcbLeapYearLiveCodeBench / atcoder / easy: Leap Year (abc365_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 373 OUT150 |
| × | LcbSecondBestLiveCodeBench / atcoder / easy: Second Best (abc365_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SecondBest case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 305 OUT204 |
| ✓ | LcbTransportationExpensesLiveCodeBench / atcoder / medium: Transportation Expenses (abc365_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 365 OUT347 |
| × | LcbXorSigmaProblemLiveCodeBench / atcoder / hard: Xor Sigma Problem (abc365_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: XorSigmaProblem case0: expected [3] got [9223372036854775805] |
Failed | IN9 328 OUT435 |
Code Generation - LiveCodeBench v3 LeetCode (set 1)
6 passed2 failed |
75.0% |
IN74 433 OUT1 493 | |
| ✓ | LcbChildWithBallLiveCodeBench / leetcode / easy: find-the-child-who-has-the-ball-after-k-seconds (id 3450)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 311 OUT70 |
| ✓ | LcbCompleteDayPairsLiveCodeBench / leetcode / easy: count-pairs-that-form-a-complete-day-i (id 3421)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 270 OUT125 |
| ✓ | LcbCountDaysWithoutMeetingsLiveCodeBench / leetcode / medium: count-days-without-meetings (id 3430)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 328 OUT147 |
| ✓ | LcbMaxTotalDamageLiveCodeBench / leetcode / medium: maximum-total-damage-with-spell-casting (id 3437)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 290 OUT396 |
| × | LcbMaxTotalRewardLiveCodeBench / leetcode / medium: maximum-total-reward-using-operations-i (id 3442)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxTotalReward case0: expected 4 got 0 |
Failed | IN9 304 OUT320 |
| ✓ | LcbMinimumChairsLiveCodeBench / leetcode / easy: minimum-number-of-chairs-in-a-waiting-room (id 3426)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 293 OUT120 |
| ✓ | LcbSubarrayOrClosestKLiveCodeBench / leetcode / hard: find-subarray-with-bitwise-or-closest-to-k (id 3436)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 312 OUT155 |
| × | LcbValueAfterKSecondsLiveCodeBench / leetcode / medium: find-the-n-th-value-after-k-seconds (id 3422)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: ValueAfterKSeconds case0: expected 56 got 35 |
Failed | IN9 325 OUT160 |
Code Generation - LiveCodeBench v3 LeetCode (set 2)
3 passed5 failed |
37.5% |
IN74 603 OUT2 580 | |
| × | LcbEncryptedStringLiveCodeBench / leetcode / easy: find-the-encrypted-string (id 3468)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: EncryptedString case0: expected "tdar" got "artd" |
Failed | IN9 257 OUT112 |
| × | LcbMaxTotalCostAlternatingLiveCodeBench / leetcode / medium: maximize-total-cost-of-alternating-subarrays (id 3464)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: MaxTotalCostAlternating case1: expected 4 got 2 |
Failed | IN9 348 OUT141 |
| ✓ | LcbMaxTriangleHeightLiveCodeBench / leetcode / easy: maximum-height-of-a-triangle (id 3469)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 314 OUT203 |
| × | LcbMaxValidSubsequenceLiveCodeBench / leetcode / medium: find-the-maximum-length-of-valid-subsequence-i (id 3490)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: MaxValidSubsequence case0: expected 4 got 3 |
Failed | IN9 315 OUT448 |
| × | LcbMinDiameterMergeTreesLiveCodeBench / leetcode / hard: find-minimum-diameter-after-merging-two-trees (id 3439)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MinDiameterMergeTrees case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
62:21: Variable 'adj' has type 'List<List<int>>' but its initializer has type 'List<int>'.
Values:
run.Compiled = false |
Failed | IN9 414 OUT810 |
| ✓ | LcbMinimumAreaCoverOnesLiveCodeBench / leetcode / medium: find-the-minimum-area-to-cover-all-ones-i (id 3461)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 260 OUT263 |
| ✓ | LcbMinimumAverageLiveCodeBench / leetcode / easy: minimum-average-of-smallest-and-largest-elements (id 3471)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 315 OUT121 |
| × | LcbPeaksInArrayLiveCodeBench / leetcode / hard: peaks-in-array (id 3438)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: PeaksInArray case1: expected [0, 1] got [1,1] |
Failed | IN9 380 OUT482 |
Code Generation - LiveCodeBench v3 LeetCode (set 3)
5 passed2 failed1 skipped |
71.4% |
IN65 707 OUT1 684 | |
| ✓ | LcbBitChangesEqualLiveCodeBench / leetcode / easy: 3508 number-of-bit-changes-to-make-two-integers-equal1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 353 OUT106 |
| × | LcbCountEqualXYSubmatricesLiveCodeBench / leetcode / medium: 3492 count-submatrices-with-equal-frequency-of-x-and-y1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfSubmatrices case0: expected 3 got 1 |
Failed | IN9 368 OUT311 |
| × | LcbMaxOpsMoveOnesLiveCodeBench / leetcode / medium: 3493 maximum-number-of-operations-to-move-ones-to-the-end1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxOperations case0: expected 4 got 5 |
Failed | IN9 350 OUT148 |
| ✓ | LcbMinCostConstructStringLiveCodeBench / leetcode / hard: 3482 construct-string-with-minimum-cost1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 464 OUT322 |
| ✓ | LcbMinCostCutCakeLiveCodeBench / leetcode / medium: 3494 minimum-cost-for-cutting-cake-i1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 480 OUT270 |
| ✓ | LcbSmallestStringAfterSwapLiveCodeBench / leetcode / easy: 3484 lexicographically-smallest-string-after-a-swap1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 346 OUT426 |
| – | LcbValidBinaryStringsLiveCodeBench / leetcode / medium: 3453 generate-binary-strings-without-adjacent-zeros0 AI queries
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| ✓ | LcbVowelsGameLiveCodeBench / leetcode / medium: 3462 vowels-game-in-a-string1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 346 OUT101 |
Code Generation - LiveCodeBench v3 LeetCode (set 4)
5 passed3 failed |
62.5% |
IN75 337 OUT6 958 | |
| ✓ | LcbAlternatingGroupsIIILiveCodeBench / leetcode / hard: 3527 alternating-groups-iii1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 533 OUT2 588 |
| ✓ | LcbDigitGameWinLiveCodeBench / leetcode / easy: 3515 find-if-digit-game-can-be-won1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 387 OUT144 |
| ✓ | LcbDominantOnesSubstringsLiveCodeBench / leetcode / medium: 3479 count-the-number-of-substrings-with-dominant-ones1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 317 OUT149 |
| × | LcbMinOpsArrayToTargetLiveCodeBench / leetcode / hard: 3454 minimum-operations-to-make-array-equal-to-target1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minimumOperations case1: expected 5 got 8 |
Failed | IN9 346 OUT178 |
| × | LcbNonSpecialCountLiveCodeBench / leetcode / medium: 3507 find-the-count-of-numbers-which-are-not-special1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: nonSpecialCount case1: expected 11 got 8 |
Failed | IN9 371 OUT739 |
| × | LcbRectangleCornerReachableLiveCodeBench / leetcode / hard: 3478 check-if-the-rectangle-corner-is-reachable1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: canReachCorner case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
20:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 522 OUT2 587 |
| ✓ | LcbShortestDistanceQueriesLiveCodeBench / leetcode / medium: 3517 shortest-distance-after-road-addition-queries-i1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 463 OUT444 |
| ✓ | LcbSnakeFinalPositionLiveCodeBench / leetcode / easy: 3533 snake-in-matrix1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 398 OUT129 |
Code Generation - LiveCodeBench v3 LeetCode (set 5)
5 passed3 failed |
62.5% |
IN74 550 OUT23 106 | |
| ✓ | LcbAlternatingGroupsCircleLiveCodeBench / leetcode / easy: alternating-groups-i (LCB id 3463)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 310 OUT125 |
| ✓ | LcbClearDigitsLiveCodeBench / leetcode / easy: clear-digits (LCB id 3447)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 257 OUT132 |
| × | LcbCountInversionPermutationsLiveCodeBench / leetcode / hard: count-the-number-of-inversions (LCB id 3460)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countInversionPermutations case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
643:17: Expected '>' to close the List type, found '['.
Values:
run.Compiled = false |
Failed | IN9 412 OUT18 435 |
| ✓ | LcbCountMonotonicPairsLiveCodeBench / leetcode / hard: find-the-count-of-monotonic-pairs-i (LCB id 3535)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 374 OUT3 428 |
| ✓ | LcbFirstPlayerWinKGamesLiveCodeBench / leetcode / medium: find-the-first-player-to-win-k-games-in-a-row (LCB id 3413)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 358 OUT158 |
| × | LcbMaxGoodSubsequenceLengthLiveCodeBench / leetcode / medium: find-the-maximum-length-of-a-good-subsequence-i (LCB id 3456)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxGoodSubsequenceLength case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
24:21: Variable 'dp' has type 'List<List<int>>' but its initializer has type 'List<int>'.
Values:
run.Compiled = false |
Failed | IN9 297 OUT514 |
| ✓ | LcbMinOpsBinaryToOneLiveCodeBench / leetcode / medium: minimum-operations-to-make-binary-array-elements-equal-to-one-i (LCB id 3475)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 291 OUT204 |
| × | LcbMinOpsDivisibleThreeLiveCodeBench / leetcode / easy: find-minimum-operations-to-make-all-elements-divisible-by-three (LCB id 3476)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOpsDivisibleThree case0: expected 3 got 4 |
Failed | IN9 251 OUT110 |
Code Generation - LiveCodeBench v3 LeetCode (set 6)
4 passed4 failed |
50.0% |
IN74 715 OUT1 921 | |
| ✓ | LcbCoinGameWinnerLiveCodeBench / leetcode / easy: find-the-winning-player-in-coin-game (LCB id 3511)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 306 OUT59 |
| ✓ | LcbCountSubarraysAndKLiveCodeBench / leetcode / hard: number-of-subarrays-with-and-value-of-k (LCB id 3466)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 276 OUT147 |
| × | LcbCountWinningPlayersLiveCodeBench / leetcode / easy: find-the-number-of-winning-players (LCB id 3519)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countWinningPlayers case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:48: Expected ';' after the variable declaration, found 'color'.
Values:
run.Compiled = false |
Failed | IN9 408 OUT213 |
| × | LcbMaxPointsEnemyBattlesLiveCodeBench / leetcode / medium: maximum-points-after-enemy-battles (LCB id 3264)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxPointsEnemyBattles case0: expected 3 got 1 |
Failed | IN9 373 OUT212 |
| × | LcbMinChangesEqualDiffsLiveCodeBench / leetcode / medium: minimum-array-changes-to-make-differences-equal (LCB id 3498)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minChangesEqualDiffs case0: expected 2 got 3 |
Failed | IN9 327 OUT186 |
| ✓ | LcbMinFlipsPalindromicGridLiveCodeBench / leetcode / medium: minimum-number-of-flips-to-make-binary-grid-palindromic-i (LCB id 3526)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 322 OUT250 |
| ✓ | LcbMinLengthAfterOpsLiveCodeBench / leetcode / medium: minimum-length-of-string-after-operations (LCB id 3455)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 294 OUT160 |
| × | LcbTimeTakenMarkNodesLiveCodeBench / leetcode / hard: time-taken-to-mark-all-nodes (LCB id 3532)1 AI query
0 tool callscategory:CodeGenerationcategory:V3@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: timeTakenMarkNodes case2: expected [4, 6, 3, 5, 5] got [2,4,3,5,5] |
Failed | IN9 409 OUT694 |
Code Generation - LiveCodeBench v4 AtCoder (set 1)
4 passed4 failed1 skipped |
50.0% |
IN74 949 OUT3 219 | |
| ✓ | LcbBallsAndBagQueryLiveCodeBench / atcoder / medium: abc366_c Balls and Bag Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 343 OUT305 |
| × | LcbCuboidSumQueryLiveCodeBench / atcoder / medium: abc366_d Cuboid Sum Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CuboidSumQuery case0: expected [10
26] got [30
42] |
Failed | IN9 453 OUT503 |
| ✓ | LcbCutTrailingZerosLiveCodeBench / atcoder / easy: abc367_b Cut .0 (Pattern B, stdin) NOTE (triage): float-formatting output — model must print X with no trailing zeros and no trailing decimal point. Compared as a trimmed string. Flagged for coordinator.1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 335 OUT226 |
| ✓ | LcbElection2LiveCodeBench / atcoder / easy: abc366_a Election 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 349 OUT118 |
| × | LcbManhattanEllipseLiveCodeBench / atcoder / hard: abc366_e Manhattan Multifocal Ellipse (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ManhattanEllipse case2: expected [419] got [2589] |
Failed | IN9 366 OUT885 |
| × | LcbMaximumCompositionLiveCodeBench / atcoder / hard: abc366_f Maximum Composition (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MaximumComposition case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:45: Expected ')' to close the argument list, found 'best'.
Values:
run.Compiled = false |
Failed | IN9 370 OUT518 |
| × | LcbShoutEverydayLiveCodeBench / atcoder / easy: abc367_a Shout Everyday (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ShoutEveryday case0: expected [Yes] got [No] |
Failed | IN9 330 OUT155 |
| ✓ | LcbVerticalWritingLiveCodeBench / atcoder / easy: abc366_b Vertical Writing (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 403 OUT509 |
| – | LcbXorNeighborsLiveCodeBench / atcoder / hard: abc366_g XOR Neighbors (Pattern B, stdin) NOTE (triage): non-unique answer — any valid assignment is accepted, so exact-string comparison against the sample output will reject other correct solutions. Flagged for coordinator.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
Code Generation - LiveCodeBench v4 AtCoder (set 2)
3 passed5 failed |
37.5% |
IN75 084 OUT36 765 | |
| ✓ | LcbAddMultiplyQueriesLiveCodeBench / atcoder / hard: abc368_g Add and Multiply Queries (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 427 OUT490 |
| ✓ | LcbCardCutLiveCodeBench / atcoder / easy: abc368_a Cut (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 349 OUT240 |
| × | LcbDecreaseTwoMaxLiveCodeBench / atcoder / easy: abc368_b Decrease 2 max elements (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: DecreaseTwoMax case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 334 OUT201 |
| × | LcbEnumerateSequencesLiveCodeBench / atcoder / medium: abc367_c Enumerate Sequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: EnumerateSequences case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:30: Expected ')' to close the argument list, found 'sb'.
Values:
run.Compiled = false |
Failed | IN9 346 OUT434 |
| × | LcbPedometerLiveCodeBench / atcoder / medium: abc367_d Pedometer (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Pedometer case2: expected [11] got [12] |
Failed | IN9 399 OUT305 |
| × | LcbPermuteKTimesLiveCodeBench / atcoder / hard: abc367_e Permute K times (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PermuteKTimes case0: expected [7 2 3 5 1 9 3] got [7 2 3 5 1 9 11] |
Failed | IN9 374 OUT579 |
| ✓ | LcbRearrangeQueryLiveCodeBench / atcoder / hard: abc367_f Rearrange Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 427 OUT872 |
| × | LcbSumXorKLiveCodeBench / atcoder / hard: abc367_g Sum of (XOR^K or 0) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: SumXorK case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 428 OUT33 644 |
Code Generation - LiveCodeBench v4 AtCoder (set 3)
4 passed5 failed |
44.4% |
IN84 663 OUT13 785 | |
| ✓ | LcbArrangeArithmeticLiveCodeBench / atcoder / abc369_a: 369 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 341 OUT133 |
| × | LcbAsFarAsPossibleLiveCodeBench / atcoder / abc369_g: As far as possible (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AsFarAsPossible case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
33:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 500 OUT10 942 |
| × | LcbBonusExpLiveCodeBench / atcoder / abc369_d: Bonus EXP (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: BonusExp case0: expected [28] got [29] |
Failed | IN9 405 OUT200 |
| × | LcbCountArithmeticSubarraysLiveCodeBench / atcoder / abc369_c: Count Arithmetic Subarrays (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CountArithmeticSubarrays case0: expected [8] got [5] |
Failed | IN9 386 OUT199 |
| ✓ | LcbDividingGameLiveCodeBench / atcoder / abc368_f: Dividing Game (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 390 OUT305 |
| ✓ | LcbMinimumSteinerTreeLiveCodeBench / atcoder / abc368_d: Minimum Steiner Tree (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 348 OUT483 |
| ✓ | LcbPiano3LiveCodeBench / atcoder / abc369_b: Piano 3 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 438 OUT236 |
| × | LcbTrainDelayLiveCodeBench / atcoder / abc368_e: Train Delay (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TrainDelay case0: expected [0 10 0 0 5] got [-1152921504606846976 10 0 -1152921504606846976 5] |
Failed | IN9 472 OUT1 103 |
| × | LcbTripleAttackLiveCodeBench / atcoder / abc368_c: Triple Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: TripleAttack case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 383 OUT184 |
Code Generation - LiveCodeBench v4 AtCoder (set 4)
2 passed6 failed |
25.0% |
IN75 423 OUT12 354 | |
| × | LcbAvoidKPartitionLiveCodeBench / atcoder / abc370_e: Avoid K Partition (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AvoidKPartition case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:16: Type 'List<long>' has no supported method named 'Last' with 0 argument(s).
Values:
run.Compiled = false |
Failed | IN9 448 OUT1 107 |
| ✓ | LcbBinaryAlchemyLiveCodeBench / atcoder / abc370_b: Binary Alchemy (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 380 OUT234 |
| × | LcbCakeDivisionLiveCodeBench / atcoder / abc370_f: Cake Division (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CakeDivision case0: expected [13 1] got [0 5] |
Failed | IN9 499 OUT1 143 |
| × | LcbCrossExplosionLiveCodeBench / atcoder / abc370_d: Cross Explosion (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CrossExplosion case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:48: Expected ';' after the variable declaration, found 'c'.
Values:
run.Compiled = false |
Failed | IN9 476 OUT572 |
| × | LcbDivisibleByThreeLiveCodeBench / atcoder / abc370_g: Divisible by 3 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: DivisibleByThree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
136:24: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed | IN9 324 OUT4 298 |
| ✓ | LcbRaiseBothHandsLiveCodeBench / atcoder / abc370_a: Raise Both Hands (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 395 OUT115 |
| × | LcbSightseeingTourLiveCodeBench / atcoder / abc369_e: Sightseeing Tour (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SightseeingTour case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
74:44: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '('.
Values:
run.Compiled = false |
Failed | IN9 496 OUT4 568 |
| × | LcbWordLadderLiveCodeBench / atcoder / abc370_c: Word Ladder (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WordLadder case0: expected [3
acbe
acbc
bcbc] got [3
bdbe
bcbe
bcbc] |
Failed | IN9 405 OUT317 |
Code Generation - LiveCodeBench v4 AtCoder (set 5)
5 passed4 failed |
55.6% |
IN84 625 OUT4 286 | |
| × | LcbCountABCAgainLiveCodeBench / atcoder / medium: abc372_c Count ABC Again (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 371 OUT447 |
| ✓ | LcbDistinctSubarraySumLiveCodeBench / atcoder / hard: abc371_e I Hate Sigma Problems (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 339 OUT217 |
| × | LcbKthLargestConnectedLiveCodeBench / atcoder / hard: abc372_e K (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: KthLargestConnected case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
27:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 409 OUT621 |
| ✓ | LcbLexSmallestPermLiveCodeBench / atcoder / hard: abc371_g Lexicographically Smallest Permutation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 378 OUT525 |
| × | LcbMakeIsomorphicLiveCodeBench / atcoder / medium: abc371_c Make Isomorphic (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MakeIsomorphic case0: expected [9] got [12] |
Failed | IN9 532 OUT891 |
| ✓ | LcbMiddleBrotherLiveCodeBench / atcoder / easy: abc371_a Jiro (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 358 OUT244 |
| × | LcbNarrowRoadLiveCodeBench / atcoder / hard: abc371_f Takahashi in Narrow Road (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NarrowRoad case0: expected [239] got [110] |
Failed | IN9 419 OUT501 |
| ✓ | LcbOneDCountryLiveCodeBench / atcoder / medium: abc371_d 1D Country (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 411 OUT622 |
| ✓ | LcbTaroNameLiveCodeBench / atcoder / easy: abc371_b Taro (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 408 OUT218 |
Code Generation - LiveCodeBench v4 AtCoder (set 6)
5 passed2 failed1 skipped |
71.4% |
IN65 641 OUT4 006 | |
| ✓ | LcbAxByLessCLiveCodeBench / atcoder / hard: abc372_g Ax + By < C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 434 OUT475 |
| ✓ | LcbDeleteDotsLiveCodeBench / atcoder / easy: abc372_a delete . (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 283 OUT77 |
| ✓ | LcbDiminishingKnapsackLiveCodeBench / atcoder / hard: abc373_f Knapsack with Diminishing Values (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 375 OUT2 387 |
| – | LcbNoCrossMatchingLiveCodeBench / atcoder / hard: abc373_g No Cross Matching (Pattern B, stdin) NOTE: non-unique answer — any non-crossing perfect matching R is valid; the exact-match oracle may reject a different valid R.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| ✓ | LcbOneDKeyboardLiveCodeBench / atcoder / easy: abc373_b 1D Keyboard (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 376 OUT152 |
| ✓ | LcbSeptemberLengthsLiveCodeBench / atcoder / easy: abc373_a September (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 313 OUT74 |
| × | LcbTeleportingWalksLiveCodeBench / atcoder / hard: abc372_f Teleporting Takahashi 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: TeleportingWalks case1 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 494 OUT509 |
| × | LcbVisibleBuildingsLiveCodeBench / atcoder / medium: abc372_d Buildings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: VisibleBuildings case0: expected [3 2 2 1 0] got [1 0 1 0 0] |
Failed | IN9 366 OUT332 |
Code Generation - LiveCodeBench v4 AtCoder (set 7)
2 passed4 failed1 skipped |
33.3% |
IN56 358 OUT5 759 | |
| × | LcbAnnoyingStringLiveCodeBench / atcoder / hard: arc181_b Annoying String Problem (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AnnoyingString case0: expected [Yes
No
No] got [No
No
No] |
Failed | IN9 393 OUT487 |
| × | LcbIncDecAgainLiveCodeBench / atcoder / hard: arc182_d Increment Decrement Again (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: IncDecAgain case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
147:14: Unknown variable 'System'.
148:12: Unknown variable 'System'.
Values:
run.Compiled = false |
Failed | IN9 438 OUT1 598 |
| ✓ | LcbMaxAiBjLiveCodeBench / atcoder / medium: abc373_c Max Ai+Bj (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 304 OUT231 |
| × | LcbPrefixBubbleSortLiveCodeBench / atcoder / hard: arc181_d Prefix Bubble Sort (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PrefixBubbleSort case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
9:67: Char literals must contain exactly one character and close with '''.
24:15: Char literals must contain exactly one character and close with '''.
28:119: Char literals must contain exactly one character and close with '''.
30:5: Char literals must contain exactly one character and close with '''.
Values:
run.Compiled = false |
Failed | IN9 424 OUT1 420 |
| – | LcbRowColumnOrderLiveCodeBench / atcoder / hard: arc181_c Row and Column Order (Pattern B, stdin) TRIAGE: non-unique answer (any valid grid accepted) — exact string compare may fail correct solutions.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| × | LcbSortLeftRightLiveCodeBench / atcoder / medium: arc181_a Sort Left and Right (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 396 OUT662 |
| ✓ | LcbWinElectionLiveCodeBench / atcoder / hard: abc373_e How to Win the Election (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 403 OUT1 361 |
Code Generation - LiveCodeBench v4 AtCoder (set 8)
1 passed5 failed1 skipped |
16.7% |
IN56 172 OUT31 219 | |
| × | LcbChmaxRushLiveCodeBench / atcoder / hard: arc182_a Chmax Rush! (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ChmaxRush case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
399:22: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 490 OUT6 343 |
| × | LcbDivisorsOfProductLiveCodeBench / atcoder / hard: arc182_c Sum of Number of Divisors of Product (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: DivisorsOfProduct case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
596:20: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 335 OUT15 496 |
| – | LcbKeepMatchedLiveCodeBench / atcoder / hard: arc183_d Keep Perfectly Matched (Pattern B, stdin) TRIAGE: non-unique answer (any maximizing procedure accepted) — exact string compare may fail correct solutions.0 AI queries
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| × | LcbMedianGoodSeqLiveCodeBench / atcoder / medium: arc183_a Median of Good Sequences (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MedianGoodSeq case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
157:32: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 307 OUT4 979 |
| × | LcbNearAssignmentLiveCodeBench / atcoder / hard: arc183_b Near Assignment (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: NearAssignment case0: expected [Yes
Yes
No
Yes] got [Yes
Yes
No
No] |
Failed | IN9 372 OUT442 |
| × | LcbNotArgmaxLiveCodeBench / atcoder / hard: arc183_c Not Argmax (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: NotArgmax case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
66:9: Expected ';' after the statement, found '{'.
Values:
run.Compiled = false |
Failed | IN9 357 OUT3 712 |
| ✓ | LcbSumMinModLinearLiveCodeBench / atcoder / hard: arc182_e Sum of Min of Mod of Linear (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 311 OUT247 |
Code Generation - LiveCodeBench v4 LeetCode (set 1)
3 passed3 failed |
50.0% |
IN55 973 OUT1 834 | |
| × | LcbAlmostEqualPairsLiveCodeBench / leetcode / medium: count-almost-equal-pairs-i (id 3534)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: almostequal case0: expected 2 got 1 |
Failed | IN9 330 OUT463 |
| ✓ | LcbBeautifulArraySumLiveCodeBench / leetcode / medium: find-the-minimum-possible-sum-of-a-beautiful-array (id 3026)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 319 OUT242 |
| ✓ | LcbFinalArrayStateLiveCodeBench / leetcode / easy: final-array-state-after-k-multiplication-operations-i (id 3555)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 342 OUT128 |
| × | LcbKConstraintSubstringsLiveCodeBench / leetcode / easy: count-substrings-that-satisfy-k-constraint-i (id 3543)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: kconstraint case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'countKConstraintSubstrings' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, CountKConstraintSubstrings.
Values:
run.Compiled = false |
Failed | IN9 323 OUT559 |
| ✓ | LcbLargestKPalindromeLiveCodeBench / leetcode / hard: find-the-largest-palindrome-divisible-by-k (id 3552)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 294 OUT267 |
| × | LcbMaxEnergyBoostLiveCodeBench / leetcode / medium: maximum-energy-boost-from-two-drinks (id 3525)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: energyboost case0: expected 5 got 7 |
Failed | IN9 365 OUT175 |
Code Generation - LiveCodeBench v4 LeetCode (set 2)
3 passed3 failed |
50.0% |
IN56 060 OUT4 070 | |
| ✓ | LcbConvertDateBinaryLiveCodeBench / leetcode / easy: convert-date-to-binary (id 3567)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 313 OUT257 |
| ✓ | LcbKthNearestObstacleLiveCodeBench / leetcode / medium: k-th-nearest-obstacle-queries (id 3495)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 382 OUT194 |
| × | LcbMaxCellScoreLiveCodeBench / leetcode / hard: select-cells-in-grid-with-maximum-score (id 3563)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxcell case1: expected 15 got 8 |
Failed | IN9 310 OUT441 |
| × | LcbMaxRangeScoreLiveCodeBench / leetcode / medium: maximize-score-of-numbers-in-ranges (id 3485)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: rangescore case0: expected 4 got 0 |
Failed | IN9 305 OUT824 |
| × | LcbMaxXorSubarrayLiveCodeBench / leetcode / hard: maximum-xor-score-subarray-queries (id 3551)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxxor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
93:13: Expected ';' after the statement, found ','.
Values:
run.Compiled = false |
Failed | IN9 445 OUT2 225 |
| ✓ | LcbSameChessboardColorLiveCodeBench / leetcode / easy: check-if-two-chessboard-squares-have-the-same-color (id 3553)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 305 OUT129 |
Code Generation - LiveCodeBench v4 LeetCode (set 3)
4 passed2 failed |
66.7% |
IN56 301 OUT2 069 | |
| × | LcbKnightKillPawnsLiveCodeBench / leetcode / hard: maximum-number-of-moves-to-kill-all-pawns (LCB 3560)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: knightPawns case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 475 OUT921 |
| ✓ | LcbMaxMultiplicationScoreLiveCodeBench / leetcode / medium: maximum-multiplication-score (LCB 3518)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 420 OUT234 |
| × | LcbMinValidStringsLiveCodeBench / leetcode / medium: minimum-number-of-valid-strings-to-form-target-i (LCB 3559)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minValid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minValidStrings' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinValidStrings.
Values:
run.Compiled = false |
Failed | IN9 317 OUT269 |
| ✓ | LcbPowerKSizeSubarraysLiveCodeBench / leetcode / medium: find-the-power-of-k-size-subarrays-i (LCB 3522)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 385 OUT181 |
| ✓ | LcbReachEndMaxScoreLiveCodeBench / leetcode / medium: reach-end-of-array-with-max-score (LCB 3528)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 341 OUT182 |
| ✓ | LcbThreeRooksMaxSumLiveCodeBench / leetcode / hard: maximum-value-sum-by-placing-three-rooks-i (LCB 3550)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 363 OUT282 |
Code Generation - LiveCodeBench v4 LeetCode (set 4)
2 passed4 failed |
33.3% |
IN56 452 OUT6 418 | |
| × | LcbCountGoodIntegersLiveCodeBench / leetcode / hard: find-the-count-of-good-integers (LCB 3548)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: goodInt case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
123:38: Expected a parameter name, found 'long'.
Values:
run.Compiled = false |
Failed | IN9 402 OUT1 057 |
| × | LcbGenerateKeyNumbersLiveCodeBench / leetcode / easy: find-the-key-of-the-numbers (LCB 3568)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: genKey case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:32: Type 'string' has no supported method named 'PadLeft'.
4:32: Type 'string' has no supported method named 'PadLeft'.
5:32: Type 'string' has no supported method named 'PadLeft'.
Values:
run.Compiled = false |
Failed | IN9 426 OUT203 |
| ✓ | LcbHashDividedStringLiveCodeBench / leetcode / medium: hash-divided-string (LCB 3540)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 403 OUT171 |
| ✓ | LcbMaxSequenceValueLiveCodeBench / leetcode / hard: find-the-maximum-sequence-value-of-array (LCB 3575)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 366 OUT4 398 |
| × | LcbMinDamageToBobLiveCodeBench / leetcode / hard: minimum-amount-of-damage-dealt-to-bob (LCB 3531)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minDmg case1: expected 20 got 30 |
Failed | IN9 401 OUT200 |
| × | LcbSafeWalkGridLiveCodeBench / leetcode / medium: find-a-safe-walk-through-a-grid (LCB 3558)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: safeWalk case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 454 OUT389 |
Code Generation - LiveCodeBench v4 LeetCode (set 5)
2 passed4 failed |
33.3% |
IN56 067 OUT1 897 | |
| × | LcbKthCharacterGameLiveCodeBench 3600 / leetcode / easy: find the k-th character in string game I (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson) == FromJson<string>(expected[i])
Context: kthCharacterGame case0: expected "b" got "c" |
Failed | IN9 338 OUT93 |
| × | LcbLongestIncreasingPathLiveCodeBench 3571 / leetcode / hard: length of the longest increasing path (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: longestIncreasingPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:20: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed | IN9 411 OUT545 |
| × | LcbMountainHeightZeroLiveCodeBench 3496 / leetcode / medium: minimum number of seconds to make mountain height zero (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: mountainHeightZero case1: expected 12 got 11 |
Failed | IN9 411 OUT311 |
| ✓ | LcbReportSpamLiveCodeBench 3541 / leetcode / medium: report spam message (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 294 OUT102 |
| × | LcbSubstringsVowelsKConsonantsLiveCodeBench 3570 / leetcode / medium: count of substrings containing every vowel and k consonants I (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: substringsVowelsKConsonants case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
43:13: The foreach variable has type 'char' but the iterated elements have type 'string'.
Values:
run.Compiled = false |
Failed | IN9 298 OUT381 |
| ✓ | LcbValidSubstringCountLiveCodeBench 3573 / leetcode / medium: count substrings that can be rearranged to contain a string I (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 315 OUT465 |
Code Generation - LiveCodeBench v4 LeetCode (set 6)
3 passed3 failed |
50.0% |
IN56 092 OUT2 786 | |
| ✓ | LcbFirstAlmostEqualSubstringLiveCodeBench 3580 / leetcode / hard: find the occurrence of first almost equal substring (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 330 OUT180 |
| × | LcbLexSmallestValidSequenceLiveCodeBench 3584 / leetcode / medium: find the lexicographically smallest valid sequence (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: lexSmallestValidSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
36:15: Variable 'BestSeq' has type 'List<int>' but its initializer has type 'null'.
109:25: Operator '==' cannot compare 'List<int>' and 'null'. Only numbers, bools, chars, strings, date/time values and recursively comparable containers/structs can be compared.
Values:
run.Compiled = false |
Failed | IN9 411 OUT1 145 |
| ✓ | LcbMaxBinaryConcatenationLiveCodeBench 3579 / leetcode / medium: maximum possible number by binary concatenation (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 298 OUT416 |
| × | LcbMaxUniqueTowerHeightsLiveCodeBench 3510 / leetcode / medium: maximize the total height of unique towers (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: maxUniqueTowerHeights case0 runtime error: Integer overflow.
Values:
run.Success = false |
Failed | IN9 335 OUT182 |
| ✓ | LcbMinAfterDigitSumLiveCodeBench 3606 / leetcode / easy: minimum element after replacement with digit sum (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 291 OUT160 |
| × | LcbSortedGcdPairQueriesLiveCodeBench 3583 / leetcode / hard: sorted GCD pair queries (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V4@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: sortedGcdPairQueries case0: expected [1, 2, 2] got [1,1,1] |
Failed | IN9 427 OUT703 |
Code Generation - LiveCodeBench v5 AtCoder (set 1)
4 passed6 failed2 skipped |
40.0% |
IN93 523 OUT6 153 | |
| × | LcbAbaPalindromeLiveCodeBench / atcoder / medium: abc375_d ABA (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AbaPalindrome case0: expected [5] got [11] |
Failed | IN9 321 OUT193 |
| – | LcbLaserMarkingLiveCodeBench / atcoder / medium: abc374_d Laser Marking (Pattern B, stdin) [float output, tolerance 1e-6]0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Skipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped | IN0 OUT0 |
| × | LcbOnlyOneProductNameLiveCodeBench / atcoder / hard: abc374_g Only One Product Name (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: OnlyOneProductName case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
12:9: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 359 OUT1 484 |
| ✓ | LcbSeatsLiveCodeBench / atcoder / easy: abc375_a Seats (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 314 OUT109 |
| × | LcbSensorOptimizationLiveCodeBench / atcoder / hard: abc374_e Sensor Optimization Dilemma 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SensorOptimization case0: expected [4] got [3] |
Failed | IN9 410 OUT703 |
| × | LcbSeparatedLunchLiveCodeBench / atcoder / medium: abc374_c Separated Lunch (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SeparatedLunch case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 331 OUT277 |
| × | LcbShippingLiveCodeBench / atcoder / hard: abc374_f Shipping (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Shipping case0: expected [2] got [-8] |
Failed | IN9 413 OUT2 129 |
| ✓ | LcbSpiralRotationLiveCodeBench / atcoder / medium: abc375_c Spiral Rotation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 381 OUT415 |
| ✓ | LcbTakahashiSanLiveCodeBench / atcoder / easy: abc374_a Takahashi san 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 283 OUT48 |
| × | LcbThreeTeamDivisionLiveCodeBench / atcoder / hard: abc375_e 3 Team Division (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ThreeTeamDivision case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
10:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 400 OUT665 |
| – | LcbTravelingTakahashiLiveCodeBench / atcoder / easy: abc375_b Traveling Takahashi Problem (Pattern B, stdin) [float output, tolerance 1e-6]0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench Skipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped | IN0 OUT0 |
| ✓ | LcbUnvarnishedReportLiveCodeBench / atcoder / easy: abc374_b Unvarnished Report (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 311 OUT130 |
Code Generation - LiveCodeBench v5 AtCoder (set 2)
6 passed5 failed |
54.5% |
IN103 532 OUT12 585 | |
| ✓ | LcbAvoidKnightAttackLiveCodeBench / atcoder / medium: abc377_c Avoid Knight Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 437 OUT790 |
| ✓ | LcbAvoidRookAttackLiveCodeBench / atcoder / easy: abc377_b Avoid Rook Attack (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 385 OUT224 |
| ✓ | LcbCandyButtonLiveCodeBench / atcoder / easy: abc376_a Candy Button (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 353 OUT183 |
| ✓ | LcbCycleMinEdgesLiveCodeBench / atcoder / medium: abc376_d Cycle (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 375 OUT384 |
| × | LcbHandsOnRingEasyLiveCodeBench / atcoder / medium: abc376_b Hands on Ring (Easy) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: HandsOnRingEasy case0: expected [8] got [6] |
Failed | IN9 419 OUT241 |
| × | LcbHandsOnRingHardLiveCodeBench / atcoder / hard: abc376_f Hands on Ring (Hard) (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: HandsOnRingHard case0: expected [6] got [5] |
Failed | IN9 424 OUT2 478 |
| × | LcbMaxTimesSumLiveCodeBench / atcoder / hard: abc376_e Max x Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MaxTimesSum case0: expected [42
60
14579] got [77
60
19266] |
Failed | IN9 411 OUT941 |
| × | LcbPrepareAnotherBoxLiveCodeBench / atcoder / medium: abc376_c Prepare Another Box (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PrepareAnotherBox case0: expected [3] got [7] |
Failed | IN9 408 OUT253 |
| ✓ | LcbRoadBlockedLiveCodeBench / atcoder / hard: abc375_f Road Blocked (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 434 OUT670 |
| ✓ | LcbRoadBlocked2LiveCodeBench / atcoder / hard: abc375_g Road Blocked 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 423 OUT1 230 |
| × | LcbTreasureHuntingLiveCodeBench / atcoder / hard: abc376_g Treasure Hunting (Pattern B, stdin) [answer modulo 998244353]1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: TreasureHunting case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
350:20: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 463 OUT5 191 |
Code Generation - LiveCodeBench v5 AtCoder (set 3)
7 passed7 failed |
50.0% |
IN130 622 OUT4 914 | |
| × | LcbAddOneEdge2LiveCodeBench / atcoder abc378_f: Add One Edge 2 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AddOneEdge2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
32:46: Expected ')' to close the argument list, found 'parentRoot'.
Values:
run.Compiled = false |
Failed | IN9 306 OUT650 |
| × | LcbAvoidQueenAttackLiveCodeBench / atcoder abc377_f: Avoid Queen Attack (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AvoidQueenAttack case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:48: Expected ')' to close the argument list, found 'c'.
Values:
run.Compiled = false |
Failed | IN9 377 OUT476 |
| × | LcbCountSimplePathsLiveCodeBench / atcoder abc378_d: Count Simple Paths (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CountSimplePaths case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:42: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 325 OUT732 |
| ✓ | LcbEditToMatchLiveCodeBench / atcoder abc377_g: Edit to Match (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 380 OUT327 |
| × | LcbEverlastingLidsLiveCodeBench / atcoder abc378_g: Everlasting LIDS (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: EverlastingLids case0: expected [10] got [0] |
Failed | IN9 357 OUT166 |
| ✓ | LcbGarbageCollectionLiveCodeBench / atcoder abc378_b: Garbage Collection (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 347 OUT335 |
| ✓ | LcbManySegments2LiveCodeBench / atcoder abc377_d: Many Segments 2 (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 337 OUT324 |
| × | LcbModSigmaProblemLiveCodeBench / atcoder abc378_e: Mod Sigma Problem (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ModSigmaProblem case0: expected [10] got [1] |
Failed | IN9 313 OUT424 |
| ✓ | LcbPairingLiveCodeBench / atcoder abc378_a: Pairing (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 298 OUT155 |
| × | LcbPermuteKTimesLiveCodeBench / atcoder abc377_e: Permute K times 2 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PermuteKTimes case0: expected [6 1 3 2 4 5] got [2 4 3 5 6 1] |
Failed | IN9 333 OUT554 |
| ✓ | LcbRearrangingAbcLiveCodeBench / atcoder abc377_a: Rearranging ABC (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 294 OUT133 |
| ✓ | LcbRepeatingLiveCodeBench / atcoder abc378_c: Repeating (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 315 OUT239 |
| ✓ | LcbStrawberriesLiveCodeBench / atcoder abc379_b: Strawberries (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 318 OUT194 |
| × | LcbSumOfAllSubstringsLiveCodeBench / atcoder abc379_e: Sum of All Substrings (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SumOfAllSubstrings case0: expected [514] got [544] |
Failed | IN9 322 OUT205 |
Code Generation - LiveCodeBench v5 AtCoder (set 4)
7 passed7 failed |
50.0% |
IN130 831 OUT13 469 | |
| ✓ | Lcb1122StringLiveCodeBench / atcoder abc381_a: 11/22 String (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 403 OUT240 |
| × | Lcb1122SubsequenceLiveCodeBench / atcoder abc381_e: 11/22 Subsequence (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 398 OUT750 |
| × | LcbAnotherShuffleWindowLiveCodeBench / atcoder abc380_g: Another Shuffle Window (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AnotherShuffleWindow case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:36: Operator '%' requires numeric operands, but got 'long' and 'string'.
Values:
run.Compiled = false |
Failed | IN9 405 OUT975 |
| × | LcbBucketToolLiveCodeBench / atcoder abc380_e: 1D Bucket Tool (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 341 OUT751 |
| × | LcbBuildings2LiveCodeBench / atcoder abc379_f: Buildings 2 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Buildings2 case0: expected [2
0
1] got [2
0
0] |
Failed | IN9 351 OUT6 159 |
| × | LcbCountGrid3LiveCodeBench / atcoder abc379_g: Count Grid 3 (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CountGrid3 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
85:14: Expected ';' after the variable declaration, found ','.
Values:
run.Compiled = false |
Failed | IN9 333 OUT1 562 |
| ✓ | LcbCyclicLiveCodeBench / atcoder abc379_a: Cyclic (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 307 OUT130 |
| ✓ | LcbDigitPattern123233LiveCodeBench / atcoder abc380_a: 123233 (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 294 OUT165 |
| ✓ | LcbExchangeGameLiveCodeBench / atcoder abc380_f: Exchange Game (hard)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 385 OUT1 076 |
| ✓ | LcbHomeGardenLiveCodeBench / atcoder abc379_d: Home Garden (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 321 OUT325 |
| ✓ | LcbHurdleParsingLiveCodeBench / atcoder abc380_b: Hurdle Parsing (easy)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 305 OUT151 |
| × | LcbMoveSegmentLiveCodeBench / atcoder abc380_c: Move Segment (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MoveSegment case0: expected [010011111000001] got [01001111100000001] |
Failed | IN9 316 OUT612 |
| ✓ | LcbSowingStonesLiveCodeBench / atcoder abc379_c: Sowing Stones (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 348 OUT332 |
| × | LcbStrangeMirroringLiveCodeBench / atcoder abc380_d: Strange Mirroring (medium)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: StrangeMirroring case0: expected [a B A b A b a B A b a B a B A b] got [a B a B a B a B a B a B a B a B] |
Failed | IN9 324 OUT241 |
Code Generation - LiveCodeBench v5 AtCoder (set 5)
6 passed6 failed1 broken |
46.2% |
IN122 813 OUT69 041 | |
| ✓ | Lcb1122StringLiveCodeBench / atcoder / easy: 1122 String (abc381_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 386 OUT241 |
| × | Lcb1122SubarrayLiveCodeBench / atcoder / hard: 1122 Substring (abc381_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: 1122Subarray case0 runtime error: The dictionary does not contain the key 1. Use ContainsKey to check for it first.
Values:
run.Success = false |
Failed | IN9 426 OUT958 |
| ✓ | LcbDailyCookieLiveCodeBench / atcoder / easy: Daily Cookie (abc382_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 392 OUT149 |
| × | LcbDailyCookie2LiveCodeBench / atcoder / easy: Daily Cookie 2 (abc382_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DailyCookie2 case0: expected [.@...] got [....@] |
Failed | IN9 414 OUT152 |
| × | LcbFallingBarsLiveCodeBench / atcoder / hard: Falling Bars (abc382_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: FallingBars case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
30:53: Expected ';' after the variable declaration, found 'i'.
Values:
run.Compiled = false |
Failed | IN9 535 OUT700 |
| ✓ | LcbHumidifier1LiveCodeBench / atcoder / easy: Humidifier 1 (abc383_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 385 OUT160 |
| × | LcbHumidifier2LiveCodeBench / atcoder / medium: Humidifier 2 (abc383_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: Humidifier2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:22: Variable 'isFloor' has type 'List<List<bool>>' but its initializer has type 'List<bool>'.
Values:
run.Compiled = false |
Failed | IN9 449 OUT457 |
| ✓ | LcbHumidifier3LiveCodeBench / atcoder / medium: Humidifier 3 (abc383_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 439 OUT983 |
| × | LcbKaitenSushiLiveCodeBench / atcoder / medium: Kaiten Sushi (abc382_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 445 OUT489 |
| ✓ | LcbKeepDistanceLiveCodeBench / atcoder / hard: Keep Distance (abc382_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 408 OUT463 |
| ✓ | LcbMax1122SubstringLiveCodeBench / atcoder / medium: 11/22 Substring (abc381_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 511 OUT797 |
| × | LcbSumOfMaxMatchingLiveCodeBench / atcoder / hard: Sum of Max Matching (abc383_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SumOfMaxMatching case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
47:20: Expected ')' to close the parenthesized expression, found 'i'.
Values:
run.Compiled = false |
Failed | IN9 497 OUT1 348 |
| ! | LcbTileDistance3LiveCodeBench / atcoder / hard: Tile Distance 3 (abc382_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Skipped: AI output broken/partial |
Broken | IN9 526 OUT62 144 |
Code Generation - LiveCodeBench v5 AtCoder (set 6)
6 passed7 failed1 skipped |
46.2% |
IN122 617 OUT7 348 | |
| × | LcbAbsSumLiveCodeBench / atcoder / hard: Abs Sum (abc384_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AbsSum case0: expected [1
4
2
6] got [9
15
16
26] |
Failed | IN9 446 OUT993 |
| ✓ | LcbArcDivisionLiveCodeBench / atcoder / easy: ARC Division (abc384_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 493 OUT202 |
| × | LcbDoubleSum2LiveCodeBench / atcoder / hard: Double Sum 2 (abc384_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: DoubleSum2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1:2: Char literals must contain exactly one character and close with '''.
3:60: Unexpected character '→'.
5:52: Unexpected character '≤'.
15:69: Unexpected character '≤'.
17:102: Unexpected character '→'.
19:74: Unexpected character '×'.
21:40: Unexpected character '≤'.
21:63: Unexpected character '×'.
21:107: Unexpected character '×'.
21:145: Unexpected character '²'.
27:50: Unexpected character '·'.
27:164: Unexpected character '∈'.
27:171: Unexpected character '·'.
29:31: Unexpected character 'Σ'.
29:40: Unexpected character '⌊'.
29:44: Unexpected character '⌋'.
29:177: Unexpected character 'Σ'.
31:50: Unexpected character '·'.
31:113: Char literals must contain exactly one character and close with '''.
35:11: Unexpected character '×'.
35:27: Unexpected character '√'.
37:54: Unexpected character '≤'.
37:57: Unexpected character '×'.
37:68: Unexpected character '²'.
37:73: Unexpected character '×'.
37:76: Unexpected character '¹'.
37:77: Unexpected character '⁰'.
39:76: Unexpected character '×'.
39:107: Unexpected character '₁'.
39:108: Unexpected character '·'.
39:113: Unexpected character '₁'.
39:119: Unexpected character '₂'.
39:120: Unexpected character '·'.
39:125: Unexpected character '₂'.
39:136: Unexpected character '₁'.
39:138: Unexpected character '≤'.
39:141: Unexpected character '₂'.
40:9: Unexpected character '₁'.
40:13: Unexpected character '₁'.
40:18: Unexpected character '₂'.
40:19: Unexpected character '·'.
40:24: Unexpected character '₂'.
40:27: Unexpected character '₁'.
41:12: Unexpected character '₁'.
41:17: Unexpected character '₂'.
41:18: Unexpected character '·'.
41:23: Unexpected character '₂'.
41:26: Unexpected character '₁'.
43:5: Unexpected character '₁'.
43:10: Unexpected character '₂'.
43:14: Unexpected character '₁'.
43:19: Unexpected character '₂'.
43:20: Unexpected character '·'.
43:25: Unexpected character '₂'.
43:28: Unexpected character '₁'.
43:46: Unexpected character '₁'.
43:82: Unexpected character '₁'.
43:87: Unexpected character '₂'.
43:88: Unexpected character '·'.
43:93: Unexpected character '₂'.
43:96: Unexpected character '₁'.
44:5: Unexpected character '₁'.
44:10: Unexpected character '₂'.
44:14: Unexpected character '₁'.
44:19: Unexpected character '₂'.
50:37: Unexpected character '×'.
50:85: Unexpected character '≈'.
50:88: Unexpected character '×'.
Values:
run.Compiled = false |
Failed | IN9 383 OUT1 614 |
| × | LcbEquallyLiveCodeBench / atcoder / easy: Equally (abc385_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Equally case1: expected [Yes] got [No] |
Failed | IN9 316 OUT161 |
| × | LcbIlluminateBuildingsLiveCodeBench / atcoder / medium: Illuminate Buildings (abc385_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: IlluminateBuildings case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 356 OUT227 |
| ✓ | LcbMaximizeXorLiveCodeBench / atcoder / hard: Maximize XOR (abc386_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 372 OUT309 |
| ✓ | LcbPerfectStandingsLiveCodeBench / atcoder / medium: Perfect Standings (abc384_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 434 OUT626 |
| ✓ | LcbRepeatedSequenceLiveCodeBench / atcoder / medium: Repeated Sequence (abc384_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 378 OUT940 |
| ✓ | LcbReplaceExceptCharLiveCodeBench / atcoder / easy: aaaadaa (abc384_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 334 OUT144 |
| × | LcbSantaClaus1LiveCodeBench / atcoder / easy: Santa Claus 1 (abc385_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SantaClaus1 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
27:45: Expected ';' after the variable declaration, found 'cy'.
Values:
run.Compiled = false |
Failed | IN9 599 OUT499 |
| ✓ | LcbSantaClaus2LiveCodeBench / atcoder / hard: Santa Claus 2 (abc385_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 574 OUT556 |
| × | LcbSlime2LiveCodeBench / atcoder / hard: Takahashi is Slime 2 (abc384_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: Slime2 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
35:11: Expected ';' after the variable declaration, found '['.
Values:
run.Compiled = false |
Failed | IN9 508 OUT534 |
| × | LcbSnowflakeTreeLiveCodeBench / atcoder / hard: Snowflake Tree (abc385_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SnowflakeTree case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
74:19: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed | IN9 424 OUT543 |
| – | LcbVisibleBuildingsLiveCodeBench / atcoder / hard: Visible Buildings (abc385_f)0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: float output compared as exact string -- a correct answer within tolerance would be rejected |
Skipped | IN0 OUT0 |
Code Generation - LiveCodeBench v5 AtCoder (set 7)
2 passed9 failed1 broken1 skipped |
16.7% |
IN112 569 OUT100 390 | |
| ✓ | LcbAccumulatingManyTimesLiveCodeBench / atcoder / hard: arc184_e Accumulating Many Times (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 421 OUT1 953 |
| × | LcbAdjacentGcdLiveCodeBench / atcoder / hard: arc185_e Adjacent GCD (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AdjacentGcd case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
7:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 397 OUT4 043 |
| ✓ | LcbCalculatorMinPressesLiveCodeBench / atcoder / easy: abc386_b Calculator (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 338 OUT128 |
| × | LcbDiagonalSeparationLiveCodeBench / atcoder / hard: abc386_d Diagonal Separation (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: DiagonalSeparation case0: expected [Yes] got [No] |
Failed | IN9 417 OUT898 |
| ! | LcbEraseBalls2DLiveCodeBench / atcoder / hard: arc184_d Erase Balls 2D (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Skipped: AI output broken/partial |
Broken | IN9 383 OUT62 144 |
| × | LcbFullHouse2LiveCodeBench / atcoder / easy: abc386_a Full House 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FullHouse2 case2: expected [Yes] got [No] |
Failed | IN9 332 OUT621 |
| × | LcbMountainValleyFoldsLiveCodeBench / atcoder / hard: arc184_c Mountain and Valley Folds (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MountainValleyFolds case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
11:5: Unterminated block comment. Close it with '*/'.
Values:
run.Compiled = false |
Failed | IN9 441 OUT9 252 |
| × | LcbOperate1LiveCodeBench / atcoder / medium: abc386_c Operate 1 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: Operate1 case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
104:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 341 OUT955 |
| × | LcbOperateKLiveCodeBench / atcoder / hard: abc386_f Operate K (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OperateK case2: expected [Yes] got [No] |
Failed | IN9 345 OUT287 |
| × | LcbPlusOneAndLiveCodeBench / atcoder / hard: arc185_b +1 and (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PlusOneAnd case0: expected [Yes
No
Yes] got [Yes
Yes
Yes] |
Failed | IN9 362 OUT298 |
| × | LcbRandomWalkOnTreeLiveCodeBench / atcoder / hard: arc185_d Random Walk on Tree (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RandomWalkOnTree case0: expected [20] got [3] |
Failed | IN9 412 OUT3 575 |
| – | LcbSumOfThreeIntegersLiveCodeBench / atcoder / hard: arc185_c Sum of Three Integers (Pattern B, stdin) NOTE: multiple valid triples may exist; exact-match oracle may reject a different valid answer.0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| × | LcbUndercluedLiveCodeBench / atcoder / hard: arc186_a Underclued (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: Underclued case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
35:52: List constructor expects a List or HashSet to copy, but got 'int'.
Values:
run.Compiled = false |
Failed | IN9 380 OUT16 236 |
Code Generation - LiveCodeBench v5 AtCoder (set 8)
1 passed9 failed2 broken1 skipped |
8.3% |
IN112 704 OUT184 027 | |
| × | LcbAbcSymmetryLiveCodeBench / atcoder / hard: arc188_a ABC Symmetry (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: AbcSymmetry case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
196:42: Type 'string' has no supported method named 'ToList'.
196:42: List constructor expects a List or HashSet to copy, but got '?'.
253:29: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
257:35: Type 'List<int>' has no fields. Only struct values support '.Item1'.
258:31: Type 'List<int>' has no fields. Only struct values support '.Item2'.
263:45: List constructor expects a List or HashSet to copy, but got '?'.
283:25: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
285:38: Type 'List<int>' has no fields. Only struct values support '.Item2'.
Values:
run.Compiled = false |
Failed | IN9 381 OUT4 322 |
| × | LcbBallAndBoxLiveCodeBench / atcoder / hard: arc186_c Ball and Box (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: BallAndBox case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
90:21: Expected a parameter name, found '['.
Values:
run.Compiled = false |
Failed | IN9 418 OUT893 |
| × | LcbBallsAndBoxesLiveCodeBench / atcoder / hard: arc189_c Balls and Boxes (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: BallsAndBoxes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
41:32: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 383 OUT800 |
| – | LcbHonestLiarConfusedLiveCodeBench / atcoder / hard: arc188_c Honest or Liar or Confused (Pattern B, stdin) NOTE: multiple valid confused sets may exist; exact-match oracle may reject a different valid answer.0 AI queries
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| × | LcbMinimizeSumLiveCodeBench / atcoder / hard: arc189_b Minimize Sum (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MinimizeSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 367 OUT1 911 |
| × | LcbMirrorAndOrderLiveCodeBench / atcoder / hard: arc188_d Mirror and Order (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MirrorAndOrder case0: expected [1] got [0] |
Failed | IN9 452 OUT13 284 |
| × | LcbMissingSubsequenceLiveCodeBench / atcoder / hard: arc186_e Missing Subsequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: MissingSubsequence case1 runtime error: Integer overflow.
Values:
run.Success = false |
Failed | IN9 375 OUT10 718 |
| × | LcbPolishManiaLiveCodeBench / atcoder / hard: arc186_d Polish Mania (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PolishMania case0: expected [2] got [0] |
Failed | IN9 396 OUT13 158 |
| ! | LcbReversi2LiveCodeBench / atcoder / medium: arc189_a Reversi 2 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Mediumsource:LiveCodeBench Skipped: AI output broken/partial |
Broken | IN9 409 OUT62 144 |
| ! | LcbSumOfCCLiveCodeBench / atcoder / hard: arc187_b Sum of CC (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Skipped: AI output broken/partial |
Broken | IN9 414 OUT62 144 |
| × | LcbSymmetricPaintingLiveCodeBench / atcoder / hard: arc188_b Symmetric Painting (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SymmetricPainting case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
537:21: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 371 OUT11 332 |
| ✓ | LcbTakahashiIsSlimeLiveCodeBench / atcoder / hard: arc189_d Takahashi is Slime (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 363 OUT2 142 |
| × | LcbTypicalPermutationDescriptorLiveCodeBench / atcoder / hard: arc186_b Typical Permutation Descriptor (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: TypicalPermutationDescriptor case0: expected [3] got [0] |
Failed | IN9 375 OUT1 179 |
Code Generation - LiveCodeBench v5 LeetCode (set 1)
6 passed5 failed |
54.5% |
IN102 488 OUT6 322 | |
| ✓ | LcbCountWinningSequencesLiveCodeBench / leetcode / hard: count-the-number-of-winning-sequences1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 385 OUT1 480 |
| × | LcbDfsPalindromesLiveCodeBench / leetcode / hard: check-if-dfs-strings-are-palindromes1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: dfsPalindromes case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
19:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 389 OUT728 |
| ✓ | LcbFindXSumLiveCodeBench / leetcode / easy: find-x-sum-of-all-k-long-subarrays-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 361 OUT506 |
| ✓ | LcbIsBalancedStringLiveCodeBench / leetcode / easy: check-balanced-string1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 251 OUT120 |
| × | LcbLengthAfterTransformationsLiveCodeBench / leetcode / medium: total-characters-in-string-after-transformations-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: lengthAfter case0: expected 7 got 10 |
Failed | IN9 303 OUT232 |
| ✓ | LcbMaxFactorScoreLiveCodeBench / leetcode / medium: find-the-maximum-factor-score-of-array1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 296 OUT808 |
| × | LcbMinDivisionOperationsLiveCodeBench / leetcode / medium: minimum-division-operations-to-make-array-non-decreasing1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minOperations case0: expected 1 got -1 |
Failed | IN9 285 OUT491 |
| × | LcbMinTimeToReachRoomLiveCodeBench / leetcode / medium: find-minimum-time-to-reach-last-room-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minTimeToReach case2: expected 3 got 2 |
Failed | IN9 355 OUT838 |
| ✓ | LcbNumberOfSubstringsKFreqLiveCodeBench / leetcode / medium: count-substrings-with-k-frequency-characters-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 241 OUT251 |
| ✓ | LcbStringSequenceLiveCodeBench / leetcode / medium: find-the-sequence-of-strings-appeared-on-the-screen1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 304 OUT358 |
| × | LcbSubsequencePairCountGcdLiveCodeBench / leetcode / hard: find-the-number-of-subsequences-with-equal-gcd1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: subseqPair case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
28:28: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
36:20: Cannot assign a value of type 'long' to an element of type 'int'.
44:28: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
50:26: Cannot assign a value of type 'long' to an element of type 'int'.
60:24: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
Values:
run.Compiled = false |
Failed | IN9 318 OUT510 |
Code Generation - LiveCodeBench v5 LeetCode (set 2)
4 passed6 failed |
40.0% |
IN93 412 OUT58 626 | |
| × | LcbCountBalancedPermutationsLiveCodeBench / leetcode / hard: count-number-of-balanced-permutations1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countBalPerm case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
89:14: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 288 OUT3 398 |
| × | LcbCountKReducibleNumbersLiveCodeBench / leetcode / hard: count-k-reducible-numbers-less-than-n1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countKReducible case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
115:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 315 OUT4 914 |
| × | LcbCountValidSelectionsLiveCodeBench / leetcode / easy: make-array-elements-equal-to-zero1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countValid case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:41: Expected ')' to close the argument list, found 'pos'.
Values:
run.Compiled = false |
Failed | IN9 371 OUT455 |
| ✓ | LcbHasIncreasingSubarraysLiveCodeBench / leetcode / easy: adjacent-increasing-subarrays-detection-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 351 OUT275 |
| ✓ | LcbIsZeroArrayLiveCodeBench / leetcode / medium: zero-array-transformation-i1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 343 OUT222 |
| ✓ | LcbMaximizeSumOfWeightsLiveCodeBench / leetcode / hard: maximize-sum-of-weights-after-edge-removals1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 398 OUT1 010 |
| × | LcbMinAdjacentDifferenceLiveCodeBench / leetcode / hard: minimize-the-maximum-adjacent-element-difference1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minDifference case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
425:5: Expected an expression, found 'struct'.
Values:
run.Compiled = false |
Failed | IN9 301 OUT6 408 |
| ✓ | LcbMinArraySumLiveCodeBench / leetcode / medium: minimum-array-sum1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 412 OUT41 165 |
| × | LcbMinimumPositiveSumSubarrayLiveCodeBench / leetcode / easy: minimum-positive-sum-subarray1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minPosSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumSumSubarray' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumSumSubarray.
Values:
run.Compiled = false |
Failed | IN9 345 OUT210 |
| × | LcbSumOfGoodSubsequencesLiveCodeBench / leetcode / hard: sum-of-good-subsequences1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: sumGood case0: expected 14 got 12 |
Failed | IN9 288 OUT569 |
Code Generation - LiveCodeBench v5 LeetCode (set 3)
6 passed5 failed |
54.5% |
IN102 763 OUT8 285 | |
| × | LcbMaxCollectedFruitsLiveCodeBench / leetcode / hard: find-the-maximum-number-of-fruits-collected (id 3648)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxCollectedFruits case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
137:18: Expected ';' after the statement, found ':'.
Values:
run.Compiled = false |
Failed | IN9 488 OUT1 538 |
| ✓ | LcbMaxFrequencyOpsLiveCodeBench / leetcode / medium: maximum-frequency-of-an-element-after-performing-operations-i (id 3622)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 307 OUT222 |
| ✓ | LcbMaxRemovalsSourceLiveCodeBench / leetcode / medium: find-maximum-removals-from-source-string (id 3487)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 405 OUT1 939 |
| ✓ | LcbMaxScoreTouristLiveCodeBench / leetcode / medium: maximum-points-tourist-can-earn (id 3587)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 397 OUT316 |
| × | LcbMinBitwiseArrayLiveCodeBench / leetcode / easy: construct-the-minimum-bitwise-array-i (id 3605)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: minBitwiseArray case0: expected [-1, 1, 4, 3] got [-1,2,4,6] |
Failed | IN9 319 OUT260 |
| × | LcbNumberOfWaysEventLiveCodeBench / leetcode / hard: find-the-number-of-possible-ways-for-an-event (id 3604)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfWays case0: expected 6 got 8 |
Failed | IN9 346 OUT657 |
| × | LcbPossibleStringCountLiveCodeBench / leetcode / easy: find-the-original-typed-string-i (id 3617)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: possibleStringCount case0: expected 5 got 7 |
Failed | IN9 265 OUT167 |
| × | LcbShiftDistanceLiveCodeBench / leetcode / medium: shift-distance-between-two-strings (id 3591)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: shiftDistance case0: expected 2 got 0 |
Failed | IN9 333 OUT826 |
| ✓ | LcbSmallestDivisibleDigitProductLiveCodeBench / leetcode / easy: smallest-divisible-digit-product-i (id 3626)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 255 OUT192 |
| ✓ | LcbStoneRemovalGameLiveCodeBench / leetcode / easy: stone-removal-game (id 3625)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 264 OUT395 |
| ✓ | LcbZeroArrayTransformIIILiveCodeBench / leetcode / medium: zero-array-transformation-iii (id 3647)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 384 OUT1 773 |
Code Generation - LiveCodeBench v5 LeetCode (set 4)
6 passed4 failed |
60.0% |
IN93 169 OUT4 952 | |
| ✓ | LcbButtonLongestTimeLiveCodeBench / leetcode / easy: button-with-longest-push-time (id 3632)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 334 OUT183 |
| × | LcbCountComponentsLcmLiveCodeBench / leetcode / hard: count-connected-components-in-lcm-graph (id 3680)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countComponents case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
83:47: The dictionary key must be 'int', but it has type 'long'.
84:47: The dictionary key must be 'int', but it has type 'long'.
Values:
run.Compiled = false |
Failed | IN9 303 OUT688 |
| × | LcbDigitOpsEqualLiveCodeBench / leetcode / medium: digit-operations-to-make-two-integers-equal (id 3655)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: digitOpsEqual case0: expected 85 got 4 |
Failed | IN9 329 OUT746 |
| ✓ | LcbLargestOutlierLiveCodeBench / leetcode / medium: identify-the-largest-outlier-in-an-array (id 3594)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 328 OUT649 |
| × | LcbMaxRectangleAreaLiveCodeBench / leetcode / medium: maximum-area-rectangle-with-point-constraints-i (id 3681)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxRectangleArea case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:20: Expected an expression, found 'List'.
Values:
run.Compiled = false |
Failed | IN9 342 OUT997 |
| ✓ | LcbMaxSubarraySumDivKLiveCodeBench / leetcode / medium: maximum-subarray-sum-with-length-divisible-by-k (id 3653)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 278 OUT822 |
| ✓ | LcbMinOpsEqualKLiveCodeBench / leetcode / easy: minimum-operations-to-make-array-values-equal-to-k (id 3621)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 317 OUT88 |
| × | LcbMinTimeBreakLocksLiveCodeBench / leetcode / medium: minimum-time-to-break-locks-i (id 3649)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: findMinimumTime case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 312 OUT544 |
| ✓ | LcbSmallestAllSetBitsLiveCodeBench / leetcode / easy: smallest-number-with-all-set-bits (id 3676)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 251 OUT67 |
| ✓ | LcbTransformedArrayLiveCodeBench / leetcode / easy: transformed-array (id 3651)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 375 OUT168 |
Code Generation - LiveCodeBench v5 LeetCode (set 5)
5 passed5 failed |
50.0% |
IN93 065 OUT4 316 | |
| × | LcbBeautifulSplitsLiveCodeBench / leetcode / medium: count beautiful splits in an array (LCB id 3686)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: beautifulSplits case1: expected 0 got 2 |
Failed | IN9 276 OUT299 |
| ✓ | LcbCountGoodArraysLiveCodeBench / leetcode / hard: count arrays with k matching adjacent elements (LCB id 3682)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 313 OUT1 044 |
| ✓ | LcbCountLen3SubarraysLiveCodeBench / leetcode / easy: count length-3 subarrays with a condition (LCB id 3685)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 245 OUT114 |
| × | LcbCountSpecialSubsequencesLiveCodeBench / leetcode / medium: count special subsequences (LCB id 3699)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: numberOfSubsequences case1: expected 3 got 9 |
Failed | IN9 327 OUT297 |
| × | LcbLargestBoxStringLiveCodeBench / leetcode / medium: lexicographically largest string from the box I (LCB id 3683)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: answerString case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 277 OUT476 |
| ✓ | LcbMakeStringGoodLiveCodeBench / leetcode / hard: minimum operations to make character frequencies equal (LCB id 3638)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 281 OUT474 |
| × | LcbMaxCurrencyAmountLiveCodeBench / leetcode / medium: maximize amount after two days of conversions (LCB id 3613)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxAmount case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:74: Expected ')' to close the Dictionary constructor. Dictionaries start empty; fill them with 'dict[key] = value;', found 'amountsAfterDay1'.
Values:
run.Compiled = false |
Failed | IN9 457 OUT844 |
| ✓ | LcbMaxDistinctElementsLiveCodeBench / leetcode / medium: maximum number of distinct elements after operations (LCB id 3620)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 284 OUT136 |
| × | LcbMinIdenticalSubstringLiveCodeBench / leetcode / hard: smallest substring with identical characters I (LCB id 3690)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minLength case0: expected 2 got 4 |
Failed | IN9 306 OUT478 |
| ✓ | LcbMinOpsDistinctLiveCodeBench / leetcode / easy: minimum operations to make elements distinct (LCB id 3656)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 299 OUT154 |
Code Generation - LiveCodeBench v5 LeetCode (set 6)
3 passed7 failed |
30.0% |
IN93 360 OUT7 568 | |
| ✓ | LcbCheckValidCutsLiveCodeBench / leetcode / medium: check if grid can be cut into sections (LCB id 3657)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 395 OUT727 |
| × | LcbCountXorPathsLiveCodeBench / leetcode / medium: count paths with the given xor value (LCB id 3659)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countPathsWithXorValue case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
14:29: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
41:37: foreach can only iterate over a List, HashSet or string, but the collection has type 'int'.
56:33: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
71:19: Type 'Dictionary<int, int>' has no supported method named 'Add' with 2 argument(s).
74:9: Cannot assign a value of type 'int' to variable 'prevLayer' of type 'Dictionary<int, int>'.
Values:
run.Compiled = false |
Failed | IN9 380 OUT590 |
| × | LcbLongestDecreasingDiffLiveCodeBench / leetcode / medium: longest subsequence with decreasing adjacent difference (LCB id 3716)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: longestSubsequence case1: expected 4 got 3 |
Failed | IN9 336 OUT268 |
| × | LcbMaxCoinsKBagsLiveCodeBench / leetcode / medium: maximum coins from k consecutive bags (LCB id 3715)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maximumCoins case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
3:18: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed | IN9 325 OUT747 |
| × | LcbMaxIntervalScoreLiveCodeBench / leetcode / hard: maximum score of non-overlapping intervals (LCB id 3562)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maximumWeight case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
83:15: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 389 OUT1 790 |
| × | LcbMaxProductEquivalentLiveCodeBench / leetcode / easy: maximum subarray with equal products (LCB id 3702)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxLength case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'maxLength' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, Gcd, Lcm, MaxLength.
Values:
run.Compiled = false |
Failed | IN9 313 OUT314 |
| × | LcbMaxSubarrayAfterRemovalLiveCodeBench / leetcode / hard: maximize subarray sum after removing all occurrences of one element (LCB id 3688)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxSubarraySum case0: expected 7 got 4 |
Failed | IN9 284 OUT624 |
| × | LcbMiddleModeSubsequencesLiveCodeBench / leetcode / hard: subsequences with a unique middle mode I (LCB id 3700)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: subsequencesWithMiddleMode case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
218:23: Expected ';' after the variable declaration, found 'a'.
Values:
run.Compiled = false |
Failed | IN9 341 OUT1 847 |
| ✓ | LcbMirrorScoreLiveCodeBench / leetcode / medium: find mirror score of a string (LCB id 3634)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 314 OUT245 |
| ✓ | LcbSubstringPatternLiveCodeBench / leetcode / easy: substring matching pattern (LCB id 3684)1 AI query
0 tool callscategory:CodeGenerationcategory:V5@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 283 OUT416 |
Code Generation - LiveCodeBench v6 AtCoder (set 1)
8 passed8 failed |
50.0% |
IN149 321 OUT17 972 | |
| × | LcbComingOfAgeLiveCodeBench / atcoder abc388_d / medium: Coming of Age Celebration1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ComingOfAge case0: expected [2 0 10 5] got [5 1 11 6] |
Failed | IN9 338 OUT229 |
| × | LcbCountArraysLiveCodeBench / atcoder abc387_f / hard: Count Arrays1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: CountArrays case0 runtime error: List index 3 is out of range. The list has 3 element(s).
Values:
run.Success = false |
Failed | IN9 360 OUT776 |
| × | LcbDangerousSugorokuLiveCodeBench / atcoder abc388_f / hard: Dangerous Sugoroku1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: DangerousSugoroku case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
18:13: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 377 OUT420 |
| ✓ | LcbDigitProductLiveCodeBench / atcoder abc389_a / easy: 9x91 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 278 OUT66 |
| ✓ | LcbFactorialInverseLiveCodeBench / atcoder abc389_b / easy: tcaF1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 284 OUT69 |
| ✓ | LcbHeavySnakeLiveCodeBench / atcoder abc388_b / easy: Heavy Snake1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 323 OUT264 |
| × | LcbKagamimochi2LiveCodeBench / atcoder abc388_g / hard: Simultaneous Kagamimochi 21 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 387 OUT681 |
| ✓ | LcbNotXSumLiveCodeBench / atcoder abc387_b / easy: 9x9 Sum1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 339 OUT115 |
| × | LcbOddEvenGraphLiveCodeBench / atcoder abc389_g / hard: Odd Even Graph1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: OddEvenGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
662:49: Expected '>' to close the List type, found '('.
Values:
run.Compiled = false |
Failed | IN9 374 OUT13 781 |
| ✓ | LcbQuestionUpcLiveCodeBench / atcoder abc388_a / easy: ?UPC1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 284 OUT41 |
| ✓ | LcbRatedRangeLiveCodeBench / atcoder abc389_f / hard: Rated Range1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 333 OUT260 |
| × | LcbSimultaneousKagamimochiLiveCodeBench / atcoder abc388_e / hard: Simultaneous Kagamimochi1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SimultaneousKagamimochi case1: expected [0] got [1] |
Failed | IN9 326 OUT148 |
| × | LcbSnakeNumbersLiveCodeBench / atcoder abc387_c / medium: Snake Numbers1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SnakeNumbers case0: expected [6] got [4] |
Failed | IN9 333 OUT649 |
| ✓ | LcbSquareOfSumLiveCodeBench / atcoder abc387_a / easy: Happy New Year 20251 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 265 OUT70 |
| × | LcbSquaresInCircleLiveCodeBench / atcoder abc389_d / medium: Squares in Circle1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SquaresInCircle case0: expected [5] got [7] |
Failed | IN9 364 OUT223 |
| ✓ | LcbVariousKagamimochiLiveCodeBench / atcoder abc388_c / medium: Various Kagamimochi1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 356 OUT180 |
Code Generation - LiveCodeBench v6 AtCoder (set 2)
7 passed8 failed |
46.7% |
IN140 164 OUT24 342 | |
| ✓ | LcbDoubleSum3LiveCodeBench / atcoder abc390_f / hard: Double Sum 31 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 363 OUT4 380 |
| × | LcbGeometricSequenceLiveCodeBench / atcoder abc390_b / easy: Geometric Sequence1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: GeometricSequence case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 294 OUT166 |
| ✓ | LcbGravityLiveCodeBench / atcoder abc391_d / medium: Gravity1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 425 OUT3 954 |
| × | LcbKthLargestLiveCodeBench / atcoder abc391_f / hard: K1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: KthLargest case0 runtime error: Integer overflow.
Values:
run.Success = false |
Failed | IN9 366 OUT565 |
| × | LcbMajorityVoteLiveCodeBench / atcoder abc391_e / hard: Hierarchical Majority Vote1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MajorityVote case0: expected [1] got [0] |
Failed | IN9 352 OUT1 166 |
| × | LcbManyLcsLiveCodeBench / atcoder abc391_g / hard: Many LCS1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ManyLcs case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
131:8: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 343 OUT2 778 |
| ✓ | LcbOneSwapSortLiveCodeBench / atcoder abc390_a / easy: 124351 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 327 OUT336 |
| ✓ | LcbOppositeDirectionLiveCodeBench / atcoder abc391_a / easy: Lucky Direction1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 304 OUT127 |
| ✓ | LcbPaintRectangleLiveCodeBench / atcoder abc390_c / medium: Paint to make a rectangle1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 350 OUT493 |
| × | LcbPermutationConcatLiveCodeBench / atcoder abc390_g / hard: Permutation Concatenation1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PermutationConcat case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
22:22: Assignment target must be a variable or an element like 'items[0]'. Assigning to struct fields is not supported.
Values:
run.Compiled = false |
Failed | IN9 375 OUT5 382 |
| ✓ | LcbSeekGridLiveCodeBench / atcoder abc391_b / easy: Seek Grid1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 342 OUT379 |
| ✓ | LcbShuffledEquationLiveCodeBench / atcoder abc392_a / easy: Shuffled Equation1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 319 OUT123 |
| × | LcbSquarePriceLiveCodeBench / atcoder abc389_e / hard: Square Price1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: SquarePrice case0: expected [3] got [2] |
Failed | IN9 326 OUT624 |
| × | LcbStoneXorLiveCodeBench / atcoder abc390_d / medium: Stone XOR1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: StoneXor case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
5:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 338 OUT1 007 |
| × | LcbVitaminBalanceLiveCodeBench / atcoder abc390_e / hard: Vitamin Balance1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: VitaminBalance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
17:35: Expected '(' after the List type. Lists are created with 'new List<T>()' or 'new List<T>(items)', found '{'.
Values:
run.Compiled = false |
Failed | IN9 340 OUT2 862 |
Code Generation - LiveCodeBench v6 AtCoder (set 3)
2 passed8 failed |
20.0% |
IN93 587 OUT4 470 | |
| × | LcbABCIntervalsLiveCodeBench / atcoder / abc393_b A..B..C (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ABCIntervals case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:21: Type 'List<int>' has no supported method named 'BinarySearchOrIndex' with 1 argument(s).
37:57: Type 'List<int>' has no supported method named 'BinarySearchOrIndex' with 1 argument(s).
Values:
run.Compiled = false |
Failed | IN9 322 OUT287 |
| ✓ | LcbBibStaringLiveCodeBench / atcoder / abc392_c Bib (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 365 OUT281 |
| × | LcbDoublesProbabilityLiveCodeBench / atcoder / abc392_d Doubles (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: DoublesProbability case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
68:20: Type 'double' has no supported method named 'ToString'.
Values:
run.Compiled = false |
Failed | IN9 375 OUT501 |
| × | LcbFineTripletsLiveCodeBench / atcoder / abc392_g Fine Triplets (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: FineTriplets case0 runtime error: Integer overflow.
Values:
run.Success = false |
Failed | IN9 345 OUT330 |
| × | LcbGcdOfSubsetLiveCodeBench / atcoder / abc393_e GCD of Subset (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: GcdOfSubset case2 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 344 OUT631 |
| × | LcbInsertSequenceLiveCodeBench / atcoder / abc392_f Insert (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: InsertSequence case0: expected [4 2 3 1] got [1 0 0 0] |
Failed | IN9 360 OUT258 |
| ✓ | LcbPoisonousOysterLiveCodeBench / atcoder / abc393_a Poisonous Oyster (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 380 OUT143 |
| × | LcbPrefixLisQueryLiveCodeBench / atcoder / abc393_f Prefix LIS Query (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PrefixLisQuery case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
47:14: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 390 OUT1 369 |
| × | LcbSwapToGatherLiveCodeBench / atcoder / abc393_d Swap to Gather (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: SwapToGather case0 runtime error: A loop exceeded the iteration limit of 10000000.
Values:
run.Success = false |
Failed | IN9 343 OUT391 |
| × | LcbWhoIsMissingLiveCodeBench / atcoder / abc392_b Who is Missing? (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 363 OUT279 |
Code Generation - LiveCodeBench v6 AtCoder (set 4)
5 passed5 failed |
50.0% |
IN93 657 OUT4 614 | |
| × | LcbAlkaneSubtreeLiveCodeBench / atcoder / abc394_f Alkane (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AlkaneSubtree case0: expected [8] got [6] |
Failed | IN9 378 OUT698 |
| × | LcbCatConcatLiveCodeBench / atcoder / abc394_b cat (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: CatConcat case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
9:17: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed | IN9 305 OUT116 |
| ✓ | LcbColorfulBracketsLiveCodeBench / atcoder / abc394_d Colorful Bracket Sequence (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 318 OUT176 |
| ✓ | LcbDebugWaAcLiveCodeBench / atcoder / abc394_c Debug (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 289 OUT90 |
| × | LcbDenseBuildingsLiveCodeBench / atcoder / abc394_g Dense Buildings (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 510 OUT1 200 |
| ✓ | LcbFilterTwosLiveCodeBench / atcoder / abc394_a 22222 (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 281 OUT81 |
| ✓ | LcbMakeTargetLiveCodeBench / atcoder / abc395_b Make Target (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 393 OUT347 |
| × | LcbPalindromicShortestPathLiveCodeBench / atcoder / abc394_e Palindromic Shortest Path (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: PalindromicShortestPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
34:29: Expected '{' after the struct name. Structs are created with 'new Name { Field = value }', found '<'.
Values:
run.Compiled = false |
Failed | IN9 444 OUT1 277 |
| ✓ | LcbShortestDuplicateSubarrayLiveCodeBench / atcoder / abc395_c Shortest Duplicate Subarray (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 346 OUT225 |
| × | LcbSmoothOcclusionLiveCodeBench / atcoder / abc395_f Smooth Occlusion (Pattern B, stdin)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: SmoothOcclusion case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 393 OUT404 |
Code Generation - LiveCodeBench v6 AtCoder (set 5)
10 passed5 failed1 skipped |
66.7% |
IN140 125 OUT46 402 | |
| ✓ | LcbBuyBallsLiveCodeBench / atcoder / medium: Buy Balls (abc396_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 361 OUT460 |
| × | LcbCardPileLiveCodeBench / atcoder / easy: Card Pile (abc396_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: CardPile case0 runtime error: List index -1 is out of range. The list has 0 element(s).
Values:
run.Success = false |
Failed | IN9 340 OUT214 |
| – | LcbCubesLiveCodeBench / atcoder / hard: Cubes (abc397_d)0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| ✓ | LcbFlipEdgeLiveCodeBench / atcoder / hard: Flip Edge (abc395_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 354 OUT720 |
| × | LcbFlipRowOrColLiveCodeBench / atcoder / hard: Flip Row or Col (abc396_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FlipRowOrCol case2: expected [13] got [18] |
Failed | IN9 350 OUT362 |
| × | LcbMaximizeDistanceLiveCodeBench / atcoder / hard: Maximize Distance (abc397_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MaximizeDistance case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
1649:13: Expected ';' after the statement, found ','.
Values:
run.Compiled = false |
Failed | IN9 334 OUT33 807 |
| ✓ | LcbMinRestrictedSumLiveCodeBench / atcoder / hard: Min of Restricted Sum (abc396_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 375 OUT781 |
| × | LcbMinXorPathLiveCodeBench / atcoder / medium: Minimum XOR Path (abc396_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: MinXorPath case0 runtime error: List index 2 is out of range. The list has 2 element(s).
Values:
run.Success = false |
Failed | IN9 357 OUT1 320 |
| ✓ | LcbPathDecompositionLiveCodeBench / atcoder / hard: Path Decomposition of a Tree (abc397_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 327 OUT2 494 |
| × | LcbRotatedInversionsLiveCodeBench / atcoder / hard: Rotated Inversions (abc396_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 363 OUT4 929 |
| ✓ | LcbStrictlyIncreasingLiveCodeBench / atcoder / easy: Strictly Increasing? (abc395_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 322 OUT197 |
| ✓ | LcbThermometerLiveCodeBench / atcoder / easy: Thermometer (abc397_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 317 OUT65 |
| ✓ | LcbTicketGateLogLiveCodeBench / atcoder / medium: Ticket Gate Log (abc397_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 320 OUT219 |
| ✓ | LcbTripleFourLiveCodeBench / atcoder / easy: Triple Four (abc396_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 325 OUT151 |
| ✓ | LcbVarietySplitEasyLiveCodeBench / atcoder / medium: Variety Split Easy (abc397_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 333 OUT326 |
| ✓ | LcbVarietySplitHardLiveCodeBench / atcoder / hard: Variety Split Hard (abc397_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 347 OUT357 |
Code Generation - LiveCodeBench v6 AtCoder (set 6)
5 passed10 failed |
33.3% |
IN140 148 OUT9 926 | |
| ✓ | LcbAbc400PartyLiveCodeBench / atcoder / easy: ABC400 Party (abc400_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 310 OUT66 |
| × | LcbAbcbaLiveCodeBench / atcoder / hard: ABCBA (abc398_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Abcba case0: expected [ABCBA] got [ABCCB] |
Failed | IN9 275 OUT352 |
| × | LcbBonfireLiveCodeBench / atcoder / hard: Bonfire (abc398_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Bonfire case0: expected [001010] got [000000] |
Failed | IN9 459 OUT400 |
| ✓ | LcbDoorsInCenterLiveCodeBench / atcoder / easy: Doors in the Center (abc398_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 290 OUT238 |
| × | LcbFullHouse3LiveCodeBench / atcoder / medium: Full House 3 (abc398_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FullHouse3 case2: expected [No] got [Yes] |
Failed | IN9 332 OUT230 |
| × | LcbGoodIntegersLiveCodeBench / atcoder / medium: 2^a b^2 good integers (abc400_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: GoodIntegers case1: expected [24] got [45] |
Failed | IN9 330 OUT548 |
| ✓ | LcbHammingDistanceLiveCodeBench / atcoder / easy: Hamming Distance (abc399_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 314 OUT98 |
| × | LcbMakeItForestLiveCodeBench / atcoder / medium: Make it Forest (abc399_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MakeItForest case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 312 OUT329 |
| × | LcbNotOnlyTreeGameLiveCodeBench / atcoder / hard: Not Only Tree Game (abc398_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: NotOnlyTreeGame case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
13:17: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 373 OUT630 |
| × | LcbRangePowerSumLiveCodeBench / atcoder / hard: Range Power Sum (abc399_f)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: RangePowerSum case0: expected [75] got [14] |
Failed | IN9 347 OUT4 306 |
| ✓ | LcbRankingWithTiesLiveCodeBench / atcoder / easy: Ranking with Ties (abc399_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 341 OUT463 |
| × | LcbReplaceLiveCodeBench / atcoder / hard: Replace (abc399_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: Replace case1: expected [0] got [3] |
Failed | IN9 343 OUT313 |
| × | LcbSwitchSeatsLiveCodeBench / atcoder / medium: Switch Seats (abc399_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 388 OUT581 |
| ✓ | LcbUniquenessLiveCodeBench / atcoder / medium: Uniqueness (abc398_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 316 OUT279 |
| × | LcbWallBreakerLiveCodeBench / atcoder / medium: Takahashi the Wall Breaker (abc400_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: WallBreaker case0: expected [1] got [0] |
Failed | IN9 418 OUT1 093 |
Code Generation - LiveCodeBench v6 AtCoder (set 7)
1 passed12 failed2 skipped |
7.7% |
IN122 795 OUT90 278 | |
| – | LcbAPowNLiveCodeBench / atcoder / hard: A^n (arc191_c) NOTE: special judge - any valid (A,M) pair is accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| × | LcbArcArcLiveCodeBench / atcoder / medium: ARC Arc (arc192_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ArcArc case0: expected [Yes] got [No] |
Failed | IN9 487 OUT373 |
| × | LcbBasicGridUpdatesLiveCodeBench / atcoder / hard: Basic Grid Problem with Updates (arc190_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: BasicGridUpdates case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
169:48: Unknown variable 'prevLeft'.
170:48: Unknown variable 'prevLeft'.
Values:
run.Compiled = false |
Failed | IN9 550 OUT1 965 |
| × | LcbBrokenWheelLiveCodeBench / atcoder / hard: Broken Wheel (arc193_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: BrokenWheel case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
23:49: Operator '+' cannot be applied to 'long' and 'string'. It supports numbers and string/char concatenation.
28:49: Operator '+' cannot be applied to 'long' and 'string'. It supports numbers and string/char concatenation.
57:16: Operator '+' cannot be applied to 'long' and 'string'. It supports numbers and string/char concatenation.
Values:
run.Compiled = false |
Failed | IN9 467 OUT853 |
| × | LcbComplementIntervalGraphLiveCodeBench / atcoder / hard: Complement Interval Graph (arc193_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ComplementIntervalGraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
27:16: Expected ')' to close the parenthesized expression, found ','.
Values:
run.Compiled = false |
Failed | IN9 514 OUT36 146 |
| × | LcbFennecVsSnuke2LiveCodeBench / atcoder / hard: Fennec VS. Snuke 2 (arc192_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: FennecVsSnuke2 case1: expected [Snuke] got [Fennec] |
Failed | IN9 407 OUT419 |
| × | LcbFractionLineLiveCodeBench / atcoder / hard: Fraction Line (arc192_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: FractionLine case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:11: Expected a variable name, found '['.
Values:
run.Compiled = false |
Failed | IN9 450 OUT914 |
| – | LcbInsideOrOutsideLiveCodeBench / atcoder / hard: Inside or Outside (arc190_a) NOTE: special judge - multiple minimum-cost operation sequences are accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| × | LcbMatrixPowSumLiveCodeBench / atcoder / hard: Matrix Pow Sum (arc190_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MatrixPowSum case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
258:28: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 435 OUT6 242 |
| × | LcbMovingPiecesOnGraphLiveCodeBench / atcoder / hard: Moving Pieces on Graph (arc191_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: MovingPiecesOnGraph case0: expected [3] got [0] |
Failed | IN9 444 OUT465 |
| × | LcbPatisserieAbc3LiveCodeBench / atcoder / hard: Patisserie ABC 3 (abc400_g)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: PatisserieAbc3 case0: expected [12] got [9] |
Failed | IN9 464 OUT526 |
| × | LcbReplaceDigitsLiveCodeBench / atcoder / medium: Replace Digits (arc191_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: ReplaceDigits case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
15:29: 'Max' expects numeric arguments, but got 'char' and 'char'.
Values:
run.Compiled = false |
Failed | IN9 371 OUT177 |
| × | LcbRingoFavoriteNumbers3LiveCodeBench / atcoder / hard: Ringo's Favorite Numbers 3 (abc400_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: RingoFavoriteNumbers3 case0 runtime error: The script exceeded the timeout of 30000 ms.
Values:
run.Success = false |
Failed | IN9 413 OUT603 |
| × | LcbSnukeKyotoTripLiveCodeBench / atcoder / hard: Snuke's Kyoto Trip (arc192_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SnukeKyotoTrip case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
2006:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 430 OUT41 381 |
| ✓ | LcbSumOfGeometricSeriesLiveCodeBench / atcoder / easy: Sum of Geometric Series (abc400_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Easysource:LiveCodeBench |
Passed | IN9 363 OUT214 |
Code Generation - LiveCodeBench v6 AtCoder (set 8)
2 passed10 failed2 broken1 skipped |
14.3% |
IN132 244 OUT280 455 | |
| × | LcbAdjacentDeleteLiveCodeBench / atcoder / hard: Adjacent Delete (arc196_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: AdjacentDelete case0: expected [5] got [3] |
Failed | IN9 335 OUT195 |
| × | LcbCostToFlipLiveCodeBench / atcoder / hard: Cost to Flip (arc194_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: CostToFlip case0: expected [16] got [-1] |
Failed | IN9 408 OUT678 |
| – | LcbHamiltonianPiecesLiveCodeBench / atcoder / hard: Hamiltonian Pieces (arc195_c) NOTE: special judge - any valid placement is accepted; exact-match may unfairly fail a correct model output.0 AI queries
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Skipped: skip: non-unique answer -- any valid output is accepted; exact-match oracle would reject a correct-but-different answer |
Skipped | IN0 OUT0 |
| ! | LcbMagnetsLiveCodeBench / atcoder / hard: Magnets (arc193_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Skipped: AI output broken/partial |
Broken | IN9 485 OUT62 144 |
| × | LcbMinimumCostSortLiveCodeBench / atcoder / hard: Minimum Cost Sort (arc194_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: MinimumCostSort case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
29:5: Expected an expression, found 'void'.
Values:
run.Compiled = false |
Failed | IN9 348 OUT4 017 |
| × | LcbOperationsOnStackLiveCodeBench / atcoder / medium: Operations on a Stack (arc194_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: OperationsOnStack case0: expected [8] got [5] |
Failed | IN9 367 OUT174 |
| × | LcbRandomTreeDistanceLiveCodeBench / atcoder / hard: Random Tree Distance (arc195_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: RandomTreeDistance case0 runtime error: List index 6 is out of range. The list has 4 element(s).
Values:
run.Success = false |
Failed | IN9 499 OUT12 280 |
| × | LcbReverseBracketsLiveCodeBench / atcoder / hard: Reverse Brackets (arc194_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<string>(run.ResultJson).Trim() == expected[i].Trim()
Context: ReverseBrackets case0: expected [2] got [1] |
Failed | IN9 410 OUT663 |
| × | LcbRoadwayLiveCodeBench / atcoder / hard: Roadway (arc196_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Success
Context: Roadway case0 runtime error: List index 100002 is out of range. The list has 16 element(s).
Values:
run.Success = false |
Failed | IN9 556 OUT55 085 |
| × | LcbStronglyConnectedLiveCodeBench / atcoder / hard: Strongly Connected (arc196_c)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: StronglyConnected case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
59:23: Expected ')' to close the argument list, found 'dp'.
Values:
run.Compiled = false |
Failed | IN9 431 OUT51 944 |
| ! | LcbSwap0X1YLiveCodeBench / atcoder / hard: Swap 0^X and 1^Y (arc194_e)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Skipped: AI output broken/partial |
Broken | IN9 518 OUT62 144 |
| × | LcbSwapAndEraseLiveCodeBench / atcoder / hard: Swap and Erase (arc195_d)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: SwapAndErase case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
212:28: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
Values:
run.Compiled = false |
Failed | IN9 446 OUT3 132 |
| × | LcbTorusLoopLiveCodeBench / atcoder / hard: Torus Loop (arc196_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench External function 'RunScript' threw InvalidOperationException: ScSharp execution failed unexpectedly. This is a bug in the ScSharp library. |
Failed | IN9 623 OUT27 113 |
| ✓ | LcbTwiceSubsequenceLiveCodeBench / atcoder / medium: Twice Subsequence (arc195_a)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Mediumsource:LiveCodeBench |
Passed | IN9 388 OUT443 |
| ✓ | LcbUniformSumLiveCodeBench / atcoder / hard: Uniform Sum (arc195_b)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@AtCoderlevel:Hardsource:LiveCodeBench |
Passed | IN9 430 OUT443 |
Code Generation - LiveCodeBench v6 LeetCode (set 1)
6 passed5 failed |
54.5% |
IN102 741 OUT7 849 | |
| ✓ | LcbCountEvenDiffPartitionsLiveCodeBench 3704 / leetcode / easy: count partitions with even sum difference (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 305 OUT174 |
| × | LcbCountMentionsPerUserLiveCodeBench 3721 / leetcode / medium: count mentions per user (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: countMentionsPerUser case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
66:29: foreach cannot iterate over a Dictionary directly. Iterate over dict.Keys() or dict.Values() instead.
Values:
run.Compiled = false |
Failed | IN9 425 OUT484 |
| ✓ | LcbCountNonDecreasingSubarraysLiveCodeBench 3674 / leetcode / hard: count non-decreasing subarrays after k operations (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 344 OUT180 |
| × | LcbMaxFrequencyAfterOpLiveCodeBench 3751 / leetcode / medium: maximum frequency after subarray operation (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxFrequencyAfterOp case0: expected 2 got 1 |
Failed | IN9 310 OUT366 |
| × | LcbMinimizeMaxEdgeWeightLiveCodeBench 3720 / leetcode / medium: minimize the maximum edge weight of graph (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minimizeMaxEdgeWeight case0: expected 1 got -1 |
Failed | IN9 368 OUT1 221 |
| ✓ | LcbMinMaxSubarraySumsLiveCodeBench 3725 / leetcode / hard: max and min sums of at most size k subarrays (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 254 OUT163 |
| × | LcbMinMaxSubsequenceSumsLiveCodeBench 3714 / leetcode / medium: max and min sums of at most size k subsequences (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minMaxSubsequenceSums case0: expected 24 got 20 |
Failed | IN9 292 OUT749 |
| × | LcbPaintHouseIVLiveCodeBench 3737 / leetcode / medium: paint house IV (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: paintHouseIV case0: expected 9 got 13 |
Failed | IN9 395 OUT2 269 |
| ✓ | LcbRobotMaxCoinsLiveCodeBench 3677 / leetcode / medium: maximum amount of money robot can earn (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 394 OUT1 847 |
| ✓ | LcbVariableLengthSubarraySumLiveCodeBench 3731 / leetcode / easy: sum of variable length subarrays (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 273 OUT138 |
| ✓ | LcbZigzagGridSkipLiveCodeBench 3708 / leetcode / easy: zigzag grid traversal with skip (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 381 OUT258 |
Code Generation - LiveCodeBench v6 LeetCode (set 2)
6 passed4 failed |
60.0% |
IN93 225 OUT3 610 | |
| ✓ | LcbAssignElementsToGroupsLiveCodeBench 3760 / leetcode / medium: assign elements to groups with constraints (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 386 OUT156 |
| ✓ | LcbCountSubstringsDivLastDigitLiveCodeBench 3696 / leetcode / hard: count substrings divisible by last digit (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 255 OUT134 |
| × | LcbEatPizzasMaxWeightLiveCodeBench 3779 / leetcode / medium: eat pizzas (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: eatPizzasMaxWeight case0: expected 14 got 11 |
Failed | IN9 357 OUT122 |
| ✓ | LcbMaxEvenOddFreqDiffLiveCodeBench 3753 / leetcode / easy: maximum difference between even and odd frequency I (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 293 OUT236 |
| × | LcbMaximizeMinGameScoreLiveCodeBench 3762 / leetcode / hard: maximize the minimum game score (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maximizeMinGameScore case0: expected 4 got 0 |
Failed | IN9 363 OUT395 |
| × | LcbMaxManhattanAfterKLiveCodeBench 3754 / leetcode / medium: maximum manhattan distance after k changes (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxManhattanAfterK case1: expected 6 got 5 |
Failed | IN9 363 OUT726 |
| × | LcbMinIncrementsTargetMultiplesLiveCodeBench 3697 / leetcode / hard: minimum increments for target multiples in an array (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minIncrementsTargetMultiples case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
8:19: Expected a type name after 'new', found 'int'.
Values:
run.Compiled = false |
Failed | IN9 289 OUT604 |
| ✓ | LcbSelectKDisjointSpecialLiveCodeBench 3771 / leetcode / medium: select k disjoint special substrings (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 308 OUT449 |
| ✓ | LcbSortMatrixDiagonalsLiveCodeBench 3748 / leetcode / medium: sort matrix by diagonals (Pattern A, single-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 314 OUT536 |
| ✓ | LcbSpecialSubstringLengthKLiveCodeBench 3709 / leetcode / easy: find special substring of length k (Pattern A, multi-arg)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 297 OUT252 |
Code Generation - LiveCodeBench v6 LeetCode (set 3)
4 passed6 failed1 broken |
36.4% |
IN102 835 OUT17 335 | |
| ✓ | LcbChooseKMaxSumLiveCodeBench / leetcode / medium / id 3759: for each i, sum of k largest nums2[j] where nums1[j] < nums1[i]1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 389 OUT230 |
| ✓ | LcbFruitsIntoBasketsLiveCodeBench / leetcode / medium / id 3791: count fruit types that remain unplaced into baskets1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 363 OUT212 |
| × | LcbLargestAlmostMissingLiveCodeBench / leetcode / easy / id 3705: largest integer appearing in exactly one size-k subarray1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: largestmissing case1: expected 3 got 2 |
Failed | IN9 312 OUT392 |
| × | LcbLexSmallestGeneratedLiveCodeBench / leetcode / hard / id 3770: lexicographically smallest string generated from str1 and str21 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: gencase0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
18:38: Cannot assign a value of type 'int' to variable 'c' of type 'char'.
Values:
run.Compiled = false |
Failed | IN9 368 OUT1 180 |
| × | LcbLongestPalinSubseqKOpsLiveCodeBench / leetcode / medium / id 3786: longest palindromic subsequence after at most k letter shifts1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: palinsubseq case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
6:13: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 334 OUT439 |
| ! | LcbLongestVDiagonalLiveCodeBench / leetcode / hard / id 3733: length of longest V-shaped diagonal segment1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Skipped: AI output broken/partial |
Broken | IN9 411 OUT12 176 |
| ✓ | LcbMaxDistanceSquareLiveCodeBench / leetcode / hard / id 3781: maximize the minimum Manhattan distance among k boundary points1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 406 OUT668 |
| × | LcbMaxSubarraysConflictLiveCodeBench / leetcode / hard / id 3789: max non-empty subarrays after removing one conflicting pair1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxsub case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
116:31: Unknown variable 'lossWithoutLoss'.
Values:
run.Compiled = false |
Failed | IN9 350 OUT932 |
| × | LcbMaxSumKSubarraysLiveCodeBench / leetcode / medium / id 3722: max sum of k non-overlapping subarrays each length >= m1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxsumksub case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
16:16: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 287 OUT590 |
| × | LcbMaxSumWithLimitsLiveCodeBench / leetcode / medium / id 3764: maximum sum of at most k elements with per-row limits1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: maxsumlimits case0: expected 7 got 6 |
Failed | IN9 322 OUT369 |
| ✓ | LcbSameDigitsAfterOpsLiveCodeBench / leetcode / easy / id 3768: check if final two digits are equal after repeated pairwise sums1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 293 OUT147 |
Code Generation - LiveCodeBench v6 LeetCode (set 4)
5 passed5 failed |
50.0% |
IN93 322 OUT4 128 | |
| × | LcbClosestEqualQueriesLiveCodeBench / leetcode / medium / id 3750: circular min distance to an equal-valued element per query1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: closesteq case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'solveQueries' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, SolveQueries.
Values:
run.Compiled = false |
Failed | IN9 335 OUT328 |
| ✓ | LcbCountBeautifulNumbersLiveCodeBench / leetcode / hard / id 3801: count numbers in [l,r] whose digit-product is divisible by digit-sum1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 287 OUT144 |
| × | LcbLongestPalinConcatLiveCodeBench / leetcode / medium / id 3793: longest palindrome from a substring of s concatenated with one of t1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: palinconcat case3: expected 5 got 2 |
Failed | IN9 297 OUT1 154 |
| ✓ | LcbMaxContainersShipLiveCodeBench / leetcode / easy / id 3817: max containers of weight w on an n x n deck within maxWeight1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 303 OUT59 |
| ✓ | LcbMaxUniqueSubarraySumLiveCodeBench / leetcode / easy / id 3788: max sum of a subarray with all-unique elements after deletions1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 306 OUT269 |
| × | LcbMinCostReachPositionsLiveCodeBench / leetcode / easy / id 3832: min total cost to reach each position via prefix-min swaps1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<List<int>>(run.ResultJson) == FromJson<List<int>>(expected[i])
Context: mincosts case0: expected [5, 3, 3, 1, 1, 1] got [1,1,1,1,2,2] |
Failed | IN9 357 OUT103 |
| ✓ | LcbMinOpsElementsZeroLiveCodeBench / leetcode / hard / id 3744: sum over queries of min pair-divide operations to zero a range1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 310 OUT795 |
| × | LcbMinTimeBrewPotionsLiveCodeBench / leetcode / medium / id 3794: min synchronized time for n wizards to brew m potions in order1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: mintime case0: expected 110 got 95 |
Failed | IN9 379 OUT297 |
| × | LcbPropertiesGraphLiveCodeBench / leetcode / medium / id 3809: connected components where edges need >= k shared distinct values1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: propgraph case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
12:13: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 366 OUT514 |
| ✓ | LcbZeroArrayTransformIVLiveCodeBench / leetcode / medium / id 3795: min k queries to reduce nums to a zero array (subset decrements)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 382 OUT465 |
Code Generation - LiveCodeBench v6 LeetCode (set 5)
5 passed6 failed |
45.5% |
IN102 546 OUT8 727 | |
| ✓ | LcbFindValidDigitPairLiveCodeBench / leetcode / easy: find-valid-pair-of-adjacent-digits-in-string (id 3736)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 293 OUT281 |
| × | LcbLongestSpecialPathLiveCodeBench / leetcode / hard: longest-special-path (id 3687)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: longestSpecialPath case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
63:5: Expected an expression, found 'void'.
Values:
run.Compiled = false |
Failed | IN9 413 OUT2 137 |
| × | LcbManhattanArrangementsLiveCodeBench / leetcode / hard: manhattan-distances-of-all-arrangements-of-pieces (id 3739)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: manhattanArrange case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
90:12: Unknown variable 'sumDistices'.
Values:
run.Compiled = false |
Failed | IN9 309 OUT807 |
| ✓ | LcbMaxAdjacentCircularLiveCodeBench / leetcode / easy: maximum-difference-between-adjacent-elements-in-a-circular-array (id 3747)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 236 OUT132 |
| × | LcbMaxFreeTimeLiveCodeBench / leetcode / medium: reschedule-meetings-for-maximum-free-time-i (id 3743)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: maxFreeTime case0: expected 2 got 3 |
Failed | IN9 415 OUT469 |
| ✓ | LcbMaxProductAlternatingSumLiveCodeBench / leetcode / hard: maximum-product-of-subsequences-with-an-alternating-sum-equal-to-k (id 3777)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 341 OUT440 |
| × | LcbMinCostGoodCaptionLiveCodeBench / leetcode / hard: minimum-cost-good-caption (id 3701)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minCostGoodCaption case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
148:49: Expected ')' to close the argument list, found '!'.
Values:
run.Compiled = false |
Failed | IN9 344 OUT2 103 |
| × | LcbMinOpsKSubarraysEqualLiveCodeBench / leetcode / hard: minimum-operations-to-make-elements-within-k-subarrays-equal (id 3717)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: FromJson<long>(run.ResultJson) == FromJson<long>(expected[i])
Context: minOpsKSub case0: expected 8 got 9 |
Failed | IN9 317 OUT1 158 |
| × | LcbMinPairRemovalSortLiveCodeBench / leetcode / easy: minimum-pair-removal-to-sort-array-i (id 3773)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: minPairRemoval case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
Entry point 'minimumPairRemoval' was not found. Available functions: AskAi, RunScript, SkipTest, __CreateInMemoryDatabase, __DbExecute, __DbExecuteScript, __DbInsert, __DbQueryFirstJson, __DbQueryJson, __DbScalarJson, MinimumPairRemoval.
Values:
run.Compiled = false |
Failed | IN9 264 OUT284 |
| ✓ | LcbSeparateSquaresLiveCodeBench / leetcode / medium: separate-squares-i (id 3763)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench |
Passed | IN9 328 OUT727 |
| ✓ | LcbSumOfGoodNumbersLiveCodeBench / leetcode / easy: sum-of-good-numbers (id 3723)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 286 OUT189 |
Code Generation - LiveCodeBench v6 LeetCode (set 6)
5 passed5 failed |
50.0% |
IN93 212 OUT22 301 | |
| × | LcbCountCopyArraysLiveCodeBench / leetcode / medium: find-the-number-of-copy-arrays (id 3785)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: countCopyArrays case1: expected 4 got 10 |
Failed | IN9 387 OUT305 |
| ✓ | LcbLongestCommonPrefixRemovalLiveCodeBench / leetcode / hard: longest-common-prefix-of-k-strings-after-removal (id 3784)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 329 OUT737 |
| × | LcbMaxActiveSectionsTradeLiveCodeBench / leetcode / medium: maximize-active-section-with-trade-i (id 3805)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: maxActiveTrade case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
461:17: Variable 'gain' is already declared in this scope or an enclosing one.
1:5: Not all code paths in function 'maxActiveSectionsAfterTrade' return a value.
Values:
run.Compiled = false |
Failed | IN9 373 OUT5 307 |
| ✓ | LcbMinCostDivideArrayLiveCodeBench / leetcode / hard: minimum-cost-to-divide-array-into-subarrays (id 3765)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 377 OUT13 233 |
| × | LcbMinCostRemoveElementsLiveCodeBench / leetcode / medium: find-minimum-cost-to-remove-array-elements (id 3776)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Mediumsource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: minCostRemove case0: expected 12 got 10 |
Failed | IN9 305 OUT392 |
| × | LcbPermutationsIVLiveCodeBench / leetcode / hard: permutations-iv (id 3783)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench Assertion expression evaluated to false: run.Compiled
Context: permutationsIV case0 did not compile: SC# compilation failed inside the script runner.
Compiler diagnostics:
37:14: Expected ';' after the variable declaration, found '('.
Values:
run.Compiled = false |
Failed | IN9 294 OUT1 065 |
| ✓ | LcbReverseDegreeLiveCodeBench / leetcode / easy: reverse-degree-of-a-string (id 3811)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 261 OUT96 |
| ✓ | LcbShortestMatchingSubstringLiveCodeBench / leetcode / hard: shortest-matching-substring (id 3692)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Hardsource:LiveCodeBench |
Passed | IN9 322 OUT834 |
| ✓ | LcbTransformArrayParityLiveCodeBench / leetcode / easy: transform-array-by-parity (id 3778)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench |
Passed | IN9 279 OUT167 |
| × | LcbUnique3DigitEvenLiveCodeBench / leetcode / easy: unique-3-digit-even-numbers (id 3799)1 AI query
0 tool callscategory:CodeGenerationcategory:V6@LeetCodelevel:Easysource:LiveCodeBench Assertion expression evaluated to false: FromJson<int>(run.ResultJson) == FromJson<int>(expected[i])
Context: unique3Digit case1: expected 2 got 4 |
Failed | IN9 285 OUT165 |