AI & ML
CodeLift: Your Test Suite Is Secretly a Dense SWE Curriculum
arXiv cs.SE0 views
arXiv:2610.17224v2 Announce Type: replace-cross Abstract: The pass-all-test binary reward has become a standard recipe for training software engineering agents, but its sparsity eliminates difference of partially correct patches. The test pass-rate is a natural alternative. However, we find that it is not a reliable indicator of patch quality. A functionality being tested can be covered by either one challenging test or one hundred trivial tests, where the majority might be already satisfied pass-to-pass tests. We introduce CodeLift, a reinforcement learning method that dynamically calibrates pass-rate rewards, turning the test suite into an adaptive curriculum. At each iteration, CodeLift groups tests that kill the same patches and patches that pass the same tests, then gives more weight to test groups that fewer patch groups pass. This reduces the impact of majority simple tests and focuses more on what is still challenging. Within each rollout, CodeLift applies the same scoring rule to repository snapshots, assigning credit to useful actions even when later ones introduce regressions. Experiments on SWE benchmarks show that CodeLift consistently improves pass@1/2/4/8 over binary and pass-rate baselines. Analyses show that CodeLift provides dense, stable learning signals and unlocks the inherent curriculum in the test suite.
Source:https://arxiv.org/abs/2610.17224PreviousPR4Code: A Pull Requests Dataset for AI Code GenerationNextExplain, Then Forget: Recommendation Unlearning under Asymmetric Access Constraint
Related posts

iPad Mini vs. iPad Air Buyer's Guide: Now Just $50 Apart
0MacRumors
Ask HN: Does Anthropic have live people doing model guardrail checks?
0Hacker News Ask
Show HN: Teach a speech model (en) a name by saying it once, on your Mac
0Hacker News Show
Show HN: I made a retro game remix in under 5 minutes, 1 prompt
0Hacker News Show
Show HN: Agentshield Zero-leak kernel sandbox and secret defense for AI coding
0Hacker News Show
Show HN: Cerea – yet another self-hosted LLM chat front end
0Hacker News Show