AI & ML
CodeLift: Your Test Suite Is Secretly a Dense SWE Curriculum
arXiv cs.SE0 views
arXiv:2610.17224v2 Announce Type: replace-cross Abstract: The pass-all-test binary reward has become a standard recipe for training software engineering agents, but its sparsity eliminates difference of partially correct patches. The test pass-rate is a natural alternative. However, we find that it is not a reliable indicator of patch quality. A functionality being tested can be covered by either one challenging test or one hundred trivial tests, where the majority might be already satisfied pass-to-pass tests. We introduce CodeLift, a reinforcement learning method that dynamically calibrates pass-rate rewards, turning the test suite into an adaptive curriculum. At each iteration, CodeLift groups tests that kill the same patches and patches that pass the same tests, then gives more weight to test groups that fewer patch groups pass. This reduces the impact of majority simple tests and focuses more on what is still challenging. Within each rollout, CodeLift applies the same scoring rule to repository snapshots, assigning credit to useful actions even when later ones introduce regressions. Experiments on SWE benchmarks show that CodeLift consistently improves pass@1/2/4/8 over binary and pass-rate baselines. Analyses show that CodeLift provides dense, stable learning signals and unlocks the inherent curriculum in the test suite.
Source:https://arxiv.org/abs/2610.17224PreviousPR4Code: A Pull Requests Dataset for AI Code GenerationNextExplain, Then Forget: Recommendation Unlearning under Asymmetric Access Constraint
Related posts
After blocking Muse, Amazon now lets users text Alexa to help them shop
0TechCrunch
AI for scientific discovery: How researchers use Claude
0Hacker News
FAA Begins Retiring Decades Old Floppy Disks from Air Traffic Control
0Hacker News
WARN Act and Child Support — What Happens to Support Payments After a Layoff Notice
0dev.to Data
1,084 Mercado Libre Cards Told Me How Many Were Sold. Between Them They Used Ten Numbers
0dev.to Data
Three traps when an AI assistant reads insider trades from Form 4s
0dev.to Data