AI & ML
Cross-Benchmark Transfer from RL on Agentic Coding Tasks
arXiv cs.SE0 views
arXiv:2610.00890v2 Announce Type: replace-cross Abstract: Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was supposed to stay intact, or validate against an unchecked assumption. We ask whether reinforcement learning (RL) on expert-built agentic coding tasks closes this gap, and whether what the agent learns transfers beyond the training distribution. We post-train Kimi K2.7 Code, a 1T-parameter (32B active) open-weight mixture-of-experts model, with RL alone on 1,700 tasks: 1,000 repository tasks graded by hidden fail-to-pass tests and by pass-to-pass tests of existing behavior, and 700 terminal tasks graded by expert-written hidden verifiers. The reward is the fraction of target checks passed and drops to zero if any pass-to-pass test fails. One epoch of GSPO on a rank-32 LoRA adapter improves pass@1 on each of the six external benchmarks we evaluated, across three agent harnesses: SWE-Bench Pro (60.1 to 64.8), DeepSWE (31.0 to 43.4), Terminal-Bench 2.1 (67.4 to 82.0), Terminal-Bench 3 (1.4 to 12.1), Terminal-Bench 4 (0.0 to 7.6), and SWE-Marathon (5.0 to 25.0). Pooled over the five independent task sets (Terminal-Bench 4 revises Terminal-Bench 3), the improvement is significant (p < 0.001), and it remains significant on the three sets released after the training data was collected (p = 0.004); the model also improves under both harnesses never used in training. Median trajectories on DeepSWE and Terminal-Bench 3 are 24-35% shorter in agent steps. The base model's failed DeepSWE runs are mostly near-misses, and on the tasks the trained model newly solves, paired trajectories show it avoiding each of the four failure modes above.
Source:https://arxiv.org/abs/2610.00890PreviousProgrammable Distributions of Semantics-Preserving Variants for Evaluating Code AgentsNextPRISM: Preference-based Reward and Uncertainty-Integrated Synthesis Minimization for RTL PPA Optimization
Related posts
Software jobs are back, but not for juniors. What I'd build if I were starting in 2026.
0dev.to Discuss
How to Calculate Real Throughput in an Automated Filling Line
0dev.to Productivity
WeightBench Tests Language Models as Weight-Space Programmers
0dev.to Machine Learning
Data Cleaning for messy dates dd-mm-yyyy
0dev.to Machine Learning

How Much GPU Memory Do I Need?
0dev.to Machine Learning
The Hugging Face Ecosystem in 2026: Transformers, Datasets, Trainer, LoRA/PEFT, Gradio & Spaces (Real Code)
0dev.to Machine Learning