AI & ML
Characterizing Robustness in Semantic Code Clone Detection
arXiv cs.SE0 views
arXiv:2606.25272v2 Announce Type: replace Abstract: Semantic code clone detection seeks to recognize shared functionality across different implementations, making robustness to behavior-preserving changes central to its effectiveness. Prior studies have exposed failures under such changes, motivating a systematic examination of how sensitivity varies with the rewrites used for evaluation. We present Systematic Code Transformation (SCT), a framework organizing 51 operators by code level, target, and edit category, and evaluate 14 token, tree, graph, and prompt-based detectors on BigCloneBench, Google Code Jam, and OJClone. By comparing transformation groups, repeated application, and alternative constructions, we connect overall performance changes with specific rewriting conditions. Of 25 token, tree, and graph detector and dataset combinations starting above 90% F1, 21 lose at least 10 percentage points under the composition of all operators. These losses vary with transformation type and application intensity, but are not determined by edit counts or code growth alone. Even at matched locations and edit counts, alternative constructions can expose different errors despite similar aggregate losses. Further analysis distinguishes reduced score separability from changed fixed-threshold decisions with retained ranking ability, revealing different scope for threshold recovery. Overall, these findings show why benchmark effectiveness needs to extend to stable discrimination across equivalent implementations and how systematic transformation testing can assess that capability.
Source:https://arxiv.org/abs/2606.25272PreviousAI Slop and the Software CommonsNextLarge Language Models for Multi-Lingual Equivalent Mutant Detection: An Extended Empirical Study
Related posts

Galaxy SmartTag 3 hits Samsung online store as single tracker or four-pack
09to5Google
After blocking Muse, Amazon now lets users text Alexa to help them shop
0TechCrunch
AI for scientific discovery: How researchers use Claude
0Hacker News
FAA Begins Retiring Decades Old Floppy Disks from Air Traffic Control
0Hacker News
WARN Act and Child Support — What Happens to Support Payments After a Layoff Notice
0dev.to Data
1,084 Mercado Libre Cards Told Me How Many Were Sold. Between Them They Used Ten Numbers
0dev.to Data