AI & ML
Analyzing Code Provenance in LLMs for Programming Education
arXiv cs.SE0 views
arXiv:2611.09177v1 Announce Type: cross Abstract: Large language models (LLMs) play an increasing role in programming education, challenging traditional assessment methods as they can generate fully functional code instantly. This disrupts the reliability of student submissions as indicators of learning and effort. To address this issue, code provenance is essential for tracing the origins of code and distinguishing student-authored, AI-generated, and externally sourced components. We extend the Min-K% probability approach by incorporating synthetic non-member code generation and apply it to Python code for membership inference and provenance tracking. Evaluations cover free LLMs such as GPT-2, DistilGPT-2, and DialoGPT, as well as state-of-the-art commercial models including GPT-5, Gemini 2.5, Mistral, DeepSeek, and LLaMA 3.3. Before evaluating our approach on higher-education student assignments, we validate its feasibility using Stack Overflow data. The results show that free models enable more sensitive code memorization detection than commercial models. By systematically comparing these LLMs, we assess trade-offs between detection accuracy and privacy protection, taking an initial step toward effective assessment of AI-assisted programming education through combined provenance detection and process visibility.
Source:https://arxiv.org/abs/2611.09177PreviousPdb or Not to Be: Comparing Debugging Interfaces With Pass$_{X}$@$k$NextAutoRefine: Auto-Calibrating Code Judges from Workflow Outcomes
Related posts
What free AI training in Nepal can teach you before you pay for a class
0dev.to

Agents draft. People decide.
0dev.to UX

โค้ดเพิ่ม 400% แต่คุณไม่ได้เร็วขึ้น 400%: GitHub ชี้คอขวดใหม่อยู่ที่การรีวิว
0dev.to GitHub

Drago: Your GitHub Dragon
0dev.to GitHub
Case Study: Automating Tableau to Power BI Migration with Up to 90% Accuracy.
0dev.to Data
Vom Bluttest ins Molekül: Warum die Alamar/Nightingale-Studie räumliches Denken braucht...
0dev.to Data