AI & ML
Analyzing Code Provenance in LLMs for Programming Education
arXiv cs.SE0 views
arXiv:2611.09177v1 Announce Type: cross Abstract: Large language models (LLMs) play an increasing role in programming education, challenging traditional assessment methods as they can generate fully functional code instantly. This disrupts the reliability of student submissions as indicators of learning and effort. To address this issue, code provenance is essential for tracing the origins of code and distinguishing student-authored, AI-generated, and externally sourced components. We extend the Min-K% probability approach by incorporating synthetic non-member code generation and apply it to Python code for membership inference and provenance tracking. Evaluations cover free LLMs such as GPT-2, DistilGPT-2, and DialoGPT, as well as state-of-the-art commercial models including GPT-5, Gemini 2.5, Mistral, DeepSeek, and LLaMA 3.3. Before evaluating our approach on higher-education student assignments, we validate its feasibility using Stack Overflow data. The results show that free models enable more sensitive code memorization detection than commercial models. By systematically comparing these LLMs, we assess trade-offs between detection accuracy and privacy protection, taking an initial step toward effective assessment of AI-assisted programming education through combined provenance detection and process visibility.
Source:https://arxiv.org/abs/2611.09177PreviousPdb or Not to Be: Comparing Debugging Interfaces With Pass$_{X}$@$k$NextAutoRefine: Auto-Calibrating Code Judges from Workflow Outcomes
Related posts
Startup Founder Interview: How Novera Is Turning Enterprise Infrastructure Into an Autonomous System
0HackerNoon
Startup Founder Interview: The Startup Using AI and Data to Improve Adoption and Foster Care
0HackerNoon

TurnWise Outperforms Streaming Video RL Baselines Across Three Benchmarks
0HackerNoon
TokenPulse: Building the Leadership Team Behind Cost-Efficient AI Inference
0HackerNoon
From Raw Sensor Signals to Regulatory-Grade Data: How Air Quality Monitoring Networks Are Engineered
0HackerNoon

Researchers Decrypt Hidden Chain-of-Thought From OpenAI, Anthropic and Google APIs
0HackerNoon