AI & ML
Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming
arXiv cs.CR0 views
arXiv:2611.17395v1 Announce Type: new Abstract: Recent advances in weight-space programming (WSP) have demonstrated that large language models (LLMs) can directly inspect and modify neural network weights without conventional gradient-based training. While this capability opens new possibilities for model improvement, it also raises an important security question: Can LLMs exploit weight-space programming to inject backdoors into neural networks? To answer this question, we present an empirical security study of LLM-driven weight-space programming for backdoor injection. Specifically, we investigate whether general-purpose LLMs can directly modify the weights of target models according to natural-language instructions, without accessing task-specific data, performing forward passes, or computing gradients. We evaluate five LLMs as weight editors across three target architectures: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News. Our experiments reveal three key findings. First, LLM-driven weight editing can successfully inject backdoors across different architectures while largely preserving clean performance. Second, successful attacks can require remarkably few parameter modifications, including only six parameters for the MLP and 0.02% of the weights for a fine-tuned Qwen model. Third, editing capabilities vary substantially across LLMs and target architectures: although all five LLMs successfully backdoor the MLP, only GPT-6 Astra succeeds across all three targets. These findings highlight an emerging security risk of weight-space programming and underscore the importance of safeguarding LLM-driven weight editing.
Source:https://arxiv.org/abs/2611.17395PreviousEvolutionary-based Transfer Learning for Physical Attack on End-to-End Autoencoder CommunicationsNextI2C-GRPO: Insight Importance Checklist for Reinforcement Learning in a Multi-Agent DDoS Alert Triage Workflow
Related posts
Coordinate Frame Recovery, Per Class NMS and Null Return Modelling for Sim2Real V2X LiDAR Detection
0arXiv cs.RO
Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection
0arXiv cs.RO
D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
0arXiv cs.RO
Beyond Trajectory Similarity: Evaluating Surgical Planning by Protocol Adherence
0arXiv cs.RO
HEIR: Harness Egocentric Intent for Human-Robot Interactions
0arXiv cs.RO
EmbodiExp: Compositional Experience Transfer for Embodied Agents
0arXiv cs.RO