AI & ML
Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection
arXiv cs.CR0 views
arXiv:2610.00685v2 Announce Type: replace-cross Abstract: With the rapid adoption of large language models (LLMs) and parameter-efficient fine-tuning (PEFT) methods, the risk of backdoor attacks has become more severe. Existing backdoor purification methods typically rely on at least one of the strong assumptions, such as prior knowledge of triggers, access to clean references, or aggressive retraining, and they often lack comprehensive evaluations. These constraints substantially limit their practical applicability. To overcome these challenges, our work proposes purifying LoRA-tuned LLMs without these assumptions and even without post-hoc retraining of the suspect parameters. Our objective is to significantly reduce the attack success rates (ASR) while preserving both (i) the base model's general capabilities and (ii) the new downstream skills learned through the adapter. Through a series of examinations, we progressively scale our approach from a single layer in a text classification setting to a full-parameter LLM in the generative task. Through careful data curation and feature approximation, we extract high-fidelity backdoor directions and, for each layer or head, construct orthogonal null spaces in both the input and output channels, onto which the LoRA updates are projected. Empirically, our null-space projection method reduces the ASR from nearly 100% to less than 10%, while preserving the base model's benign performance and the adapter's learned abilities during downstream task adaptation.
Source:https://arxiv.org/abs/2610.00685PreviousBackdoor Containment via Expert Quarantine and Shutdown in LLMsNextInformation Loss Is All You Need: $TC^0$ Pseudorandomness from Worst-Case Lattices
Related posts
Lakewood Shooting: One Police Officer Killed, Three Wounded in Colorado
0dev.to News
OpenAI: Images 2.6, GIF e Workspace
0dev.to News
How to Plan a Healthtech Product Video Without Sounding Like a Brochure
0dev.to Design
Building a Stress Relief Toy Brand From Wholesale Sourcing to Custom Molds
0dev.to Design
Người trẻ lễ Phật trên điện thoại: trào lưu đi chùa online lan nhanh
0dev.to Discuss

My AI agent says "all fixed." 3 times out of 4, it never measured the before.
0dev.to Discuss