AI & ML
Condensing Law of Security Information
arXiv cs.CR0 views
arXiv:2611.17918v1 Announce Type: new Abstract: Scaling laws indicate that the generalization error of a large language model (LLM) decreases monotonically with model size once data is sufficient, and the corollary that larger models outperform smaller ones is now widely accepted. We argue that this corollary fails in security: decisions there hinge on anomalous inputs, which are long-tailed (e.g., zero-day attacks) and therefore poorly served by likelihood fitting. Worse still, the capability brought about by scale (e.g., inferring the intended whole from a corrupted part) leads a LLM silently \emph{repair} the anomaly before any downstream security head ever sees it. We formalize this as a tradeoff between two quantities: the cognitive capability $K(W)=I(W;P)$ that a model holds about the world distribution, and the security-information retention $R(W)=\frac{I(Z;S)}{I(A;S)}$ that survives in its intermediate representation. We derive three information-theoretic bounds to connect them: an upper bound on restoration accuracy in terms of $K(W)$, an upper bound on $R(W)$ in terms of the restoration error, and a Fano lower bound on security-task error in terms of $R(W)$. These theorems approximately imply that usable security information is a product of how much security information survives and how well it can be extracted, indicating that optimal usable security information can be achieved at a certain model size. We state this as the \emph{condensing law of security information}: usable security information is unimodal in model size, so security generalization error is U-shaped and is minimized at a finite $N^{\star}$ rather than decreasing without bound. The experiments on various real-world LLMs validate the condensing law, indicating that there is an optimal model size to solve security issues.
Source:https://arxiv.org/abs/2611.17918PreviousGame-Theoretic Analysis of RIS-Assisted Challenge-Response PLA - Part II: Partial Channel KnowledgeNextBeyond Bot Labels: Retrospective Consequence-Grounded Ranking of Attributed Cache Harm in E-Commerce Logs
Related posts
Cross-Benchmark Transfer from RL on Agentic Coding Tasks
0arXiv cs.SE
Programmable Distributions of Semantics-Preserving Variants for Evaluating Code Agents
0arXiv cs.SE
Kusner's conjecture on equilateral sets is false
0arXiv cs.NA
Nearly Optimal Fixed-Confidence Best-Arm Identification with 1-Bit Feedback
0arXiv stat.ML
When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
0arXiv stat.ML
Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts
0arXiv stat.ML