AI & ML
Minimum Neural Weight Norm = Kolmogorov Complexity
arXiv math.IT0 views
arXiv:2605.10878v2 Announce Type: replace-cross Abstract: Why do neural networks generalize better when their parameter norm is small? We prove that, in looped neural networks with constant-precision weights, the smallest weight norm of a network computing a function is asymptotically equivalent to the Kolmogorov complexity of that function, divided by its logarithm. The proof has two reductions: looped neural networks can encode information at a rate of logarithmically many bits per non-zero parameter encoded via the sparsity pattern, while any fixed-precision network can be described by enumerating its non-zero parameters with logarithmic addressing cost. Both bounds have the same leading constant factors, giving an asymptotic equivalence. More broadly, by establishing a theoretical link between norm minimization and algorithmic simplicity, we provide an avenue towards understanding generalization in modern deep learning.
Source:https://arxiv.org/abs/2605.10878PreviousSemifields and rank-metric codes from Drinfeld modulesNextCost-Efficient LLM Evaluation: A Framework for Narrow Confidence Intervals Under Fixed Budgets
Related posts
Ruby2ßhell: Pre-authentication RCE on stock Rails
0Hacker News
Generalizing self-normalized importance sampling with couplings
0arXiv stat.CO
Bayesian Computation of Copula-based Multivariate Models with an Application to Doping Detection
0arXiv stat.CO
WAIVES: Weak-form Asymptotically-unbiased Instrumental-Variable Estimation for SDEs
0arXiv stat.CO
Treat Your Honeypot Analysis Like Code: Git, Pull Requests and a Human Gate for AI [Guest Diary], (Wed, Nov 18th)
0SANS ISC
ISC Stormcast For Thursday, November 19th, 2026 https://isc.sans.edu/podcastdetail/10186, (Thu, Nov 19th)
0SANS ISC