AI & ML
Speed limits for information acquisition in stochastic learning dynamics
arXiv stat.ML0 views
arXiv:2609.08219v2 Announce Type: replace-cross Abstract: Neural networks acquire internal representations through learning. In this work, we formulate stochastic gradient descent (SGD) as a Markovian stochastic process and derive a Fisher-information flow speed limit that bounds the rate at which trainable parameters can acquire information about data-generating parameters. The resulting inequality decomposes the upper bound on the information flow into drift and noise contributions, thereby quantifying the roles of deterministic learning forces and SGD-induced fluctuations from an information-theoretic perspective. We verify the bound in analytically tractable basis-function linear regression, where the information budget predicted by the bound reproduces the ordering and characteristic time scales with which different data-generating parameters are encoded in the learned parameters. These results establish Fisher-information speed limits as a quantitative framework for understanding which properties of the data-generating process are acquired, in what order, and how rapidly they can be encoded in the trainable parameters.
Source:https://arxiv.org/abs/2609.08219PreviousDeep Barycentric Regression for Optimal Transport Map Estimation and Its Statistical OptimalityNextSymplectic Hamiltonian Direct Discontinuous Galerkin Method for Wave Propagation
Related posts
YuGPT backs up ChatGPT and Codex chats locally and offers flexible search across them. It's quietly finding buyers. Thank you to everyone who bought it. I'm working on improvements and plan to release
0dev.to
GEO 的第一步,不是多發幾篇文章
0dev.to AI
Sol's Take: Saturday — November 07, 2026
0dev.to AI

Meet LiveReview: Auto Approve 30-50% of Your Low-Risk PRs, Safely
0dev.to AI
Snakes and Ladders as a Markov chain: 39.23 throws, computed and simulated
0dev.to WebDev
The 5 Best Next.js RAG and Chat-With-Your-Documents Templates in 2026
0dev.to WebDev