AI & ML
High-probability convergence of adaptive, proximal, and clipped Adam-type methods for nonconvex stochastic optimization
arXiv cs.NA0 views
arXiv:2611.17736v1 Announce Type: cross Abstract: This paper develops a unified high-probability analysis of adaptive, proximal, and clipped first-order methods for nonconvex stochastic optimization. For a smooth objective whose stochastic gradient error has sub-Gaussian norm, the normalized adaptive gradient method attains the standard stochastic nonconvex complexity of the square root of the number of iterations, with an additional logarithmic dependence on the inverse confidence level that is inversely proportional to the number of iterations. The proof uses an adaptive descent decomposition together with Freedman and Bernstein concentration inequalities. The same guarantee is established for the full adaptive moment method under an explicit momentum-alignment condition. The analysis is extended to projected and proximal updates through a stationarity measure based on the subdifferential distance, to objectives satisfying the Polyak-Lojasiewicz condition, and to heavy-tailed noise through truncation. The framework is applied to stochastic inverse problems in elasticity, and numerical experiments illustrate the theoretical scaling and the practical behavior of the methods.
Source:https://arxiv.org/abs/2611.17736PreviousProximal Adam algorithm for nonsmooth composite optimization and linear convergence under proximal PL geometryNextScalar Entropic Damage Transport: Well-posed softening with a single damage variable
Related posts
AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling
0arXiv cs.CR
Denoising the US Census: Succinct Block Hierarchical Regression
0arXiv cs.CR
Adversarial Attacks on Downstream Weather Forecasting Models: Application to Tropical Cyclone Trajectory Prediction
0arXiv cs.CR
Cyber-Zero: Training Cybersecurity Agents without Runtime
0arXiv cs.CR
Coordinate Frame Recovery, Per Class NMS and Null Return Modelling for Sim2Real V2X LiDAR Detection
0arXiv cs.RO
Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection
0arXiv cs.RO