AI & ML
High-probability convergence of adaptive, proximal, and clipped Adam-type methods for nonconvex stochastic optimization
arXiv cs.NA0 views
arXiv:2611.17736v1 Announce Type: cross Abstract: This paper develops a unified high-probability analysis of adaptive, proximal, and clipped first-order methods for nonconvex stochastic optimization. For a smooth objective whose stochastic gradient error has sub-Gaussian norm, the normalized adaptive gradient method attains the standard stochastic nonconvex complexity of the square root of the number of iterations, with an additional logarithmic dependence on the inverse confidence level that is inversely proportional to the number of iterations. The proof uses an adaptive descent decomposition together with Freedman and Bernstein concentration inequalities. The same guarantee is established for the full adaptive moment method under an explicit momentum-alignment condition. The analysis is extended to projected and proximal updates through a stationarity measure based on the subdifferential distance, to objectives satisfying the Polyak-Lojasiewicz condition, and to heavy-tailed noise through truncation. The framework is applied to stochastic inverse problems in elasticity, and numerical experiments illustrate the theoretical scaling and the practical behavior of the methods.
Source:https://arxiv.org/abs/2611.17736PreviousProximal Adam algorithm for nonsmooth composite optimization and linear convergence under proximal PL geometryNextScalar Entropic Damage Transport: Well-posed softening with a single damage variable
Related posts

94% of Singapore Small Business Owners Surveyed Say More AI Guidance Is Needed
0Meta Newsroom
Nashville, Tennessee Real Estate Dataset: 7,436 Listings, Excel File (Nov 10, 2026)
0dev.to Data
A Hundred and Eighty Six Thousand Pounds of Stock Had Left One Branch and Never Arrived at Another
0dev.to Data
Đánh giá thông số 20 tiền đạo xuất sắc nhất bóng đá châu Âu thế kỷ 21 từ BBC Sport
0dev.to Data
Thống Kê SHB Đà Nẵng: Bài Toán Phòng Ngự Chờ HLV Lee Young-jin Hóa Giải
0dev.to Data
Alibaba's Qwen4-Max Is Here — and It's Gunning for Claude's Coding Crown
0dev.to News