AI & ML
Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise
arXiv stat.ML0 views
arXiv:2411.13711v2 Announce Type: replace-cross Abstract: This paper establishes, to our knowledge, the first almost sure convergence rate and the first maximal concentration bound with sub-Weibull tails for general contractive stochastic approximation algorithms with Markovian noise. As a corollary, we also obtain convergence rates in $L^p$. Key to our successes is a novel discretization of the mean ODE of stochastic approximation algorithms using intervals with diminishing (instead of constant) length. Applications yield the first almost sure convergence rate for $Q$-learning without count-based learning rates and the first maximal concentration bound for off-policy temporal difference learning, both with Markovian samples.
Source:https://arxiv.org/abs/2411.13711PreviousDesigning Algorithmic Recommendations to Achieve Human-AI ComplementarityNexttempdisagg: A Python Framework for Temporal Disaggregation of Time Series Data
Related posts

Your guide to containers at AWS re:Invent 2026
0AWS Containers Blog

Three Asian forwarders, one corridor: what Nippon Express, Yusen and CJ Logistics actually publish
0dev.to Data
Tương quan số liệu Nice vs PSG: Phép thử phòng thủ tại Allianz Riviera
0dev.to Data
Jacksonville, Florida Property Listings Dataset (2026-11-04): 6534 Rows, Excel Spreadsheet
0dev.to Data
Tương quan phong độ Parma vs AS Roma: Bài toán kiểm soát hỏa lực đội khách
0dev.to Data
Schalke 04 chạm trán Stuttgart: Bài toán hiệu suất chuyển hóa tại VELTINS-Arena
0dev.to Data