Backend
On Federated Averaging with Partial Participation under a General Variance Condition
arXiv cs.DC0 views
arXiv:2611.14239v1 Announce Type: cross Abstract: In this paper, we study the convergence of Federated Averaging (FedAvg) with partial participation under a general variance condition. In each communication round, the server samples $S$ of the $n$ clients uniformly at random without replacement. The stochastic gradients are assumed to satisfy the so-called ABC condition, which allows their variance to grow with the local objective gaps. For smooth nonconvex objectives, we establish a finite-time stationarity bound in terms of the heterogeneity measure $\sigma_f^\star:=f^\star-\frac{1}{n}\sum_{i=1}^n f_i^\star$ and the client sampling factor $I_S:=(n-S)/(n-1)$. Here, $f^\star$ and $f_i^\star$ denote the infima of the global average objective and client $i$'s local objective, respectively. The bound exhibits a linear speedup in the number of sampled clients $S$ and recovers the full-participation result when $S=n$. Under diminishing stepsizes, we further prove that the gradient norm converges to zero both in expectation and almost surely. Notably, compared with existing analyses of FedAvg with partial participation, our results require no additional data heterogeneity assumption such as bounded gradient dissimilarity (BGD), and the sampling-induced term in our bound is not amplified by the number of local updates at the leading order. The importance of removing the BGD assumption is illustrated by an example that is smooth and lower bounded but violates BGD, yet is covered by our analysis.
Source:https://arxiv.org/abs/2611.14239PreviousDeterministic, Reproducible Inference in Exact Integer Arithmetic, and Verifiable Distributed Inference as Its ConsequenceNextSelf-stabilization and Byzantine Tolerance for Maximal Independent Set
Related posts
Postgres 14 hits end of life on November 12
0dev.to

MySQL to Snowflake: Binlog CDC Without Kafka or Debezium, Step by Step
0dev.to
Node.js Template Ownership: 3 PDF Watermark API Decisions Before Sharing SaaS Reports
0dev.to
I Built an Open-Source Tool to Test What Happens When MCP Servers Fail
0dev.to
Streamlining complex Helm tooling reduces team overhead and administrative friction
0dev.to

100% annotation agreement can mean one labelled record out of twelve
0dev.to