AI & ML
Geometry-Aware Score-Repellent Monte Carlo
arXiv stat.ML0 views
arXiv:2611.08909v1 Announce Type: new Abstract: The recently proposed Score-Repellent Monte Carlo (SRMC) method improves MCMC sampling by tilting the target density $\pi$ in $\mathbb{R}^d$ with the factor $\exp(-\alpha\,\theta_n^\top s(x))$, where the score function $s(x) = \nabla_x \log \pi(x)$, and $\theta_n$ is the score average from past samples. However, a single scalar repellence strength $\alpha\ge0$ cannot handle both the steep and the flat directions in anisotropic energy landscapes at once. We introduce \emph{Geometry-Aware SRMC} (GA-SRMC), which generalizes the Euclidean alignment to $\theta^\top M s(x)$ for any symmetric positive-definite matrix $M$. A stochastic-approximation central limit theorem under a trace constraint on $M$ identifies the trace-normalized inverse Fisher $M_{\rm opt}\propto S^{-1}$ as the unique minimizer of the worst-direction covariance bound, elevating the Fisher information $S=\text{Cov}_\pi(s,s)$ from a proposal-side preconditioner (e.g., FisherMALA, natural gradient) to the bound-optimal repulsion-side matrix, with a single online estimate $\widehat S_n$ serving both roles at no additional computational cost. The full-Fisher online metric $M_n$ converges almost surely to $M_{\rm opt}$ and satisfies a CLT, providing a theoretical guarantee for the online implementation of the GA-SRMC framework. On an $8$-target simulation spanning the condition number $\kappa\in[1.3,100]$ and dimension $d\in[2,785]$, GA-SRMC dominates SRMC in ESS by $\boldsymbol{4.3\times}$ on anisotropic Gaussian ($\kappa=100$) and $\boldsymbol{6.8\times}$ on MNIST under the MALA baseline, by $\boldsymbol{16\text{--}32\%}$ on three well-conditioned logistic-regression posteriors, and by up to $\boldsymbol{+48\%}$ under FisherMALA; gains peak on the most anisotropic targets, and posterior-mean MSE is lower than with SRMC in all $16$ full-Fisher implementations.
Source:https://arxiv.org/abs/2611.08909PreviousBeyond the Lower-Star Filtration: Alternative Filtration Methods for Persistent HomologyNextKernel-Based Learning of Analytic Operators and Parametric PDEs
Related posts

AI Needs a Body: Three Builds from Tomorrow Lab® Where AI and Hardware Work as One
0HackerNoon

Meet the Hackathon Winner: Shugert on Smarter Shopify Engineering and the Future of E-Commerce
0HackerNoon

Rillet Opens in London to Help UK Finance Teams Close the Books in Real Time
0HackerNoon
Anthropic joins AI’s natural gas frenzy, buys $3.4B worth of turbines
0TechCrunch
Uber launches parking service following SpotHero acquisition, with an eye on robotaxis
0TechCrunch
Conectar WordPress a ChatGPT y Claude: puente OAuth
0dev.to