Backend
Blind Noise Equalization for Large Data Matrices under General Variance Profiles
arXiv stat.ML0 views
arXiv:2611.08784v1 Announce Type: new Abstract: Low-rank approximation is a basic tool for inference from large and noisy data matrices. However, when noise variances differ across entries, a setting known as heteroskedasticity, the noise can distort the observed spectrum, producing spurious components and obscuring weak signals. To address these challenges, we develop BRINE (Blind Resolvent-based Iterative Noise Equalizer), a data-driven procedure that iteratively rescales rows and columns to equalize the average noise variance across both dimensions. Under suitable conditions, this procedure restores the Marchenko-Pastur law for the noise spectrum and can amplify signal components without prior knowledge of the signal or noise structure. A key idea underlying our approach is to equalize the diagonal entries of the data resolvent as a proxy for equalizing the unknown noise variances. At the population level, we identify a potential function that characterizes this equalization and motivates an iterative minimization procedure. We prove convergence of the iterations to the true equalizer for arbitrary positive variance profiles, with a geometric rate under mild conditions. The updates can be estimated directly from noisy data, and we establish probabilistic error bounds for the scaling factors. Simulations and experiments across diverse biological data modalities and preprocessing pipelines illustrate the effectiveness of our approach.
Source:https://arxiv.org/abs/2611.08784PreviouspESTO: Iterative p-greedy kernel regression with risk-driven Early STOppingNextHigh-Dimensional Off-Policy Evaluation without Sample Splitting using an Onsager Correction
Related posts
How AI Agents Pick Their Own Tools at Runtime (Without Hardcoding)
0dev.to
My On-Call Agent Stopped Suggesting the Fix That Failed
0dev.to
One forward pass per level: replaying Next-Concept Modeling in the browser
0dev.to
Ein Zwei-Personen-Studio, eine Quest im Regal — und ein unerwarteter Blick aufs Sehtraining
0dev.to

Polymarket Trading Bot: Building a Real-Time Polymarket Market Maker( How WebSockets and Low-Latency Architecture Work)
0dev.to
Building OneLifeVibe: Designing a Hub So Occasional Tools Stay Findable — What Worked, What Broke
0dev.to