AI & ML
Rare-Tail Statistics For Learning Biased Gaussian Halfspaces with Label Noise
arXiv stat.ML0 views
arXiv:2611.24181v1 Announce Type: new Abstract: We revisit the classical problem of learning general Gaussian halfspaces in the presence of label noise, with particular focus on the large-bias regime, where information-computation gaps are conjectured in several standard label-noise models. Concretely, we present and analyze the rare-tail mean estimator, an extremely simple one-shot statistic that averages only the observed minority class. In the large-bias regime, classical Averaging (Servedio, 1999) and related global first-moment statistics are provably suboptimal, while the rare-tail mean estimator recovers the target halfspace direction at the information-theoretic sample rate (modulo log factors) under a broad class of label noise models, including probit noise and margin-dependent noise with very slow decay away from the boundary. Notably, our guarantees allow labels arbitrarily close to the decision boundary to be nearly random, while still giving strong directional recovery from a small number of samples. The estimator also cleanly interpolates between the noiseless and noisy regimes: without noise, it attains the same sample rate as classical separability-based methods for realizable halfspaces, and under low noise, or noise that decays away from the boundary, it keeps this rate even after linear separability has been destroyed. As an application, we consider a two-stage curriculum in which source samples, whose noise may be arbitrarily close to $1/2$ near the boundary and decay very slowly away from it, precede target samples, whose noise flips every label at the same rate, i.e., random classification noise (RCN). Since both stages share the same Bayes optimal halfspace, our estimator supplies the directional warm start needed by existing refinement routines, and together they learn the RCN target at the information-theoretic sample rate in the large-bias regime, up to log factors.
Source:https://arxiv.org/abs/2611.24181PreviousNOTREKS: Marginal Independence in Score-based Structure LearningNextStability-compatible kernels: beyond Gaussianity for impulse response estimation
Related posts
Razorpay webhook not verified: anyone can fake a payment
0dev.to
From FAQPage Schema to Answerable AI Documentation
0dev.to
Wiring Android's MediaPipe LLM Inference API to a Multi-Turn Conversation Engine
0dev.to
What Is LLM Throughput? Tokens per Second, Concurrency, and Batching
0dev.to
Diese Woche haben mir zwei Tools etwas vorgeschlagen und es direkt abgeschickt.
0dev.to
AI Agent Observability: What to Log Before Production
0dev.to