AI & ML
Centroid Spectral Coupling: Implicit Alignment and Representation Diversity in Self-Supervised Learning
arXiv stat.ML0 views
arXiv:2611.22158v1 Announce Type: cross Abstract: Self-Supervised Learning (SSL) aims to align augmented views while preserving representation diversity and preventing collapse. Existing non-contrastive methods often address these goals using multiple weighted loss terms. We introduce Centroid Spectral Coupling (CSC), a single-term SSL objective that implicitly couples view alignment, representation diversity, and collapse resistance. CSC constructs centroids from independently normalized augmented-view pairs and applies a ridge-based spectral objective to their uncentered second-moment matrix. Preserving centroid magnitudes encodes view agreement while encouraging balanced representation energy across spectral directions. We establish an exact decomposition into nonnegative alignment and spectral-imbalance components, characterize the geometry of global optima, and derive finite-loss guarantees for alignment and diversity. We also show that complete constant-output collapse is a strict saddle in normalized representation space. Experiments on CIFAR-10, CIFAR-100, and Imagenette show that CSC achieves the highest linear Top-1 accuracy among the evaluated non-contrastive methods on all three benchmarks, and the highest kNN Top-1 accuracy on both CIFAR datasets. A ridge-parameter sensitivity study reveals an empirical alignment--diversity trade-off: increasing $\kappa$ strengthens view alignment but reduces representation effective rank. These results show that a unified spectral objective yields competitive visual representations without explicit contrastive negatives or separately weighted alignment and diversity penalties.
Source:https://arxiv.org/abs/2611.22158PreviousCOSI: A Framework for Robustly Valid InferenceNextTraining Neural Networks to be Locally Adaptive
Related posts
Microsoft’s new Licensed Weights lets businesses customize AI models with their own data
0ZDNet

“Collaboration over competition”: Hugging Face hands model inference traffic to OpenRouter
0The New Stack
Robots learn human movement patterns to find paths through dense crowds
0Phys.org Tech
Researchers find eight ways AI could lead to our downfall
0Phys.org Tech
Q&A: Why transportation is critical to rebuilding countries after war and terrorism
0Phys.org Tech
Wine Merges Remote OpenGL Swapchain Image Presentation, CEF Now Works With Wayland
0Phoronix