AI & ML
Asymptotic Margins and Feature Confinement in Conic Classification Heads under Softmax Cross-Entropy
arXiv eess.SP0 views
arXiv:2611.18507v1 Announce Type: cross Abstract: Classification heads built on conic functions give each class a bounded region of the feature space, and such regions are used to reject samples of unseen classes. We ask whether these regions also keep the features bounded when the head is trained with softmax cross-entropy. For a head whose class scores combine a linear term with a Euclidean norm, we first show that the region of each class is bounded by a quadric of revolution, whose eccentricity is the reciprocal of one learned ratio. We then attach to each class an asymptotic margin, which compares the growth of its score with the growth of all the other scores and can be computed by a second-order cone program. The loss of a class has infimum zero exactly when its margin is positive, and it then approaches zero only along diverging features. When the margin is negative, the loss is coercive and its minimizers lie in an explicit ball. Since softmax depends only on the differences between scores, a class with a bounded region can still have a positive margin, so bounded regions do not confine the features. The result extends to other norms, including the polyhedral conic functions. On CIFAR-10 and CIFAR-100, all 1210 cells of 67 trained conic heads have positive margins, whereas a prototype head keeps nine of its ten cells bounded, as the theory predicts.
Source:https://arxiv.org/abs/2611.18507PreviousPACT: A Decoder-Confidence-Guided Two-Round Receiver for Joint Channel and Phase-Noise Estimation in Sub-THz OFDMNextMulti-Band Sensing in FR3 with Background Dense Multipath Components
Related posts
AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling
0arXiv cs.CR
Denoising the US Census: Succinct Block Hierarchical Regression
0arXiv cs.CR
Adversarial Attacks on Downstream Weather Forecasting Models: Application to Tropical Cyclone Trajectory Prediction
0arXiv cs.CR
Cyber-Zero: Training Cybersecurity Agents without Runtime
0arXiv cs.CR
Coordinate Frame Recovery, Per Class NMS and Null Return Modelling for Sim2Real V2X LiDAR Detection
0arXiv cs.RO
Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection
0arXiv cs.RO