AI & ML
Sigmoid Tensor Decomposition for Compact Representations of Binary Data
arXiv cs.NA0 views
arXiv:2611.08020v1 Announce Type: new Abstract: Nonlinear low-rank decompositions are receiving growing attention as a way to represent structured data with fewer parameters than conventional linear methods. Existing nonlinear approaches have focused mainly on matrices, which limits their ability to capture multidimensional structure. We propose a nonlinear tensor decomposition based on sigmoid nonlinearization in the Tucker and tensor-train formats. A binary tensor is represented by a low-rank real-valued score tensor, whose parameters are estimated by minimizing the binary cross-entropy loss. The resulting optimization problem is jointly nonconvex, but becomes convex with respect to one factor or score block when the remaining factors are fixed, as in the corresponding matrix case. We evaluate the method on several structured binary tensors and compare it with conventional Tucker and tensor-train decompositions across a range of common ranks. These results indicate that sigmoid nonlinearization is particularly effective in the low-rank regime, where it can provide more accurate binary reconstructions than conventional linear tensor decompositions at comparable model sizes.
Source:https://arxiv.org/abs/2611.08020PreviousLithoSAGE: Spatially Adaptive and Geometrically Equivariant Neural Operators for Computational LithographyNextA Second-Order Semi-Implicit Projection Time-Stepping Scheme for the Shallow-Water Equations on the Sphere Using Compatible Finite Elements
Related posts

Save 26% on this 142-piece toolkit for your PC repairs and hobbyist projects, now under $28
0Tom's Hardware
‘Mathematics is more than a mechanical process’ — why I think the field will survive OpenAI’s disruption
0Nature News
Preprint platform offers to pay researchers to review papers, but questions remain
0Nature News

I’ve reviewed almost every Android foldable this year — here’s how the iPhone Duo compares
0The Verge
Physicl raises $10M to make the digital world real enough for robots
0TechCrunch

Kubernetes, mark II? Why Google’s giving away gVisor, a code sandbox used by Anthropic and OpenAI
0The New Stack