Backend
The State Is Not the Bottleneck: A Paged-State Gated-DeltaNet Tile on a Tri-Mode Multicast Mesh, Bit-Exact from RTL to Silicon
arXiv cs.RO0 views
arXiv:2611.18422v1 Announce Type: new Abstract: Linear-attention language models replace a growing key-value cache with a recurrent state of fixed size. On an edge FPGA, this state can exceed the on-chip memory and must be paged to external memory. This paper asks whether such paging dominates the cost of decoding (generating tokens). We built a hardware tile that computes one complete layer of a custom 369-million-parameter hybrid Gated DeltaNet model on an Agilex 5 FPGA and pages its recurrent state. The model state needs 6.2 MiB, whereas the device has 0.87 MiB of block memory. Paging adds 26.0 MB of memory traffic per decoded token, at most 11.2 percent of the calculated total. On the board at 75 MHz, with synthetic weights, one token takes 11.15 ms in this layer. The matrix-vector products with the weights take about 62 percent of this time and the state phase, including the paging, 13 percent. The tile is one node of a multicast mesh of tri-mode processing elements. We measure the error of the fixed-point arithmetic against floating point and check that the hardware agrees bit for bit with a fixed-point reference. In simulation, the state logic agrees with the reference on all 18 recurrent layers. On the board, six builds pass all 30 runs of 64 tokens and two runs of 1,024 tokens. For this model and memory interface, the delivery of the weights limits the decoding performance, and the recurrent state does not. State paging is therefore practical for such a model on an edge FPGA.
Source:https://arxiv.org/abs/2611.18422PreviousVision-Language-Action Models: A Life-Cycle Survey of Design, Training and DeploymentNextBeyond Attack-Based Accuracy: Rethinking Adversarial Robustness Evaluation Through Structural Gradient Alignment
Related posts
I built Trongate CLI, a command-line toolkit for the Trongate PHP Framework
0dev.to OpenSource
sheet_route: Flutter bottom sheets as real routes, with snap points and go_router pages
0dev.to OpenSource

Introducing Sawala Mono: An Ergonomic Coding Font Built for Code Reviews and Terminal Workflows
0dev.to OpenSource
![Robot Middleware and ROS2 — Ch.06: ROS2 Security [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3oowmzpb4qkk5km0di1k.png)
Robot Middleware and ROS2 — Ch.06: ROS2 Security [EN]
0dev.to Programming
![Robot Middleware and ROS2 — Ch.07: ROS2 Lifecycle Management [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx3a51weo0z1ob95yn3x2.png)
Robot Middleware and ROS2 — Ch.07: ROS2 Lifecycle Management [EN]
0dev.to Programming
![Robot Middleware and ROS2 — Ch.08: ROS2 Deployment and DevOps [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9z4fmoxmqqngiro7boaa.png)
Robot Middleware and ROS2 — Ch.08: ROS2 Deployment and DevOps [EN]
0dev.to Programming