Backend
Manufacturing Differential Sparsity: Planner-Hardware Co-Design for Efficient World Model Planning
arXiv eess.SY0 views
arXiv:2611.22483v1 Announce Type: cross Abstract: JEPA-style latent world models let embodied agents plan toward new goals with model predictive control, but they are still too slow for real-life embodied control: sampling-based MPC repeats predictor inference tens of thousands of times per plan, and even on high-end GPUs one planning step can take tens of seconds. We observe substantial latent locality in this repeated-inference workload: the predictor maps close actions to close latents, and inside a quantized predictor this closeness takes a form that hardware can exploit, differential bit-level sparsity. Unlike an input stream, whose temporal similarity is given, a planner itself decides which candidates it evaluates together, so the planner can choose its own references and, with reference forest sampling, manufacture more sparsity: it couples the sampled action sequences of each group to a shared root, with competitive planning quality. We build the accelerator on the insights of the rich line of bit-level sparsity accelerators and design it and the planner tightly coupled as a single co-design: the planner increases sparsity and exposes structure for hardware scheduling, and the bit-exact hardware effectively uses the sparsity that the planner creates. On three JEPA world models, our co-design is 1.28-1.80x faster than a dense accelerator and uses 38-52% less energy, and it is 1.22-1.58x faster than prior bit-sparse design and uses 16-36% less energy, with the same area. To our knowledge, our planner is the first planner that manufactures differential bit-level sparsity across its candidates and exposes its structure for hardware scheduling, and our co-design is the first algorithm-hardware co-design for JEPA-style world-model planning.
Source:https://arxiv.org/abs/2611.22483PreviousCOAST: Safe Flight Corridor Planner for Fast and Robust Snap Minimized TrajectoriesNextMean-Field Learning for Storage Aggregation
Related posts
Cryptographic SARIF v2.1.0 Proofs for Autonomous AI Agents & Desktop Suite Innovations (Day 4)
0dev.to
Cybersecurity Best Practices for Small Businesses
0dev.to
Lean AI Second Brain: Babashka, Git & Local LLMs
0dev.to AI

Inside My AI Marketing Agent: More Than a Prompt
0dev.to AI

Building an Intelligent Assistant: My Architect's View of Amazon Bedrock AgentCore
0dev.to AWS
I indexed 2,606 legally free resources to learn the Indian stock market — here's how the no-build site works
0dev.to WebDev