AI & ML
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
arXiv cs.DC0 views
arXiv:2610.41160v1 Announce Type: cross Abstract: Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.
Source:https://arxiv.org/abs/2610.41160PreviousZenKO: Guided and Iterative Quantum-Classical Method for Molecular Electronic Structure using Sample-Based Quantum DiagonalizationNextTIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
Related posts
A small business running 8 projects on Claude Code for 30 days: rules ignored, projects crossing into each other, and what finally helped
0dev.to AI
L'IA trouve les peptides, la VR les révèle : pourquoi Antigravity change la donne pour la découverte de médicaments
0dev.to AI
Why Tesseract Fails on Cursive Handwriting (And How We Built a Two-Tier Vision Pipeline in Node.js)
0dev.to AI
OFX looks like XML. It isn't, and QuickBooks will reject it
0dev.to WebDev
Before You Buy GEO Monitoring Software, Build a 50-Line Baseline
0dev.to Python
Building Self-Improving AI Models Without Blinding Yourself to Failure
0dev.to Python