AI & ML
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
arXiv cs.DC0 views
arXiv:2610.41160v1 Announce Type: cross Abstract: Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.
Source:https://arxiv.org/abs/2610.41160PreviousZenKO: Guided and Iterative Quantum-Classical Method for Molecular Electronic Structure using Sample-Based Quantum DiagonalizationNextTIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
Related posts
World Models Will Soon Be Inescapable. Here’s What You Need to Know
0Wired
HN 1위 'Subvocal — 소리 없이 컴퓨터와 대화하기'가 594 업보트를 받으며 개발자 커뮤니티를 달궜습니다. 서브보컬(아성대 근육 미세 신호)만으로...
0dev.to UX
Phân tích dữ liệu Cagliari 1-0 Frosinone: Hiệu quả tối đa của Adopo và điểm tựa Caprile
0dev.to Data

Phán Quyết Kỷ Luật Tại Giải VĐQG Morocco: Biến Động Điểm Số Của Widad Temara Và Wydad Casablanca
0dev.to Data
Switching Recovery Tools Is a Data Job Before It Is a Vendor Job
0dev.to Data
⚡ What Every Developer Forgets!
0dev.to Discuss