AI & ML
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
arXiv cs.DC0 views
arXiv:2610.41160v1 Announce Type: cross Abstract: Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.
Source:https://arxiv.org/abs/2610.41160PreviousZenKO: Guided and Iterative Quantum-Classical Method for Molecular Electronic Structure using Sample-Based Quantum DiagonalizationNextTIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
Related posts
New Linux Patches Bring-Up The Samsung Galaxy XR Headset, Qualcomm Halliday SoC
0Phoronix

Some United Airlines flights now let you sign into Spotify on your flight — and its' totally free
0TechRadar

Straight Talk just dropped an unlimited data plan to $20/mo ahead of Black Friday — a brand new record
0TechRadar

Who Gives A Crap: Data breach at ethical toilet paper company exposes customer information across UK, US and Australia
0TechRadar

Since tokenmaxxing is the wrong metric for AI ROI, what is the right one?
0TechRadar

Narwal V50 Cube review: a self-emptying stick vacuum with brilliant smart features
0TechRadar