AI & ML
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
arXiv cs.DC0 views
arXiv:2610.41160v1 Announce Type: cross Abstract: Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.
Source:https://arxiv.org/abs/2610.41160PreviousZenKO: Guided and Iterative Quantum-Classical Method for Molecular Electronic Structure using Sample-Based Quantum DiagonalizationNextTIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
Related posts
The Solo Sailor and His AI Crew
0dev.to
XDR - Experience Decision Records
0dev.to UX
VR에서 YouTube를 본다는 것: 약시(amblyopia) 사용자에게 몰입형 뷰 개편이 의미하는 것
0dev.to UX
Mercedes Put Depth in the Windshield. Depth Was Never Guaranteed for Everyone.
0dev.to UX
유나이티드 항공 좌석 등받이의 Spotify가 시각 훈련에 던지는 힌트
0dev.to UX
SteamVR's Image Quality Beta Is Really a Legibility Story — And That Matters If You Render Science
0dev.to UX