AI & ML
Chiron: Cache Management Empowered by LLM Intelligence for AI Workloads
arXiv cs.DC0 views
arXiv:2611.06179v1 Announce Type: new Abstract: AI clusters keep data in remote object stores and rely on local caches to stop GPUs from waiting for data. A cache performs well only when its prefetching and eviction policies match the access pattern, yet these patterns differ across AI workloads and change between phases of one workflow. Existing methods either need code changes, which compromises user transparency, or use low-end heuristics, which cannot follow such changes. Large language models (LLMs) can infer access behavior without user hints, but direct LLM control of cache actions suffers from long reasoning delay, high token cost, and poor decision quality. We present Chiron, a cache system based on advisory switching control. An LLM asynchronously diagnoses the access pattern and recommends a policy. A deterministic harness gates this advice, executes expert-designed policies locally, and returns cache feedback for review. We implement Chiron atop JuiceFS and evaluate it on nine AI workloads. It reduces job completion time by 14.3-41.2% over the best baseline on popular AI workloads.
Source:https://arxiv.org/abs/2611.06179PreviousEfficient Stable Word-RAM Simulation by Population ProtocolsNextPerformance Portability of Out-of-Kernel Optimizations on GPUs: A Molecular Docking Case Study
Related posts
Changing the Time of Day in a Photo with a Text Prompt
0dev.to Tutorial
Soft vs Hard Subtitles: Which One Your Video Actually Needs
0dev.to Tutorial

The 5 Steps That Get Freelancers Paid (without sounding desperate)
0dev.to Career
What Are the Highest-Paying Machine Learning Jobs?
0dev.to Career
Two questions I added after readers replied
0dev.to Beginners

Government Workforce Training: How LMS Platforms Support Public Sector Learning and Development
0dev.to Beginners