AI & ML
Scaling Shapley Energy Attribution to Large LLM Batches
arXiv cs.DC0 views
arXiv:2611.13802v1 Announce Type: new Abstract: LLM services have become essential infrastructure for applications such as chat assistants and coding agents. Service providers need to attribute resource consumption to individual requests for purposes such as billing. Among these resources, energy is a key metric for assessing the cost of serving each request. However, serving engines batch requests together, and GPUs report only aggregate energy consumption. Providers typically estimate each request's share in proportion to its token consumption, but this approach can produce substantially unfair estimates. The Shapley value provides a principled solution to fair energy allocation, but computing it directly requires $2^N-1$ GPU runs for $N$ requests, which is infeasible for large batches. To address this problem, we propose StepSplit. Across GPU generations, we observe that the energy consumed by each inference step, a forward pass over a batch, consists of a shared energy, largely due to reading model weights, and a request-specific energy. We derive a closed-form Shapley value: each request is charged its own energy plus an equal part of the shared energy of every step in which it participates. A microbenchmark of 2.5-17.8 minutes calibrates both parts of the step energy once per model and GPU configuration. Under static batching, StepSplit uses this closed form in a single pass over the scheduler log. Under continuous batching, removing a request can change how the remaining requests are batched, so StepSplit adds a correction to the closed form, computed by replaying the scheduler on a CPU. We evaluate StepSplit against Shapley values measured using 529,513 GPU runs. Under static batching with batch sizes of 8-256, StepSplit achieves an $L_1$ error of 0.4-4.3%, compared with 13.4-61.3% for token-proportional allocation. Under continuous batching with 8-128 requests, the corresponding errors are 0.5-6.4% and 20.6-71.7%.
Source:https://arxiv.org/abs/2611.13802PreviousMulti-Objective Scheduling of LLM-Based Multi-Agent Workflows in Cloud ComputingNextEnergy-Proportional Serverless Computing on SBC Clusters
Related posts
Singapore Airlines elevates travel & work with OpenAI technology
0OpenAI Blog
How to Plan a Five-Slide Social Carousel with One Consistent Character
0dev.to

Kimi K2.5 to K3.1 Pro: 8 API Models, One Enterprise Task, and What Actually Got Done
0dev.to
Visa tested 22 shopping agents. 43% failed - and half of those bought anyway. Here's what actually stops it.
0dev.to
Building WUUUH: Skills Drift Faster Than Confidence — Keeping a Hand In During a Career Gap, What Worked, What Broke
0dev.to
FDR-Controlled Variable Selection for Generalized Linear Models and Cox Regression with Virtual Dummies
0arXiv eess.SP