"prefill energy grows quadratically with the hidden dimension, so larger LLMs suffer most from long inputs;" [1]. That closed‑form observation collapses years of ad‑hoc profiling into a single analytic curve. It simultaneously predicts how much extra work each new token imposes on the KV‑cache and where the sweet spot between prompt length and compute lies. In practice, engineers can now plug model size and desired context into one equation and obtain a realistic energy budget.
Before this work, inference cost estimators were essentially black‑box fits that required a separate calibration for every architecture. State‑of‑the‑art models swallowed architectural hyperparameters into dozens of coefficients, sacrificing interpretability for marginal accuracy. As a result, scaling studies had to rerun expensive traces whenever a new model variant appeared. The field lacked a principled way to separate the cost of loading an input from the incremental cost of generating each token.
"The Prefill Amortization term grows quadratically with , while the Decode Accumulation terms grow only linearly: the KV‑cache term is proportional to , and the Next-token term is affine in due to the constant offset ." [1]. This decomposition shows that the dominant driver of early‑stage compute is a quadratic function of hidden size, whereas the per‑step decode overhead scales linearly with the number of cached keys and values. Consequently, doubling the hidden dimension quadruples the prefill energy, but adding another thousand tokens only adds a linear increment. The formula makes the trade‑off explicit: larger models demand shorter prompts if energy consumption is to stay bounded.
KV‑cache energy is proportional to context length, contributing to the super‑linear growth observed when reasoning token budgets expand, together with the quadratic prefill term. As more tokens are generated, each new key‑value pair must be read and written for every subsequent step, inflating the memory‑access cost faster than the pure compute term. This explains why extending a reasoning window beyond an analytically derived optimum hurts “intelligence per joule” by up to threefold across a four‑fold budget increase. The linear KV term therefore becomes the bottleneck in long‑generation scenarios.
"with only four parameters fitted once across all LLMs, it reduces the error of previous SOTA models by 3.1–3.5, reaching a MAPE of 11.97%." [1]. Across traces of thirteen dense models on H100 GPUs the SIMPLE law attains an R² of 0.969, confirming that the compact formula captures real hardware behavior. Its prediction error is three to three‑and‑a‑half times lower than earlier empirical fits, despite using a single parameter set for every architecture. The result validates that a tiny analytical model can replace heavyweight per‑model calibration.
The study leaves open how well the law transfers to sparsely activated transformers, quantized weights, or non‑NVIDIA accelerators. Its derivation assumes dense matrix multiplications and ignores cache‑hierarchy effects that may dominate on CPUs or edge GPUs. Moreover, while energy is a critical proxy for cost, latency and throughput are not directly addressed; a model could be energy‑efficient yet too slow for real‑time serving. These gaps suggest a research agenda around extending the SIMPLE framework to heterogeneous hardware and richer performance metrics.
If engineers adopt the quadratic‑prefill insight as a first‑order budget check, they can prune prompts before running any inference, saving both time and electricity. Benchmarks that sweep context length should now report the SIMPLE‑predicted energy curve alongside empirical latency, allowing a direct comparison of theoretical minima versus observed sweet spots. By treating the four‑parameter law as a design constraint, product teams can decide whether to invest in longer KV caches, more aggressive GQA grouping, or simply smaller hidden dimensions for cost‑sensitive deployments. Will future KV‑cache architectures rewrite this quadratic rule, or will engineers simply learn to live within its limits?
