When the host holds the state and checks every change the model proposes, the state the host holds was exactly right on every step on 10 of 11 open models, from 1.5B to 32B parameters, in 3,200-step runs with 0 corrupted steps. Model size mattered a lot in other places: when requests were written in ordinary words, a plain model got 2.31 percent of steps right at 1.5B and 97.91 percent at 32B. The one model that failed (a reasoning distill) failed for a reason that is listed below.
Naming note: "AXI" here is the AXI host-side runtime for LLM agents, an architecture by Rajnish Choubey. It is not the ARM AMBA AXI chip-bus protocol.
What was measured?
The same jobs, gate, repair and settings on every model (Tests 35, 37 and 41): vLLM 0.31.0, one NVIDIA L4, prefix caching on, greedy decoding, no retry, 32 jobs of 100 steps (3,200 steps) with the grammar that names the coordinates. Only the model name changes. "Exact" means the state the host holds equals the true state after the step.
| Model | Exact share, 3,200 steps | Steps per GPU hour | Kept chat, computed prompt tokens per step vs exact state |
|---|---|---|---|
| Qwen2.5-1.5B | 100.00 % | 258,284 | 1.24 times |
| Qwen2.5-3B | 100.00 % | 136,952 | 1.24 times |
| Qwen2.5-7B (4-bit) | 100.00 % | 86,374 | 1.24 times |
| Qwen3-8B (4-bit, thinking off) | 100.00 % | 83,524 | 1.31 times |
| Llama 3.1 8B (4-bit copy) | 100.00 % | 83,713 | 1.24 times |
| Phi-3.5-mini | 100.00 % | 95,320 | 1.37 times |
| Mistral 7B v0.3 | 100.00 % | 58,446 | 4.11 times |
| Gemma 3 4B (changed settings, below) | 100.00 % | 104,157 | 1.34 times |
| Qwen2.5-14B (4-bit) | 100.00 % | 32,854 | 1.24 times |
| Qwen2.5-32B (4-bit, part of its weights in CPU memory, below) | 100.00 % | 1,471 | 1.24 times |
| DeepSeek-R1-Distill-Qwen-7B | 1.53 % | 16,523 | 2.34 times |
Corrupted steps: 0 in every cell of every model. Steps per hour are not comparable across rows with different cards or settings: the 32B ran on an L4 with 10 GB of its weights in ordinary memory, and Gemma ran in bf16 with a 64,000-token cache where the plan had fixed fp16 and 32,000 tokens (the engine refused the planned settings; the test page reports both changes).
Why did the DeepSeek-R1 distill fail?
Every reply to a delete (738 + 354 of the 3,200 steps in the main cell) ran to the 96-token cap and could not be read, so the host dropped those replies. Its state stayed well formed (0 corrupted steps) but fell behind the true state. Adds and edge additions were read on all 1,069 and 1,039 steps.
Where did model size matter?
When the request is in ordinary words (no coordinates named), the model has to read the sentence. The "words" cell, plain against the host reading the sentence itself:
| Model | Words read by the model | Words read by the host |
|---|---|---|
| Qwen2.5-1.5B | 2.31 % | 15.69 % |
| Phi-3.5-mini | 4.09 % | 25.00 % |
| Llama 3.1 8B | 5.16 % | 32.00 % |
| Qwen2.5-3B | 7.47 % | 23.09 % |
| Mistral 7B v0.3 | 9.88 % | 24.03 % |
| Gemma 3 4B (changed settings) | 19.34 % | 58.84 % |
| Qwen3-8B | 35.69 % | 66.06 % |
| Qwen2.5-7B | 41.53 % | 64.00 % |
| Qwen2.5-14B | 47.72 % | 90.72 % |
| Qwen2.5-32B | 97.91 % | 100.00 % |
| DeepSeek-R1-Distill-Qwen-7B | 0.00 % | 0.91 % |
The host reading the sentence lifted the share on all 11 models. It did not make a weak model exact on ordinary words, and it was not meant to: the page says so. Size alone did not order the models either (Llama 3.1 8B is below Qwen2.5 3B on the plain words cell).
What does this say, in one paragraph?
In these runs, when the grammar names the coordinates and the host checks every change, the model's size changed speed and cost, not whether the host's state was right: 100.00 percent on 10 of 11 models, 1.5B to 32B. If you make the model also find what to change in loose language, size matters a great deal, and the host's check cannot supply what the model never understood.
What is not shown?
One card type (an L4, also for the 32B with part of its weights in CPU memory), one engine and version, quantised copies for several models, synthetic service graphs, greedy decoding, the Qwen2.5 family for the size ladder, 100-step jobs.
Check it
git clone https://github.com/gaurshalendra2412/axi-runtime cd axi-runtime/python python3 quickstart_replay.py That replays the saved replies of the 4,000-request Gemini run (no key, no GPU). The model tests are docs/MODEL_LADDER_TEST.md, MODEL_LADDER_WORDS_TEST.md, MODEL_FAMILIES_TEST.md and MODEL_LADDER_COMPLETE_TEST.md in the repository: github.com/gaurshalendra2412/axi-runtime. White paper: doi.org/10.5281/zenodo.23958737. Ten open-model tests: doi.org/10.5281/zenodo.24319561.
Architecture by Rajnish Choubey. Tests built and run by Shalendra Gaur.