Trace Shape Reveals Model Capability: 3B vs 7B
Last week I traced the same agent running on two different model sizes. Same code, same task, same prompts. The only difference: the model.
- Phone: Qwen2.5-3B (3 billion parameters, running on a Pixel 8 Pro)
- Legion: Qwen2.5-7B (7 billion parameters, running on an RTX 4060)
I didn't read a single output. I just counted spans.
What the traces showed
3B model: Every trace was exactly 2 spans. One agent.run, one llm.call. Zero tool calls. Ever.
7B model: Mean 5.8 spans per trace. Multiple llm.call spans, multiple tool.call spans. It actually used tools.
The 3B model wasn't being lazy. It couldn't do tool calls — not reliably, not in this setup. It would try to answer directly, skipping the tool-use loop entirely. The trace shape was degenerate: a straight line from start to finish, no branching, no tool interaction.
The 7B model, same code, same prompts, would explore: call a tool, read the result, call another tool, synthesize. The trace shape was rich — branching, iterative, agentic.
Model Traces Mean spans Tool calls ------------ ------ ---------- ---------- Qwen2.5-3B 6 2.0 0 (degenerate) Qwen2.5-7B ~500 5.8 yes (agentic) Why this matters
You can tell how capable a model is by looking at the shape of its traces, without ever reading the outputs.
This isn't about benchmark scores. It's about behavioral fingerprinting. A 3B model and a 7B model don't just give different quality answers — they behave differently as agents. The 3B doesn't do the agentic loop. The 7B does.
If you're evaluating models for agentic use, don't just look at accuracy. Look at trace shape:
- Does it use tools, or try to answer directly?
- How many steps does it take?
- Does the shape change with task complexity?
A model that can't produce rich trace shapes can't do agentic work, no matter what its benchmark scores say.
The pattern
This is the second time trace shape has told the story:
- Prime Agent experiment: Same model, different environments → 3.8x shift in span counts. Environment is input.
- This experiment: Same environment, different models → degenerate vs rich traces. Model size determines agentic capability.
In both cases, I never read a prompt or output. The metadata was enough.
That's the thesis behind AgentPulse: trace shape is a behavioral fingerprint. Whether you're detecting environment drift, model regressions, or capability gaps, the shape tells you first.
All traces captured with AgentPulse (metadata-only). No prompts, outputs, or tool inputs were recorded.
