Adapted from three posts first published on the Novaeon Studio LinkedIn page (9, 14 and 28 October 2026).

Train a small decider. Keep your big model.

Want an agent that makes better decisions? Don't fine-tune your big model. Train a small one next to it.

Unsloth published a strong example. Daniel Han and team turned Qwen3.5 0.8B into a decision model: a small scoring head on top of the LLM, LoRA rank 64, one epoch. Average accuracy across three decision benchmarks rose from 20.7% to 74.3%, on 4 GB of VRAM. The model writes no text. It scores the options you give it and returns a choice with a probability.

That is the pattern we favour for agent decisions like which tool, which model, escalate or not. Three reasons to keep the decider small and separate:

  1. Latency. The decider runs on every turn, so its cost lands on every single request.
  2. Independent errors. A second model with different blind spots catches mistakes the big model makes with full confidence.
  3. No collateral damage. Fine-tuning a general model on a narrow task can erode what it was good at. Research on continual fine-tuning documents this kind of forgetting.

How to get training data: let the big model act as teacher and label examples, then correct the labels with real outcomes. The bottleneck is data, not GPUs. So we log every live decision.

A small model routes. The big one works.

Should one big model decide everything, including whether it is the right model for the job? We measured that on our own agent stack. The answer was no.

Every incoming task needs three quick decisions: who handles it (answer directly, hand off, or split into parallel jobs), which model does the work (our local model or a stronger cloud model), and whether it needs step-by-step reasoning.

We built a benchmark of 60 realistic requests, each with a fixed reference label, and gave the same rubric to two judges: a small 9B decision model that returns calibrated probabilities, and our main local LLM, a 35B mixture-of-experts model.

Alone, the main model picked the right handler 92% of the time. But of the 20 tasks that needed the stronger model, it kept 9 for itself: debugging, a security review, an email sent on someone's behalf. A model is not a neutral judge of its own limits.

The small model made different mistakes. Their under-escalations overlapped on exactly one task.

So we combined them: a task stays local only if both judges say so. Any doubt goes up. Result on the 60 tasks: handler 92% (55/60, unchanged), model tier 94% (49/52, media tasks excluded), reasoning on/off 97% (58/60). One under-escalation left, and it is that shared one.

Two more findings:

  • Asking the small model six times with the answer options shuffled lifted its tier accuracy from 85% to 90%.
  • A 4B router was faster, but sent 5 hard tasks to the local model. We kept the 9B.

In live use, a routing decision takes a median of about 1.5 seconds, fully on local hardware. The expensive model only gets the work that needs it.

A 4B model that rates agent changes, built in the open

Can a 4-billion-parameter model decide which release gates an agent change needs? We wanted to know, so we built it in the open.

Every change to an agent is a release: a prompt edit, a new tool, a model swap, a wider permission, a removed human approval step. Each one needs a risk class and the right gates: regression tests, security review, business approval, a rollback plan, a staged rollout.

So we published two things on Hugging Face:

  1. A dataset of 3,413 synthetic agent change requests, each labelled with a risk class (low, medium, high, critical), the gates it needs and a short rationale.
  2. A small LoRA adapter on Qwen3-4B that reads a change request and returns that classification as JSON.

On the held-out test split of 341 cases:

  • Base model with the full labelling rubric in the prompt: 54.5% risk accuracy, 32% of gate sets exactly right.
  • With the adapter and a one-line prompt: 63.3% risk accuracy, 47.8% of gate sets exactly right.

Better. Not good enough to approve anything on its own.

What it cannot do, and the dataset card says so:

  • It is 100% synthetic, written by a local open model from one rubric. Real tickets are messier.
  • A second model re-checked 100 random labels and disputed 11. That is label noise.
  • The rubric is ours, not an industry standard.

So the adapter is a decision aid. Anything rated high or critical goes to a human.

Why open? Because "trust our classifier" is not an argument. Data, rubric, filters and scores are public, so anyone can test it against their own change records: huggingface.co/NovaeonStudio.

How we run local models, evaluation sets and LoRA fine-tuning for clients who keep data in-house: Private AI Lab at Novaeon Studio.

Maik Buchenau, co-founder of Novaeon Studio