How do you fine-tune Mistral 7B on your own data?

To fine-tune Mistral 7B on your own data, train a LoRA adapter on the 16-bit model on one 48 GB GPU, then check it against held-out examples. On Nodus that is a single TrainingJob on the managed nodus/sft runtime with a hard dollar cap. The RTX A6000 it fits on is listed at $0.333/hr on the Nodus pricing page. You do not need an H100 for this model, and the arithmetic for when a faster GPU would save money is short.

How much GPU memory does Mistral 7B need for fine-tuning?

LoRA on the 16-bit base needs about 14.5 GB just to hold the frozen weights, so plan on a 48 GB card. We got that from the Mistral-7B-Instruct-v0.3 repository on Hugging Face, which lists 7,248,023,552 parameters stored in bf16. At 2 bytes per parameter that is 14.5 GB before any activations, adapter state or CUDA overhead.

The other two methods sit far apart:

Parameter math from the Hugging Face model files, checked November 4, 2026. Activations come on top and grow with sequence length and batch size.

A full fine-tune of Mistral 7B does not fit on one 80 GB GPU; LoRA fits on one 48 GB card with room to spare.

The adapter itself is tiny. With the nodus/sft defaults (rank 16 on all linear layers), Mistral's 32 layers each get 1,310,720 adapter parameters across the attention and MLP projections. That is 41,943,040 trainable parameters in total, or 0.58% of the model. Even at 16 bytes per trainable parameter for the weights, gradients and Adam state, the adapter needs 0.67 GB.

A full fine-tune is a different job. At 116.0 GB it needs several GPUs with sharding. For most "teach it our format and tone" projects, a LoRA adapter gets you there for a fraction of the cost. The LoRA vs QLoRA guide covers when the 4-bit route is worth it.

What should your training data look like?

Your data should be one JSON object per line with a prompt and an answer. Write each answer exactly the way you want the model to reply. The nodus/sft runtime reads prompt/completion JSONL or plain text JSONL, and you map your own field names to prompt and completion in the manifest.

{"question": "Where do I change my billing email?", "answer": "Go to Settings, then Billing, and edit the email field under Invoices."} 

Start from Mistral-7B-Instruct-v0.3 when your rows are instructions and answers, because it already follows instructions and you only teach it your domain. Start from the base Mistral-7B-v0.3 when your data is raw text you want the model to absorb. Both are Apache 2.0 and were not gated when we checked the base and instruct repositories.

Consistency matters more than volume. The model learns whatever your answers have in common, including their mistakes. If the same prompt appears with several answers, keep those rows together when you split, or your validation loss will look better than the model is. The repeated-prompts guide shows how that leak happens.

How do you run the fine-tune on Nodus?

You upload the JSONL to a Volume, write one TrainingJob manifest, dry-run it to see the price, then apply it. The commands below come from the training guide, the Volumes guide and the nodus/sft reference.

$ pip install nodus-compute $ nodus login $ nodus create volume support-chats --size 10Gi $ nodus volume put support-chats ./chats.jsonl /chats.jsonl 

The manifest pins Mistral to an exact commit, which Nodus imports once into a read-only Volume so later runs never pay GPU time to download weights.

apiVersion: nodus.dev/v1beta1 kind: TrainingJob metadata: name: mistral-support-sft spec: runtime: nodus/sft model: uri: hf://mistralai/Mistral-7B-Instruct-v0.3@c170c708c41dac9275d15a8fff4eca08d52bab71 data: volume: support-chats path: chats.jsonl format: JSONL columns: {prompt: question, completion: answer} validationFraction: "0.1" parameters: method: LoRA completionOnlyLoss: true steps: 2000 evalSteps: 100 lora: {r: 16, alpha: 32} validation: {patience: 3, requireImprovement: true} resources: gpu: {type: [A6000, L40S, A100-80GB], count: 1} maxCostUSD: "20.00" 

Two lines matter more than they look. The resources block is there because nodus/sft has no Mistral preset, so without it the run falls back to the runtime default, which includes a 24 GB RTX 4090. That card leaves under 10 GB for everything except the frozen weights. The validation block picks the checkpoint with the lowest validation loss and fails the evaluation if no checkpoint beats the starting model.

$ nodus apply -f sft.yaml --dry-run -o yaml $ nodus apply -f sft.yaml 

The dry run prices the job as steps times the runtime's measured seconds per step, before anything is charged. If the run hits maxCostUSD, it is suspended rather than failed, and raising the cap resumes it from the last checkpoint. The runtime saves a recovery checkpoint every 200 steps by default, so a lost machine costs you at most those steps.

When it finishes, download the adapter:

$ nodus cp tj/mistral-support-sft:outputs/adapter/adapter_model.safetensors ./adapter_model.safetensors 

Which GPU is cheapest for a Mistral 7B LoRA run?

The cheapest GPU is the one with the lowest price per step, not per hour, so a faster card only wins if its speedup beats its price ratio. Here are the cards that hold a 16-bit Mistral 7B with room for activations, plus the RTX 4090 for QLoRA:

Prices checked November 4, 2026 on the Nodus pricing page. Break-even = GPU price / $0.333.

An H100 hour costs 7.81 RTX A6000 hours, so it needs to be that much faster to break even.

The nodus/sft runtime publishes the step times its dry-run estimates use. They are 0.45 s on an H100, 0.7 s on an A100 80 GB, 1.0 s on an L40S and 1.3 s on an RTX A6000.

Those are the runtime's own measured figures, not a Mistral-specific benchmark, but they show the shape. The H100 is 2.89x faster than the A6000 there, well short of the 7.81x it would need. The A100 80 GB is 1.86x faster against a 3.27x break-even. The L40S manages 1.30x against 2.38x.

On the runtime's own step times, none of the faster cards is fast enough to beat the RTX A6000 on cost per step.

So the rule for a single-GPU Mistral 7B LoRA job is simple. Use a 48 GB card unless your deadline is worth paying 2 to 3 times more for a faster finish. Listing several types in resources.gpu.type, as the manifest does, lets the Nodus scheduler pick the one with the lowest expected cost to finish. Use QLoRA on a 24 GB card only if the 4-bit run takes less than 1.15x as long as LoRA on the A6000. The price gap between them is only $0.043/hr.

How do you know the fine-tune actually worked?

You know it worked when the tuned model beats the base model on examples it never saw, scored the way you will use it. Validation loss picks the best checkpoint, but the training guide is explicit that a separate untouched test set must establish task quality.

Hold back 50 to 200 real prompts before you start. Run them through Mistral-7B-Instruct-v0.3 and through your adapter, then grade both with the same rubric or checker. Also run a few general prompts that have nothing to do with your data. If those get worse, you are seeing the trade-off covered in our catastrophic forgetting guide, and a lower learning rate or fewer steps usually helps.

What if you want to write your own training script?

If you prefer Unsloth, Axolotl or your own TRL script, the same memory math holds and you can run the script as a regular GPU job instead of a TrainingJob. Our Unsloth on a cloud GPU walkthrough shows that path end to end. You give up the managed presets, validation gate and dry-run estimate, but you keep the dollar cap, and checkpoints your script writes to the state directory survive a lost machine.

FAQ

Can you fine-tune Mistral 7B on a 16 GB GPU?
Only with QLoRA and short sequences. A 16 GB card holds the 3.6 GB of 4-bit weights with room for activations, but the 14.5 GB bf16 base leaves almost nothing, so plain LoRA is out.

Should you fine-tune Mistral 7B v0.3 or an older version?
Use v0.3 for new projects. Its model card lists an extended 32,768-token vocabulary, the v3 tokenizer and function calling. An adapter is tied to the base it was trained on, so pick the version before you train.

Can you merge the LoRA adapter back into Mistral 7B?
Yes. Load the base model and the adapter with Hugging Face PEFT and call merge_and_unload() to get a single set of weights for serving. Keep the unmerged adapter too: its 41.9 million parameters take under 0.2 GB even in fp32, so it is easy to version.

How long does a Mistral 7B LoRA fine-tune take on Nodus?
Run nodus apply -f sft.yaml --dry-run -o yaml and read status.estimate, which multiplies your step count by the runtime's measured seconds per step on the slowest GPU type you allowed. Set spec.expectedDuration if you have timed your own data and want the estimate to use that instead.

Originally published on the Nodus blog.