Infographic cover

📅 November 20, 2026 • Hugging Face

NVIDIA has released Nemotron-Labs-Sigma, an open-weight family of continuous diffusion language models. The release ships 14 checkpoints in two sizes, 3B and 8B: base, instruct and few-step distilled variants (NFE 2, 4, 8 and 16), plus a self-speculative variant per size. All weights are published on Hugging Face under the NVIDIA Nemotron Open Model License, which permits commercial use, so teams can download, fine-tune and serve the models in their own environments.

Continuous diffusion differs from both familiar approaches. An autoregressive model writes text token by token, always moving from left to right. Discrete diffusion models such as LLaDA or Dream diffuse over discrete token IDs. Sigma instead denoises token representations in a learned 16-dimensional embedding space and generates text blockwise, 64 tokens per block: each block passes through several denoising steps before it is emitted, and the model then moves on to the next block. The training lineage is unusual as well — the models were warm-started from an autoregressive checkpoint trained on roughly 1 trillion tokens, then continued with about 0.3 trillion tokens of diffusion training.

The figures below are per NVIDIA's own internal evaluations and are not independently verified. Sigma-8B base averages 71.16 on NVIDIA's 6-task suite, against 70.22 for NLD-8B, 58.74 for Dream-7B, 41.78 for LLaDA-8B and 68.91 for the autoregressive Qwen3-8B. The 8B instruct variant reaches 91.96 on GSM8K. The known weak spot is AIME25 at 17.00, so competition-level mathematics remains clearly behind the family's other results, a useful reminder that these numbers are self-reported.

On speed, NVIDIA reports about 120-140 tokens per second on a single H100 in bfloat16. The self-speculative 8B variant decodes up to 3.46x faster than the autoregressive average on GB200 with SGLang, with a peak of 4.20x per NVIDIA. The context window is 4,096 tokens, extended to 8,192 tokens for the 8B instruct variant.

Key figures

IMPACT: diffusion language models generate whole blocks in parallel instead of moving token by token from left to right. The distilled variants trade a few extra compute steps for fewer network round-trips, which matters for self-hosted inference stacks where latency budgets are tight. Since NVIDIA does not host a first-party API, everything runs on your own infrastructure via vLLM, SGLang or Docker Model Runner, with commercial use allowed under the license. For teams that already operate their own GPU fleet, that removes a dependency on external endpoints entirely.

Sources:
huggingface.co — marktechpost.com — the-decoder.com — arxiv.org

IMPACT checklist

Continue the conversation: https://t.me/AIAutomationArchitecture · Bluesky

AI Automation