MIG vs MPS vs time-slicing: how to isolate multiple tenants on one GPU

Put two clients' inference workloads on the same card and one of two things goes wrong: a noisy neighbour destroys the other's latency, or one tenant's memory allocation evicts the other's model. NVIDIA gives you several ways to share a GPU and they are not interchangeable. Two give you throughput but no isolation, one gives you hardware isolation at a fixed grain, and CUDA 13.4 added memory limits that change the arithmetic again. This is the decision table, the Kubernetes configuration for each, and a worked four-H100 layout you can copy.

Multi-tenant GPU platforms fail in a very specific way. Everything looks fine at 20% utilisation, then client A runs a batch job at 14:00, client B's p99 latency triples, and the service level agreement (SLA) credit eats the margin on both accounts. Worse, the failure is silent on the memory side: on a shared card, whichever container allocates last gets the OOM killer, and it is usually the tenant who did nothing wrong.

NVIDIA's own GPU utilisation comparison lays out the choices: CUDA Streams, Multi-Process Service (MPS), time-slicing, Multi-Instance GPU (MIG) and vGPU. They differ on the thing that matters most in a managed offering: what is actually isolated. This article takes the four mechanisms you can run in Kubernetes today, including the memory-enforced fractioning that landed with CUDA 13.4 (NVIDIA's announcement), and gives you the resource claim for each.

We build and run these platforms for clients from our team in Dubai, and the pattern is always the same: the isolation decision comes first, the YAML second. Teams that get it backwards tend to re-platform a few months in.

The decision table

Read this before anything else. "Isolation" here means: can tenant B observe, degrade or evict tenant A?

Two rows need a caveat. CUDA Streams is not a multi-tenant mechanism at all: it is how one process overlaps its own kernels, so it sits in NVIDIA's table for completeness. vGPU needs NVIDIA licensing and a hypervisor, which is a different procurement conversation.

MIG: the only real hardware boundary

MIG splits the card into GPU Instances, each with a fixed number of SM slices and memory slices. On an A100 or H100 the memory is cut into 8 slices and the SMs into 7, which is why the profile names look odd. From the NVIDIA MIG user guide, on an H100 80 GB:

Each instance gets its own memory controllers, cache and copy engines. A tenant cannot see another tenant's memory and cannot starve another tenant's SMs. That is the whole point.

The costs are real. You cannot split arbitrarily: only the published profiles exist, so a tenant who needs 25 GB lands on 3g.40gb and wastes 15 GB. MIG instances do not do peer-to-peer, so distributed training across instances is a bad idea. And changing the geometry means destroying every instance on that GPU and draining the node first.

MIG only exists on GPUs with compute capability 8.0 or above. Per NVIDIA's supported GPU list: A100, A30, H100, H200, B200 and GB200. The Ada parts do not support it: no MIG on L4, L40, L40S, A40, A10 or A2. If your fleet is L40S inference boxes, you are in the sharing rows of the table, not here.

Set the geometry on the host once:

# Enable MIG mode on GPU 0 and see which placements this board accepts nvidia-smi -i 0 -mig 1 nvidia-smi -i 0 -lgipp # One 3g.40gb, one 2g.20gb, two 1g.10gb: 8 memory slices, 7 SM slices sudo nvidia-smi -i 0 -mig -cgi 3g.40gb,2g.20gb,1g.10gb,1g.10gb -C 

Always run -lgipp first: placements are fixed per board and driver, and an invalid combination is rejected.

On Kubernetes, use the NVIDIA DRA driver. Each tenant gets a ResourceClaim naming the profile:

apiVersion: resource.k8s.io/v1 kind: ResourceClaim metadata: name: tenant-a-inference namespace: tenant-a spec: devices: requests: - name: gpu deviceClassName: gpu.nvidia.com config: - opaque: driver: gpu.nvidia.com parameters: apiVersion: resource.nvidia.com/v1beta1 kind: GpuConfig mig: singleStrategy: 3g.40gb --- apiVersion: v1 kind: Pod metadata: name: tenant-a-inference namespace: tenant-a spec: resourceClaims: - name: gpu resourceClaimName: tenant-a-inference containers: - name: engine image: vllm/vllm-openai:v0.10.2 resources: claims: - name: gpu request: "1" args: ["--model", "/models/tenant-a", "--max-model-len", "8192"] 

The DRA driver MIG guide also advertises each profile as an extended resource such as nvidia.com/mig-3g.40gb, so a claim-less pod can request that in resources.limits. Either form works; the ResourceClaim form is what you want when tenant, profile and policy belong together in one object your admission controller can audit.

MPS: throughput with a leash, not a wall

MPS lets many CUDA processes run concurrently on one device by placing their contexts in a single server process. SMs are time-shared at the instruction level rather than the context level, so you get better utilisation than plain time-slicing and much better tail latency for small kernels. What you do not get is a memory boundary: all clients still draw from one device heap.

What you do get is a leash, and it is worth using. Per the DRA driver MPS guide, the DRA driver runs an MPS control daemon per node and lets you set three limits: activeThreadPercentage (SM share), pinnedDeviceMemory and deviceMemory. The practical ceiling is 48 client processes on Volta and 64 on Ampere.

apiVersion: resource.k8s.io/v1 kind: ResourceClaim metadata: name: tenant-b-inference namespace: tenant-b spec: devices: requests: - name: gpu deviceClassName: gpu.nvidia.com config: - opaque: driver: gpu.nvidia.com parameters: apiVersion: resource.nvidia.com/v1beta1 kind: GpuConfig mps: sharing: strategy: TimeSliced timeSlicing: maxClients: 8 resources: - name: nvidia.com/gpu replicas: 8 limit: activeThreadPercentage: 25 pinnedDeviceMemory: 2Gi deviceMemory: 8Gi 

Read that claim literally: eight concurrent clients, each capped at 25% of the SMs and 8 GB of device memory. A client that tries to allocate 20 GB fails inside its own container rather than evicting a neighbour. That is the difference between MPS and raw time-slicing, and it is the reason many teams make MPS the default for their own internal inference pools.

There is a second constraint the DRA guide states plainly: MPS cannot share a device that is already being shared by time-slicing or MIG, and the device's compute mode must be EXCLUSIVE_PROCESS because the control daemon's context owns the GPU. If your node image sets DEFAULT compute mode, MPS will not come up. Set it once at provisioning:

nvidia-smi -i 0 -c EXCLUSIVE_PROCESS 

Time-slicing: cheap, simple, and unsafe on memory

Time-slicing is the default many clusters fall into. Every container sees a whole GPU and gets 1/N of it in turn. Nothing is isolated. Its real virtue is that the DRA driver or device plugin can advertise a whole card as N identical replicas with a two-line change, which is ideal for dev clusters and for homogeneous batch jobs that are all yours.

With the DRA driver (time-slicing guide):

apiVersion: resource.k8s.io/v1 kind: ResourceClaim metadata: name: batch-embed namespace: tenant-e spec: devices: requests: - name: gpu deviceClassName: gpu.nvidia.com config: - opaque: driver: gpu.nvidia.com parameters: apiVersion: resource.nvidia.com/v1beta1 kind: GpuConfig sharing: strategy: TimeSliced timeSlicing: resources: - name: nvidia.com/gpu replicas: 4 

If you are still on the NVIDIA device plugin or GPU Operator rather than DRA, the same idea lives in a ConfigMap that the device plugin reads at startup:

apiVersion: v1 kind: ConfigMap metadata: name: time-slicing-config namespace: gpu-operator data: any: |- version: v1 sharing: timeSlicing: resources: - name: nvidia.com/gpu replicas: 4 

The memory trap is this: replicas: 4 divides the SMs' time, never the memory. Four pods on an 80 GB H100 can still collectively allocate 80 GB, and the one that overflows takes an OOM. If you cannot accept that, do not use this row of the table.

Memory-enforced fractioning: the new middle ground

CUDA 13.4 closed exactly that hole. As described in NVIDIA's announcement, a container can now take a fractional GPU plus a memory request and limit, and the CUDA driver enforces the limit per container rather than relying on the last allocator to lose. It needs driver support for CUDA 13.4 and a recent NVIDIA Container Toolkit.

apiVersion: v1 kind: Pod metadata: name: tenant-c-inference namespace: tenant-c spec: containers: - name: engine image: vllm/vllm-openai:v0.10.2 resources: limits: nvidia.com/gpu: 0.5 nvidia.com/gpu.memory: 6Gi nvidia.com/gpu.memory.limit: 8Gi 

There are two strategies and the choice matters. sm-sharing is the native MPS time-sharing mode: concurrent kernels, no memory enforcement. time-slicing is the classic context time-slice, and that is the one where the memory limit is enforced, because each container owns its own CUDA context. The two are mutually exclusive on a device and therefore need separate resource names and device classes. This mechanism is not a security boundary either, but it does remove the failure mode that causes most multi-tenant incidents.

Worked example: eight clients, four H100s

Take a managed AI provider in Dubai running eight client organisations on four H100 80 GB cards. Four of those clients run production inference (a 7B to 13B class model each, max sequence 8192); the other four consume a shared retrieval-augmented generation (RAG) embedding pool. One nightly fine-tune and one internal dev environment round out the node.

Step 1: classify. Production inference carries an SLA and untrusted data, so it is MIG. The embedding pool is ours, homogeneous and latency-tolerant, so it shares. The fine-tune needs a whole card's memory bandwidth. Dev is best-effort.

Step 2: pick the geometry. Cards 0 and 1 each run 3g.40gb, 2g.20gb, 1g.10gb, 1g.10gb: 4 + 2 + 1 + 1 = 8 memory slices and 3 + 2 + 1 + 1 = 7 SM slices. That is four production clients per card with hard boundaries, and the two cards mirror each other as replica zones.

Step 3: place the shared work. Card 2 carries the embedding pool as seven 1g.10gb instances. Card 3 stays whole as 7g.80gb for the nightly fine-tune, and the dev environment bursts onto spare capacity around it.

Step 4: compare with the naive layout. Without MIG, giving each of those seven workloads its own guaranteed capacity costs seven cards. With MIG you need four, roughly 43% less hardware for the same tenancy, before you count power, floor space in a UAE data centre and the operational cost of running more nodes. If your pricing is per tenant per month, this is where the margin comes from.

Step 5: verify. nvidia-smi -L shows the instances; DCGM gives you per-instance SM and memory utilisation; label the metrics by tenant so you can answer "who caused this" from a Prometheus graph rather than a support ticket.

nvidia-smi -L # GPU 0: NVIDIA H100 80GB # MIG 3g.40gb GPU ID 0 (UUID: ... ) # MIG 2g.20gb GPU ID 1 # MIG 1g.10gb GPU ID 2 # MIG 1g.10gb GPU ID 3 

Two adjustments worth making on day one. Reserve the largest profile for the tenant with the biggest KV cache rather than the biggest parameter count, because KV cache is what fills the memory. And keep one 1g.10gb unallocated per node as a canary: deploy a probe pod there on every node upgrade and you catch driver regressions before tenants do.

The tenant boundary is not only hardware

Isolation is a contract question as much as a technical one, and in the GCC that contract usually names a data residency commitment.

• If your agreement says "dedicated compute", only MIG delivers it below the driver. MPS and time-slicing share one memory heap and one driver surface; the boundary is software. Say so in your own documentation before a client's auditor asks.
• MPS runs one control daemon per node, and the DRA driver runs it for you. Tenants on one node therefore share that daemon. Do not pool mutually untrusting clients in one MPS claim; give each tenant its own claim with its own deviceMemory ceiling, or move them to MIG.
• Neither MPS nor time-slicing is a VM. Host-side concerns (kernel driver version, DCGM, node SSH, image provenance) are shared by every tenant on the node. If a client needs a VM boundary, use vGPU or pass a MIG instance through to their own VM.
• Record which MIG instance served which client's data. When a regulator in the UAE or Saudi Arabia asks how data is separated, an inventory of instance UUIDs to tenant IDs is a much better answer than a diagram.
• Keep the per-tenant limits in the claim object, not in a wiki, so the limit is enforceable by admission control and auditable in the cluster state.

What to do next

Work through these in order and you will have a defensible design in a day.

1. Inventory each workload's memory high-water mark and latency target. Everything else follows from those two numbers.
2. Check your hardware against the MIG support list. If the fleet is Ada (L4, L40S), skip to memory-enforced fractioning or MPS and negotiate your SLAs accordingly.
3. Choose the isolation row first, then the profile. Write the profile into the tenant's contract so procurement and platform agree.
4. Implement it as ResourceClaim objects per tenant on the NVIDIA DRA driver, so limits live in the cluster and survive an engineer leaving.
5. Set up DCGM metrics labelled by tenant before you onboard the first client. Retrofitting attribution is miserable.
6. Load test with two tenants deliberately fighting for the GPU, and check that the SLA tenant's p99 does not move. That test is the acceptance criterion.

This is the core of a GPU-as-a-service platform: hardware partitioning where money and contracts demand it, sharing where throughput wins, and an explicit answer for every tenant about which one they got. If you are designing that platform, our AI infrastructure work covers the GPU and Kubernetes layer end to end, and Kuvryn AI is the productised version of the pattern above: agents, models and GPU capacity for your teams, each in its own space, on your own servers. We are based in Dubai, and our own team does the design, deployment and support.

Originally published on Azrty.