4 WASM threads ran a matting model 2.6–3.0 faster than 1, but only behind COOP and COEP

If you run an ONNX model in the browser with ONNX Runtime Web and never touched your server headers, your WASM backend is probably running on a single thread. Turning on more threads costs two response headers. A benchmark on 2026-11-10, on one Apple M4 Mac with desktop Chromium and ONNX Runtime Web 1.27.0, put a number on the difference: the same video matting model ran 2.6–3.0× faster per frame with 4 WASM threads than with 1.

The model and how each frame was timed

The model is RVM MobileNetV3 in FP16, 7,503,483 bytes, the same file that the compatible tier of the video matting tool in ImgIng https://imging.ai/ loads in a desktop browser (that tool is still Beta). One test frame at three sizes (512×288, 1280×720, 1920×1080) was fed in repeatedly with the recurrent state passed back, the way a video would run. The timed part is inference plus reading the alpha mask back. Steady state means the median of frames 2 to 13. The 4-thread numbers are the median of 3 separate browser processes, with their range in brackets. The 1-thread setup ran once, so treat it as a single sample.

The part I didn't expect is how flat the 4-thread column is. The pixel count goes up 14 times from the smallest input to the largest, and four threads stay between 87 and 92 ms. My reading is that RVM's downsample_ratio shrinks the backbone input to about 512 pixels on the long side, with only the last upsampling and refinement at full resolution, so the heavy part costs the same at every size. That is an inference from how the model is built. I didn't time individual operators. What I can't explain is why one thread climbs from 228 to 279 ms while four threads barely move. My guess is that the full-resolution step splits well across threads, but that is only a guess. I also can't explain why the three 4-thread runs at 512×288 spread from 72 to 105 ms, the widest of any size.

Threads barely help the first frame

Creating the session took 271–278 ms with either thread count. The first frame took 214–269 ms with 4 threads and 293–347 ms with 1, so the whole cold start (session plus first frame) was 492–544 ms against 565–619 ms. That saves only 21 to 98 ms out of about half a second. For a single image, threads won't change much. Video is where it adds up. Using only the steady medians, 300 frames at 1280×720 (10 seconds at 30 fps) is about 75 seconds of inference on one thread and about 26 on four, leaving out decoding and encoding.

Where the threads come from

WASM threads in a browser need SharedArrayBuffer, and browsers only expose it when the page is cross-origin isolated. You get that state by sending a Cross-Origin-Opener-Policy header and a Cross-Origin-Embedder-Policy header with the page. Without them crossOriginIsolated is false and ONNX Runtime Web has no shared memory to start worker threads with, so you end up on the 1-thread row whatever you set. Below is a simplified sketch of the two pieces, not the benchmark page itself. The numbers above come from the full page.

// response headers for the page that runs the model: // Cross-Origin-Opener-Policy: same-origin // Cross-Origin-Embedder-Policy: require-corp ort.env.wasm.numThreads = self.crossOriginIsolated ? Math.min(4, navigator.hardwareConcurrency || 1) : 1; console.log('isolated:', self.crossOriginIsolated, 'threads:', ort.env.wasm.numThreads); 

The headers have a price, which I know from the docs rather than this benchmark. With require-corp, cross-origin images, scripts or model files that don't opt in through CORS or a Cross-Origin-Resource-Policy header get blocked, and same-origin COOP cuts the link to popup windows. The benchmark sidestepped the first problem because the model bytes were already in memory. I also only tried 1 and 4 threads. Where the curve flattens between 2 and 8 threads, I don't know.

Why not skip WASM and use WebGPU

WebGPU steady times on the same frames were 29.4, 35.0 and 43.6 ms, but on this machine the WebGPU output was an empty mask, with no error thrown anywhere. By frame 13 the foreground share at 1280×720 was 0.00% on WebGPU against 5.03% on WASM. The cause wasn't investigated and may be specific to this model, machine or browser. ImgIng runs both of its RVM tiers on WASM only, and its page says this avoids a silent empty mask on some WebGPU devices. For this model, WASM is what produces a usable mask, so threads are the lever you have.

The next slow part wasn't the model

One more step sat outside the table: converting the whole alpha mask from FP16 to a 32-bit float array in plain JavaScript on every frame. Comparing timings with and without it, that loop added roughly 77 to 88 ms per frame at 1920×1080, close to the whole 4-thread inference, and only 6 to 19 ms at 512×288. Treat those as rough figures. As far as I can tell numThreads can't touch it, since it runs outside ONNX Runtime. Moving it to a worker or keeping the mask in FP16 wasn't tested.

If you already ship a WASM model, open the console on your page and type crossOriginIsolated. If it says false, you are on one thread.