VAD, speech recognition and an 82M-parameter TTS now run free inside a browser tab. The interesting question is where that stops. Here is the line, and the cloud stack I would wire up on the other side.
Disclosure: three links below are affiliate links, marked with (af). If you sign up through them, I earn a commission at no extra cost to you. The criteria are the same ones I use when nothing is at stake.
What runs in the tab for free. Silero VAD as ONNX in the browser. Whisper-class ASR dueling live on your microphone. An 82M-parameter TTS (Kokoro) reading text in a tab, weights and all. All of it through transformers.js on WebGPU or WASM, no server, no API key, no bill per user. I keep these instruments up as static pages; open the latency waterfall and watch where a turn actually spends its milliseconds.
Where the tab ends. Three ceilings, in the order I hit them. First, the model: an 8B-class LLM fits in a tab, the model you want for real decisions does not. Second, the platform: a real deployment needs a domain, websocket endpoints, logs, a backend that survives a phone refresh; static hosting does not give you those. Third, the voice: an 82M TTS is impressive and still recognizably synthetic; brand-grade voices are server models, and the reference cascade in the literature measures them seriously.
The stack I would wire up on the other side.
- Host the orchestrator on Railway (af): websockets, a domain and logs from a git push. The latency wins live in the event loop.
- Serve open models through Novita AI (af): per-token API instead of GPUs and vLLM babysitting. OpenAI-shaped endpoint, so swapping providers is a config change.
- Premium voice on ElevenLabs (af): the reference cascade measures 755ms to first audio with per-sentence streaming (arXiv 2603.05413). Premium voices are what you pay for.
Why this split is the honest default. In-browser parts have zero marginal cost per user, keep audio on-device, and ship as a static page that stays up. Cloud parts buy quality ceilings the tab cannot reach yet: cloud speech-to-speech sits around 702ms audio-to-audio in the current literature, and a well-tuned cascade of cheap parts can land close to that. Pay for the ceilings; run everything else in the tab.
