Dubbing a video into another language is easy when one person talks. It gets hard the moment there are two speakers: a single synthetic voice for everyone makes interviews and podcasts unwatchable, and the translated sentences rarely fit the original timing.

This tutorial dubs a real two-speaker clip from English into Spanish with an open-source tool, miragea-dub (MIT). Each speaker keeps their own cloned voice, the lines are fitted into the original timing, and you get a dubbed MP4, an audio track and subtitles.

Banner for miragea-dub: speaker lanes in English above the same lanes dubbed into Spanish

What is AI video dubbing with a voice per speaker?

AI video dubbing replaces the speech in a video with translated speech. A multi-speaker dub does four jobs: it transcribes the video, separates the speakers, translates each line so it can be spoken in the same time slot, and voices each speaker with a voice that resembles the original. miragea-dub runs those steps from the command line, as an MCP server for AI agents, or as an Agent Skill. Voices come from the Miragea voice API, so no GPU is needed.

Hear the result first

Animated preview of the Spanish dub with burned-in subtitles

Both voices were cloned from the clip itself. Transcribing the Spanish track again returns the translated script almost word for word.

Step 1: install 

pipx install miragea-dub # needs ffmpeg export MIRAGEA_API_KEY=... # key from https://miragea.ai/workspace/api miragea-dub doctor miragea-dub demo # optional: offline run with test tones

Step 2: transcribe and find the speakers 

miragea-dub prepare two-voices.mp4 --to es --speakers 2

This writes two-voices.dub/dub.json. Miragea's speech recognition returns subtitle cues cut at a fixed width, sometimes mid-word, so miragea-dub regroups them into whole sentences using the punctuated full text. A built-in, GPU-free clustering step then labels who speaks (S1, S2). On the demo clip all six lines were assigned to the right speaker.

Step 3: translate with your AI agent (or any LLM)

Each line has a length budget. Fill the translation fields yourself, let Claude or Cursor do it through the MCP server, or run miragea-dub translate --translator openai with any OpenAI-compatible endpoint. Short lines matter: the dub has to fit.

Step 4: clone, voice, fit, mix 

miragea-dub render two-voices.dub/dub.json --model expressive

Terminal capture of the real run: 6 lines, 2 speakers, 4 sped up, 0 trimmed

For each speaker miragea-dub cuts a 8-20 second reference from the video, hosts it on a temporary link and synthesizes every line with that voice. Lines are sped up slightly (limit 1.35x) to fit; anything that still does not fit is listed in lines_to_shorten.

Timeline: outlined bars are the original slots, solid bars the dubbed lines after fitting

Step 5: publish

You get two-voices.es.mp4 (dub + original audio + subtitles), two-voices.es.m4a for YouTube's additional-audio feature, SRT/VTT subtitles and a JSON report.

Use it from Claude, Cursor or Codex 

{ "mcpServers": { "miragea-dub": { "command": "uvx", "args": ["--from", "miragea-dub", "miragea-dub-mcp"], "env": { "MIRAGEA_API_KEY": "your-key" } } } }

Or install the video-dubbing skill from miragea-skills. The agent does the translation, so no second LLM key is needed.

FAQ

Can I dub a two-person interview and keep both voices? Yes, each speaker is cloned from their own lines.

Is there an open-source alternative to hosted dubbing apps? The pipeline is MIT; voices need a Miragea API key. Bigger GUI projects such as VideoLingo and pyvideotrans exist too, this one is the small scriptable one for agents.

Only clone voices you have permission to clone.

Source: codeberg.org/miragea-voice/miragea-dub · PyPI