CosyVoice on Mac: CosyVoice 2 and 3 Setup Guide

Run CosyVoice 2 or 3 on an Apple Silicon Mac: what each version changed, the official install steps, the macOS GPU gaps, and when a shipped app fits better.

Short answer: CosyVoice is an open source text to speech family from researchers at Alibaba Group, built around a language model plus a flow matching decoder, with zero-shot voice cloning from a reference clip. Murmur does not run CosyVoice. On a Mac you run it yourself from the official repository, and the published install path assumes Linux with Python 3.10, so plan on CPU inference or community patches before you get Apple GPU acceleration.

That distinction matters because search results mix three different things: the research papers, the GitHub repository, and the hosted demos. The papers describe model behavior. The repository describes one install recipe. Neither one promises a supported macOS build. This guide sticks to what the project's own README, model cards, papers, and issue tracker say, and it marks the places where Mac users are on their own.

What CosyVoice is, and why Mac support is a separate question

The project now lives under the QwenAudio/CosyVoice repository (the older FunAudioLLM links redirect there) and is licensed Apache-2.0. The architecture splits the job in two: a text to speech language model predicts supervised semantic speech tokens, then a conditional flow matching model turns those tokens into a Mel spectrogram for a given speaker, and a vocoder produces the waveform. The CosyVoice 2 paper describes a gradual semantic decoding procedure and says the design supports both streaming and non-streaming synthesis in one model.

The app ships a different catalog: Kokoro (bundled), Chatterbox Turbo, Chatterbox Multilingual, Qwen3-TTS Base, Voice Design, SparkTTS, Fish Audio S2 Pro, Breeze TTS 2, and OmniVoice. CosyVoice is not among them, and no setting in the app will make it appear. If you want CosyVoice behavior specifically, you need the Python project. If you want a Mac app that handles scripts, speakers, takes, and export without a terminal, compare the shipped models first in the local TTS model guide.

One more piece of small print: the README ends with a disclaimer that the content is for academic purposes and demonstrates technical capabilities. The repository license is still Apache-2.0, and both Hugging Face model cards list apache-2.0, but read the license and the card yourself before you build a commercial product on the weights.

CosyVoice 1, 2, and 3: what the official sources say changed

Version	Checkpoint named in the sources	What the paper or README emphasizes
CosyVoice 1	CosyVoice-300M, 300M parameters	Supervised semantic tokens instead of unsupervised tokens, an LLM for text to token generation, and conditional flow matching for token to speech (arXiv 2407.05407).
CosyVoice 2	CosyVoice2-0.5B, 0.5B parameters	Unified streaming and non-streaming synthesis, finite scalar quantization, Qwen2.5-0.5B as the language model backbone, 25 Hz speech tokens, 24 kHz output (arXiv 2412.10117).
CosyVoice 3	Fun-CosyVoice3-0.5B-2512, 0.5B parameters	A new tokenizer trained with supervised multi-task learning, a differentiable reward post-training method, 9 languages, 18 or more Chinese dialects, and data scaled to one million hours (arXiv 2505.17589, README).

The CosyVoice 3 paper says the language model was scaled from 0.5B to 1.5B parameters and the flow matching model from 100M to 300M, and that this larger capacity improved its multilingual benchmark. The checkpoint the README actually tells you to download is Fun-CosyVoice3-0.5B-2512, which is 0.5B. Both facts can be true at once: the paper reports experiments at two sizes, and the released default is the smaller one. If you see a claim that CosyVoice 3 on your Mac is a 1.5B model, check which weights that person actually loaded.

The version jump also changes the language story. The CosyVoice 2 paper reports training data in Chinese, English, Japanese, and Korean, and its limitations section says the model supports only a limited number of languages. The CosyVoice 3 README lists nine languages (Chinese, English, Japanese, Korean, German, Spanish, French, Italian, Russian) plus 18 or more Chinese dialects and accents, and both the README and the paper describe cross-lingual zero-shot cloning. The paper's data section counts 19 Chinese accents or dialects while the abstract says 18, so treat the dialect count as approximate and check the current card.

Zero-shot cloning: a reference clip plus its transcript

Zero-shot here means the model is not fine-tuned on your speaker. You supply a short reference recording and the matching text, and the model imitates the timbre, prosody, and style of that reference. In the CosyVoice 2 model card, the zero-shot call takes the text to synthesize, the prompt text, and the prompt wav path. A separate cross-lingual call takes the target text and the same prompt wav, and an instruct call takes a natural language instruction as a second argument after the text.

cosyvoice = AutoModel(model_dir='pretrained_models/CosyVoice2-0.5B')
for i, j in enumerate(cosyvoice.inference_zero_shot('收到好友从远方寄来的生日礼物，那份意外的惊喜与深深的祝福让我心中充满了甜蜜的快乐，笑容如花儿般绽放。', '希望你以后能够做的比我还好呦。', './asset/zero_shot_prompt.wav')):
torchaudio.save('zero_shot_{}.wav'.format(i), j['tts_speech'], cosyvoice.sample_rate)

The reference transcript is not decoration. CosyVoice 2's in-context learning mode uses prompt text and prompt speech tokens together, and the card's own examples pass the prompt text explicitly. The CosyVoice 3 card adds an instruction prefix such as "You are a helpful assistant." before the prompt text. If your reference clip is noisy, clipped, or in a different language from its transcript, the clone will reflect that.

Only clone voices you own or have written permission to use. A local model removes the upload step, not the consent requirement. For a broader workflow around reference recordings and takes, see the voice cloning guide.

The streaming latency claim, with its context

The CosyVoice 3 README says the system supports text-in and audio-out streaming and "achieves latency as low as 150ms while maintaining high-quality audio output." That is the project's claim about its own streaming path. It is not a macOS benchmark, it does not say which hardware produced the figure, and it does not say how much text must arrive before the first audio chunk. Treat 150 ms as a best-case target to reproduce, not a number you should expect on an M1 laptop running CPU inference.

The CosyVoice 2 paper is more explicit about what the latency depends on. Its first-package latency is modeled as the chunk size M multiplied by the per-token time for the language model, the flow matching model, and the vocoder, summed together. Change the chunk size or the hardware and the number changes. The paper also reports that its streaming mode was nearly lossless compared with offline synthesis in typical tests, with a slight content consistency drop in hard cases. Those are paper results, not a measurement of your machine.

If you care about streaming behavior, measure it. The streaming latency measurement guide covers first-chunk timing and real-time factor on a Mac. A file-based editor is a different product category from a live streaming endpoint, so do not read this section as a claim about any desktop app.

Installing on macOS: the official steps and the gaps

The README's install section is short and Linux-oriented. These are its exact commands, minus the mirror flags the project adds for users in China:

git clone --recursive https://github.com/FunAudioLLM/CosyVoice.git
cd CosyVoice
git submodule update --init --recursive
conda create -n cosyvoice -y python=3.10
conda activate cosyvoice
pip install -r requirements.txt

The submodule step is not optional. The FAQ says a missing Matcha-TTS directory produces a ModuleNotFoundError, and that you should run git submodule update --init --recursive if third_party/Matcha-TTS is absent. The requirements file pins torch 2.3.1 and torchaudio 2.3.1, and it installs onnxruntime for darwin, so a macOS CPU install is plausible even though the README never walks through one.

The next step is model download. The README recommends the 3.0 checkpoint for better performance and shows both ModelScope and Hugging Face calls. For Hugging Face, the lines are:

from huggingface_hub import snapshot_download
snapshot_download('FunAudioLLM/Fun-CosyVoice3-0.5B-2512', local_dir='pretrained_models/Fun-CosyVoice3-0.5B')
snapshot_download('FunAudioLLM/CosyVoice2-0.5B', local_dir='pretrained_models/CosyVoice2-0.5B')

After that the README says to follow example.py, or to start the web UI with python3 webui.py --port 50000 --model_dir pretrained_models/CosyVoice-300M. The README also mentions an optional ttsfrd resource for better text normalization and says that if you skip it, wetext is used by default. Skip it on a Mac. The wheel it installs is named ttsfrd-0.4.2-cp310-cp310-linux_x86_64.whl, and issue #1011 shows that command failing on an M4 Mac mini.

Two more macOS details from the README: the sox troubleshooting commands are apt-get for Ubuntu and yum for CentOS, with no Homebrew line, and the deployment section builds a Docker image and runs it with --runtime=nvidia. The TensorRT-LLM acceleration path and the Docker examples are written for NVIDIA hardware. None of that blocks CPU inference, but none of it is a Mac install guide.

Apple GPU acceleration is not in the default path

Issue #1011 is the clearest public record of the Mac experience. A user on an M4 Mac mini reports the CPU at 100 percent while the GPU sits idle, then quotes the device line in cosyvoice/cli/model.py: cuda if torch.cuda.is_available(), otherwise cpu. There is no MPS branch. Another commenter says support requires changing the code, and later comments show a patch that checks torch.backends.mps.is_available() before falling back.

Forcing MPS is not automatically a win. One commenter hits NotImplementedError for aten::unfold_backward on the MPS device, with the standard advice to set PYTORCH_ENABLE_MPS_FALLBACK=1 so that unsupported operations fall back to CPU. That operator is a gradient operation, so the thread mixes training and inference reports, and other commenters in it say MPS worked after upgrading to a newer PyTorch. Others report that newer PyTorch and newer macOS are needed, and that results vary by version. In issue #134 (July 2024, before CosyVoice 2 and 3), a project collaborator writes that an earlier MPS pull request was slower than CPU on Mac and that it was maybe not mature yet. These are user reports and maintainer comments, not a supported configuration.

The practical sequence is therefore: get CPU inference working first with the 0.5B checkpoint, confirm that a short line synthesizes and saves, then try an MPS patch on a copy of the repository and compare wall-clock time against CPU. Do not start by patching a fresh clone before you know the baseline works. And do not install torch nightly over the pinned 2.3.1 in the project environment without a separate environment to fall back to.

Should you run it yourself?

Your situation	Self-hosted CosyVoice	A desktop app route
You need Chinese, English, and Chinese dialect coverage with cross-lingual cloning	Closest fit. The 3.0 README lists nine languages and 18 or more dialects.	No CosyVoice. Check the app's model list and public samples before assuming an equivalent voice.
You want a zero-shot clone from a short reference clip	Supported by the official inference calls, with a reference transcript for the 2.0 path.	Several shipped models do reference cloning, including Chatterbox Multilingual and OmniVoice. Confirm the current list in the app.
You want a live microphone or agent response	The project targets bi-streaming, but you still have to build and serve it. The 150 ms figure is the project's own claim.	Not a live streaming product. It generates and edits files.
You do not want a terminal, Conda, or Python	Poor fit. Install and troubleshooting are your job.	Fits. Native macOS app for Apple Silicon and Intel Macs, macOS 15 or newer.
You need a Linux server or Docker deployment	Fits. The README ships Docker, gRPC, and FastAPI paths.	Not a fit. It is a Mac app.
You need Windows	Possible in principle, but outside this article's scope and not covered by the Mac steps here.	Not available. macOS only.

The honest summary: CosyVoice is worth the setup if you specifically need its multilingual and dialect behavior, or you want to study or modify the model. It is the wrong tool if you want a finished desktop editor, if you need Windows, or if you need a supported Apple GPU path today. The local studio versus DIY comparison covers that trade in more detail.

What can go wrong

The install assumes Linux. The sox fixes are apt-get and yum, and the optional ttsfrd wheel is linux_x86_64 only. On macOS, skip ttsfrd and accept the wetext fallback.
The default device selection is CUDA or CPU. On a Mac you get CPU until you patch the device lines, which is why Activity Monitor can show a busy CPU and an idle GPU.
MPS can fail on specific operators. The unfold_backward error in issue #1011 is one example, and the suggested fallback variable makes those operations run on CPU, while other commenters there report MPS working on newer PyTorch.
Speed is not guaranteed by the patch. A collaborator in issue #134 reports that an earlier MPS path was slower than CPU, so measure before you commit.
Version drift breaks installs. The requirements pin torch 2.3.1, while MPS comments ask for much newer PyTorch. Keep separate environments.
The GPU deployment paths are NVIDIA-oriented. vLLM, TensorRT-LLM, and the Docker examples are not macOS acceleration.
The papers list model limits. CosyVoice 2 supports a limited language set, neither paper controls timbre through text instructions, and singing is described as a weak case.
Checkpoints take disk space. You may end up with 300M, 0.5B, and other weights side by side, plus the repository and its submodules. Plan the folder before downloading.
The README's academic-purposes disclaimer sits next to an Apache-2.0 license. Read both, plus the model card, before commercial use.

License, consent, and commercial use

The repository LICENSE is Apache License 2.0, and the CosyVoice2-0.5B and Fun-CosyVoice3-0.5B-2512 model cards both list apache-2.0. Apache-2.0 permits commercial use and modification, with notice and attribution conditions. It does not settle the rights in your reference audio, and it does not override the README disclaimer. The open source model license checklist walks through the questions to ask before shipping.

Voice cloning adds a second layer. Use recordings you made, recordings you licensed, or recordings you have written permission to imitate. Do not clone a public figure or a private person to imply endorsement, and do not use a cloned voice for fraud or harassment. If a client supplies the reference, get the permission in writing and keep it with the project files.

Frequently asked questions

Does Murmur support CosyVoice?

No. The app ships Kokoro, Chatterbox Turbo, Chatterbox Multilingual, Qwen3-TTS Base, Voice Design, SparkTTS, Fish Audio S2 Pro, Breeze TTS 2, and OmniVoice. CosyVoice is not in that catalog. To use CosyVoice you run the official Python project yourself.

Can I run CosyVoice 3 on an Apple Silicon Mac?

You can try. The README's install steps are written for Linux with Python 3.10, and the code defaults to CUDA or CPU, so a Mac runs on CPU unless you patch it. Users in the project issue tracker report working CPU installs and community MPS patches, but the project does not document macOS as a supported target.

Is the 150 ms latency figure realistic on a Mac?

Treat it as the project's claim for its streaming path, not a Mac benchmark. The CosyVoice 2 paper shows that latency depends on chunk size and the per-token time of the language model, flow matching model, and vocoder. Measure first-chunk time on your own hardware before designing around a number.

Which CosyVoice checkpoint should I download first?

Start with CosyVoice2-0.5B if you want the longest-documented path, since its card has zero-shot, cross-lingual, and instruct examples, but note that the README itself recommends Fun-CosyVoice3-0.5B for better performance and that card has the same kinds of examples. The README recommends Fun-CosyVoice3-0.5B for better performance and lists nine languages, but its card's examples use the newer instruction prefix. Either way, confirm the local folder name matches the model_dir you pass.

Can I use CosyVoice output commercially?

The repository and both model cards state Apache-2.0, which allows commercial use subject to its notice and attribution terms. The README also carries an academic-purposes disclaimer, so read the license, the card, and the disclaimer together, and confirm that you have rights to the reference voice. This is not legal advice.

Sources