How I gave my coding agent a free, fully local voice for my personal projects, and why I think this is where agentic coding is headed.

🏡 Everything here was built and used on my own side projects, on my personal laptop.

Here's a confession: I've stopped reading most of what my coding agents write.

Not because it's bad. Because there's too much of it. An agent kicks off, reads twelve files, runs the tests, edits four modules, and then hands me a 900-word wall of text with tables, file paths and a bulleted summary of a bulleted summary. By paragraph three my eyes are skimming. By paragraph five I've wandered off to another tab.

That's the quiet problem with agentic coding nobody talks about.

The agents got faster. Our attention didn't.

The bottleneck isn't the model anymore. It's me, scrolling.

So I tried something different. I gave my agents a voice and ears. Now I ask a question out loud, the agent goes off and does the work, and comes back with two spoken sentences: "Tests pass, I changed the retry logic in two files. Want the details on screen?"

💻 It runs entirely on my laptop. 💸 It cost me nothing. I built it in Claude Code, but nothing about it is Claude-specific: any agent that can use MCP tools can get the same thing.

Here's how, and why I think you should try it.

🎙️ Why voice, why now

Three things changed my mind.

1. Agents produce output faster than humans can read it.
A single agentic task can generate more text than a short blog post. Reading all of it is exhausting; skimming it means you miss the one line that mattered. Speech forces the agent to summarise. You can't speak a table. You can't speak a 40-line diff. So the agent has to tell you what actually matters, and put the rest on screen if you want it.

2. Talking keeps you in the loop without keeping you glued to the screen.
When I'm in voice mode I can lean back, look away, think. The agent works between turns and reports back by voice. It feels less like supervising a log file and more like pairing with a friend who's sitting next to you.

3. It's finally free and local.
A year ago this meant cloud speech APIs, API keys, and paying per minute. Today, open models like Whisper (speech-to-text) and Kokoro (text-to-speech) run comfortably on a laptop. No account, no key, no audio leaving your machine.

🔌 The idea: one voice layer, any agent

The trick is that the voice doesn't live inside the agent. It lives in an MCP server, the standard plug-in protocol that Claude Code and a growing list of other coding agents already speak.

One voice layer, any coding agent: you speak, Whisper turns it into text, VoiceMode hands it to your agent, Kokoro speaks the reply. All local, all free.

The MCP server I use is VoiceMode (open source, MIT licensed). It exposes one main tool, converse: speak something, then listen and return what the user said. Any agent that can call MCP tools can now hold a conversation.

That's the whole point: build the voice once, plug it into every agent you use. When the next coding assistant comes along, it gets a voice for free too.

⚡ Setup: about ten minutes

I'm on an Apple Silicon Mac with 16 GB of RAM. Nothing exotic.

1. Install VoiceMode and the local speech services

uvx voice-mode-install voicemode service install whisper # speech-to-text voicemode service install kokoro # text-to-speech

2. Pin it to local only

By default VoiceMode can fall back to a cloud API if it finds an API key. I don't want that:

voicemode config set VOICEMODE_STT_BASE_URLS http://127.0.0.1:2022/v1 voicemode config set VOICEMODE_TTS_BASE_URLS http://127.0.0.1:8880/v1

Now nothing you say leaves the machine.

3. Plug it into Claude Code

One line:

claude mcp add --scope user voicemode -- ~/.local/bin/voicemode

4. Add a /talk command

A tiny custom slash command: a markdown file in ~/.claude/commands/ whose prompt says start a voice loop with the converse tool, keep replies short, stop when I say “stop”. Type /talk, and you’re talking.

That's it.

🧩 Using a different agent?

The same recipe works in any coding agent that supports MCP. The pieces are always the same:

  1. Run the two local speech services (Whisper and Kokoro). They don't care which agent calls them.
  2. Register VoiceMode as an MCP server in your agent's MCP config, pointing at the voicemode binary.
  3. Allow its converse tool, if your agent asks you to approve tools.
  4. Add a reusable prompt (a slash command, custom prompt or rules file, whatever your agent calls it) with the voice-loop rules below.

Check your agent's docs for where its MCP config lives. Some read one global file; others want the server listed per agent or per project.

💬 Making it feel like a conversation, not a robot

Getting audio in and out is the easy part. Making it pleasant took some prompt tuning. The rules that made the biggest difference:

  • One or two sentences, ~25 words. The agent offers to say more instead of explaining everything. This alone fixes the wall-of-text problem.
  • Never read code, paths or tables aloud. Speak the summary; put the detail on screen.
  • A live transcript. Before each turn the agent prints 🎤 You: … and 🔊 Friday: …, so the screen matches what you hear. Handy when the speech recognizer mishears you.
  • A persona. In voice, my agent answers as "Friday": calm, concise, a little dry. Put the persona in the shared prompt and every agent you wire up sounds like the same assistant. Small thing, surprisingly nice.
  • Sound human. Vary the acknowledgements. Nothing kills the vibe like hearing "Great question!" eleven times.

🛑 The safety rule you must not skip

Speech recognition mishears things, especially names, numbers and jargon. "Delete the test branch" and "delete the best branch" are one syllable apart.

So my rule is simple:

  • Read-only questions ("what failed?", "summarise the last change"): the agent just answers.
  • Anything that changes or sends something (file writes, deletes, git commits and pushes): the agent repeats back what it understood in one sentence and waits for a spoken "yes".

👉 Never let a voice command run an action unconfirmed. This one rule is what makes voice safe enough to actually rely on.

🪙 "But doesn't voice burn more tokens?"

I wondered the same, so I measured it from my own session logs.

Across 16 voice exchanges:

  • ~350 output tokens per turn (spoken reply + on-screen transcript + reasoning)
  • ~800 uncached input tokens in total. What you say comes back as a tiny bit of text.
  • The big number, ~110k cached tokens per call, is just the conversation context being re-read, which is exactly the same when you type.

✅ Verdict: a voice turn costs about the same as a typed one. Speech-to-text and text-to-speech run locally, so they're free. The only extra is the transcript line, maybe 50–100 tokens a turn. If anything, voice saves tokens, because short spoken answers replace long written ones.

(Caveat: that's one session, not a controlled typed-vs-spoken comparison. Take it as a strong hint, not a benchmark.)

⏱️ Is it fast enough?

Mostly, yes. The voice gap after the agent finishes thinking is about 2 seconds: ~0.3 s for Whisper to transcribe you, ~1.6 s for Kokoro to start speaking. The real variable is how long the model thinks, which you'd wait for anyway.

One surprise: Kokoro was faster on CPU than on the Mac's GPU. Time to first audio was 0.4 s on CPU versus 2.2 s on GPU, and the GPU never caught up after warm-up. If your local text-to-speech feels slow, try forcing CPU.

And a lesson: when benchmarking TTS, use new text every time. Repeated text gets cached, and my first measurement looked 5× better than reality.

🪤 Gotchas I hit, so you don't

  • Bluetooth headset as the mic → beeps and robot voice. macOS flips the headset into "call mode" when it starts listening. Use the laptop mic for input and keep the headset for output.
  • voice-mode-install --model small broke the Whisper install on my setup. Install Whisper on its own with voicemode service install whisper.
  • Python version mismatch. Kokoro wants Python 3.10; if your machine defaults to newer, rebuild its venv on 3.10.
  • Corrupt model download. If Kokoro won't start, check the model file size. Mine was 50 MB short.
  • Not every agent reads a global MCP config. If the voice tools don't show up, add the server to the per-agent or per-project config.
  • Privacy note: VoiceMode's logs store what you said in plain text. Put them on your cleanup list.

🚫 What I didn't use, and why

I looked around before settling on this:

  • Claude Code's built-in /voice: great for dictation, but input-only and cloud-transcribed. No spoken replies.
  • Other voice MCP servers: either barely maintained, macOS-only and stale, or multiple GBs of models per project.
  • Cloud voice tools: some have nicer live transcripts, but they send your audio to the cloud and some bill separately. Not what I wanted.

VoiceMode was the only option I found that was fully local, actively maintained, and worked across every project and every agent.

🧠 "Is AI making us dumber?"

You've probably seen the term cognitive debt going around, along with cognitive surrender and AI brain fry. The worry is real, and the research is starting to back it up:

  • In a randomized trial from Anthropic, junior developers who used an AI assistant scored 17% lower on comprehension tests than those who coded by hand. The interesting part is how they used it. People who handed the whole task to the AI scored below 40%. People who asked it to explain things scored above 65%, as well as or better than the no-AI group.
  • Researchers Shaw and Nave draw a useful line between cognitive offloading (you delegate a task but keep thinking about it) and cognitive surrender (the AI's answer becomes your answer without much scrutiny).
  • Boston Consulting Group researchers coined "AI brain fry" for the mental hangover of keeping up with tools that produce more than you can absorb.

Here's my take: the wall of text is a big part of the problem. When an agent dumps 900 words on you, the easiest move is to skim, nod and accept. That's surrender by exhaustion.

Voice won't fix this on its own. You can say "yes" to everything just as lazily as you can click "accept". But it changes the default in a few useful ways:

  • You have to say what you want out loud. Putting your intent into a sentence is thinking. It's hard to explain a task without understanding it.
  • Short answers invite follow-ups. When the agent says "I changed the retry logic in two files", the natural reply is "why two?" A spoken back-and-forth pulls you into the reasoning instead of past it.
  • Less to read means less overload. You get the gist by ear and open the detail on screen only when it matters. That's the brain fry problem in reverse.
  • The confirmation step makes you decide. The agent says back what it's about to do, and you have to actively agree.

A few habits I've picked up to stay on the offloading side of the line:

  • Say your guess before you ask. ("I think it's the timeout. Check?")
  • Ask "why" at least once per task.
  • Ask the agent to explain, not just to do, especially in code you don't know yet.
  • Every so often, solve something yourself first and use the agent to check it.

Voice makes these habits feel natural, because they're just how you'd talk to a person.

🚀 Where this is going

I don't think we'll read our agents' output forever. As agents take on longer, more autonomous tasks, the human job shifts from reading everything to staying oriented: knowing what happened, what's blocked, and what needs a decision, while still doing the thinking yourself.

Voice is good at exactly that. It's short and in the moment, and you don't have to sit and stare at a screen to follow it. Detail stays on screen for when you want it; the gist comes to you.

The setup is free, local, and about ten minutes of work. If you spend your evenings building side projects with a coding agent, try talking to it for a week. I don't think you'll go back to scrolling.

🔭 Next up for me: trying NVIDIA's Parakeet model for even faster speech recognition, and moving the speech services to a home server so the laptop stays cool.