I'm building Cutora, an editor that makes the first cut of a talking-head video. Most of the interesting problems turned out not to be "AI" problems at all. They're timing problems. Here's how the cut is decided, and where it still goes wrong.
Step 1: words with times, not just text
Everything starts from a transcript where every word has a start and end time. The cut is planned on words, then rendered on media. If the word times are wrong, every later rule inherits the error, so most of our bugs live here, not in the "smart" parts.
Step 2: pauses
Long pauses get cut, with a short beat left around the words so speech doesn't sound stapled together. Short pauses stay, because they're how people breathe. We don't expose a millisecond threshold; creators pick a pacing and can restore any single cut on the timeline.
The trap: a loudness threshold alone will happily eat a quiet last word. Planning on word times instead of on the waveform avoids most of that.
Step 3: false starts
A false start is a sentence you begin, drop, and restart with the same opening words. The abandoned attempt goes, the finished sentence stays.
The bug that taught us the most: someone says the same greeting three times in a row before starting. Our rule compared a restart only with the words right after it, so a triple opener slipped through. The fix was to treat a run of near-identical openings as one group and keep the last complete one. Obvious in hindsight. It took a real recording to find it.
Step 4: repeated takes
When a line is said more than once in nearly the same words, we score each take on five things: completeness, fluency, how close its pace is to the other takes, how clearly the words were recognised, and loudness. The highest score stays. When two takes score close, a model also listens and looks before deciding.
Two rules matter more than the scoring:
- If we're not sure two takes are the same line, we keep both. A missed cut costs the creator one click; a wrong cut costs them a sentence they meant to say.
- Takes that say different things are never treated as repeats, even if they start the same way.
Step 5: guards before render
Before anything renders, a check looks for cuts that land inside a word and moves them to the word boundary. It sounds trivial. It caught real clicks.
Where it still fails
- Word timestamps from alignment sometimes stretch a word across a silence, so a short stumble hides inside a "long" word and survives the cut. We're working on it; it isn't solved.
- It can't know what you meant, so it can keep the wrong take for a reason it can't see.
- It doesn't find every repeated line, especially when the second take is reworded.
That's why every removed piece is listed on the timeline and can be put back. The edit is a proposal, not a verdict.
Disclosure: I build Cutora. The full write-up of how the first cut decides is here: How the first cut decides what to remove

