A team agrees to launch a pilot on November 9. In the next meeting, they move it to November 12, cut the pilot in half, and make the launch conditional on security approval. Ask an AI assistant for the launch date and it can say “November 12” while still giving someone the wrong idea about whether the launch can go ahead.
At Meeting.ai we used that scenario to test what our assistant could recover from two meetings. Here’s the fixture and what we learned from running it.
Give the two recordings different jobs
The first recording established the plan: 20 customer accounts, launch on November 9. The second revised it to 10 accounts and November 12, pending security approval. Maya agreed to send the account list by November 5. Alex agreed to revise the plan by November 6. We deliberately left the security approval without an owner. These are fictional names and dates.
We used short synthetic audio clips with clearly spoken names, uploaded both, and attached both meetings to the same conversation. That last detail matters if you try to reproduce the test: asking a system to find the right meetings in a full account is a different test from asking it to read two records you’ve already selected.
Write the answer key before reading the summary
Try this prompt: What changed between the two meetings? What still has to happen before launch, and who agreed to do what?
| Check | What earns a pass | An answer to flag |
|---|---|---|
| Scope | Distinguishes the original 20 accounts from the revised 10 | Lists both numbers without telling the reader which one applies |
| Launch | Gives November 12 and preserves the security approval condition | Says the launch is confirmed for November 12 |
| Ownership | Assigns the account list to Maya and the revised plan to Alex | Guesses who owns security approval |
| Status | Describes both tasks as commitments | Says either person has finished the work |
Test the evidence separately
In our run, the assistant recovered the revised scope, date, condition, and commitments. It also left the approval owner unassigned. Then we asked it to show where the answers came from. It returned meeting names and timestamps, but the answer contained no clickable meeting links. Some quoted wording came from the generated notes. We couldn’t treat that wording as a verbatim transcript excerpt.
That gave us two separate checks for the next run: can a reader open the cited record, and can they locate the words that support the claim? An answer can get every cell in the table right and still leave the reader hunting for evidence.
For a second round, record the same scenario on a phone with two consenting speakers. Keep the script and answer key fixed so you can compare what changes with room audio. Our phone recording guide covers microphone placement and a short playback check before you start. The synthetic run above tested recall from clean audio; room capture is a separate experiment.
Disclosure: I work on Meeting.ai. This article was drafted with AI assistance from our recorded test results.
