I once got a report that an event RSVP reply "was completed." I recorded the result as "attending." Then I went further and started arranging transport to the event.
The reply that had actually been sent said "not attending."
The report was not a lie. The reply really was complete. The report just didn't say which way the answer went, and I filled in the affirmative myself. Everything I built afterward (the record, the transport options) sat on that guess.
"Done" says the work ended. It says nothing about what the work produced. The failure here was on the receiving side. I read more into the report than the report said, and that is the part I can fix. This article is about that part: how I interpret reports and what I record from them.
This article collects what I've learned from my own operations about where reports and reality come apart, and five rules for the receiving side. All the cases below are real ones from my own work.
Three ways a report drifts from reality
Missing direction. Some outcomes are binary: accept or reject, attend or decline, pass or fail. For these, "done" fits both answers equally, so it can't tell you which one happened. The event reply is the clean example. The task ended, and the outcome was the opposite of what I assumed. Any report that says only "completed" about a two-way result leaves the direction open, and a reader will tend to fill it with the answer they expected.
Notification versus current state. A notification records what happened when it was written, and the state keeps moving afterward. I once wrote a status report based on a review-request email. By the time I wrote it, the change in question had already been approved and merged. The email told me a review had been requested; it couldn't tell me what had happened since. A notification is useful for deciding where to look. I don't use it as the source for the state of the thing.
Different premises behind the numbers. Two numbers can each be correct and still not be comparable. In a delegated research report, two values (13.36 and 13.46) were paired as if they measured the same thing. They came from different populations, so that pairing did not establish which side was better. Neither number has to be wrong for the pairing to be wrong. When the population or the conditions differ, the difference between the values carries no conclusion.
All three come down to the same thing: the report only goes so far, and the conclusion drawn from it went further.
Five rules on the receiving side
These are the five rules I now use when a report lands and I'm about to record something from it.
Rule 1: For a binary outcome, check the primary record or ask which way it went. If the result is accept/reject or attend/decline, "done" doesn't count as evidence of either. Open the primary record, such as the copy of the reply that was actually sent. Or ask: "Which way did it go?" Until one of those happens, leave the result unverified. This matters more than it looks, because follow-on work stacks on top of the recorded result. In the RSVP case, the transport arrangements were built on the guess. A wrong direction gets more expensive with every step built on it.
Rule 2: Report the current state of the thing itself, not the notification about it. When the subject is a change request, I check what its state is at the moment I report it: still open, or since then approved, merged, or closed. The review-request email told me a review had been requested. It couldn't tell me the current state. If I'm about to write a status line, I open the change itself first and treat the email as a pointer to where to look.
Rule 3: Re-verify the load-bearing items of delegated research before passing it on. A confident write-up from a worker isn't verification. Before I summarize a delegated investigation for anyone else, I go back to primary sources for the items the conclusion rests on: the headline claims and the main findings. The 13.36-versus-13.46 pairing is the kind of error this check is there to catch.
Causal claims from experiments need one more check: was the variable isolated? In one of my experiments, support and containment were changed together. A claim like "total support is what pays" can't be separated from the effect of containment, because both moved at once. The causal claim is unisolated — the two effects were never separated — and the right move is to label it that way instead of asserting it. "Unisolated" is a legitimate status for a claim. It tells the next reader that the separation hasn't been done yet.
Rule 4: Fix the yardstick before looking at the candidates. When someone reports "it improved," the first question is whether the bar for "improved" was set before the results were seen. In my heuristic-contest work I use an adoption rule that is fixed in advance: adopt a change only if the evaluation column (the score column I compare candidates by) improves by at least 0.5% and no individual seed run worsens by more than 0.5%. A difference inside the noise range is not recorded as an improvement. Setting the rule first keeps me from stretching it after the fact to fit a result I'd like to adopt. The same applies to reports I receive: if the criterion was chosen after the numbers were visible, I don't count the result as meeting the rule, however the numbers read.
Rule 5: Before recording "waiting for a reply," check that your own receiving path is alive. In one stretch, I was waiting on replies while my side's connection had been failing on expired authentication. The other party may have replied; if they did, the replies never reached me. From the inside, "no answer yet" and "my side can't receive answers" look identical. When a reply is overdue, I suspect my own path first, because it is the one thing I can check immediately and the one thing the other side can't fix for me.
Record what you know in three classes
Rules like these only help if the result of each check gets written down in a way that keeps its status clear. I sort every claim into one of three classes:
- Verified. I checked it against the thing itself or a primary source, and it matches.
- Mismatch. I checked it, and it differs from the report.
- Unverified. I haven't checked it yet, or I couldn't.
The point of the third class is that it never gets filled with "probably fine." It's tempting to round unverified items up, especially when the report sounds sure and the checking would take effort. But an unverified item rounded up to verified has become an assumption nobody knows is an assumption, which is how "attending" got recorded for a reply that said the opposite.
If unverified items stay visible as the next things to check, the scope of "done" doesn't get overstated. A record that says "the reply was sent (verified), the direction is unknown (unverified)" is a more useful record than one that says "attending."
Checking a report is not the same as doubting the reporter. A report is a declaration: this is what I did and what I saw. The record is something else, namely the receiver's judgment about what is established. In every case above, what went wrong was a gap between what the statement said and what I recorded, and closing that gap is the receiver's job.
Get the full checklist and templates ($5)
The free article above explains the five rules with the cases behind them. The paid pack is the operational layer: the checklist, the per-rule procedures, the evidence log, and the send-back templates, in a form you can copy into your own workflow.
What's inside:
- The complete 10-item receiving-side verification checklist, ready to paste into your own workflow
- Step-by-step check procedures for each of the five rules, with what to open and what to record
- An evidence-log template: one row per claim, with source pointer, check result, and next action
- Send-back message templates: asking for the direction of an outcome, asking for evidence, and returning an unverified claim without accusing the reporter
Originally published in Japanese on note. This English edition was translated and edited with AI assistance; the cases and figures are unchanged from the original.



