`If your team clicks Re-run failed jobs more than once a week, your end-to-end suite has stopped telling you the truth. Red no longer means "you broke something". It means "something was slow, probably".
I built QAFind to fix that. It's an open-source (MIT) CLI that attaches to the Playwright suite you already run. You don't rewrite specs, adopt a new runner or sign up for anything.
What it does, in one sentence each
-
attachwraps your existingplaywright.config.ts. Your config and specs are read, never written. -
verify-mergeruns the specs a PR touches, reruns failures once, and fails CI only on a regression that reproduces. -
flakeaggregates runs and tells you which tests flake, the probable cause, how confident it is, and who owns them. -
attach --determinismrecords and replays network traffic and freezes the browser clock, so fewer tests flake in the first place. - Signed evidence: every verdict can be signed (HMAC or Ed25519) and exported as an in-toto attestation you can verify offline.
60-second start
From the root of any Playwright project:
bash
npx qafind attach .
npx playwright test --config .qafind/playwright.qafind.config.ts
npx qafind flake .qafind/reports --markdown
That's it. The first command writes a wrapper config into .qafind/. The second is your normal test run with one extra reporter. The third gives you a table like this:
text
| Test | Runs | Flakes | Cause | Quarantine |
| cart fixture returns totals | 3 | 1 | network_fixture | yes |
| login selector exists | 3 | 1 | selector_drift | yes |
| slow recommendations endpoint | 3 | 1 | timing | yes |
| known checkout total mismatch | 3 | 3 | app_race | no |
Notice the last row: it failed every time, so it's a real failure and it is never suggested for quarantine.
The merge gate
Add one step to CI:
bash
npx qafind verify-merge --run --base origin/main --head HEAD
| First run | Rerun | Verdict | Blocks merge? |
|---|---|---|---|
| failed | failed | regression | yes |
| failed | passed | flake | no (but it's recorded) |
| failed | different failure | unreproduced | no |
Flakes stop holding your PRs hostage. Real regressions still stop the merge. Add --markdown for a PR comment, or --intoto for a signed attestation.
Why not just use retries?
retries: 2 turns a flaky failure into a green exit code and throws away the information. Nobody counts how often a test needed a retry, and an intermittent bug your users will hit looks identical to a slow CI runner. QAFind keeps that information, counts it, and puts it in front of a human. Only the reproduced failures block.
What it isn't
I'd rather you trust it than be surprised by it:
- Playwright only, for now.
- Cause labels are suggestions, based on error text plus an optional request log. On a synthetic benchmark (400 tests × 5 runs) they're 87.8% accurate on text alone and 90.3% with the request log. Check before you quarantine.
- WebSockets and service workers aren't recorded by the network replay yet.
- Not a test generator. It makes the tests you already have more trustworthy.
Free for open source
The CLI is MIT and fully local: no account needed for attach, flake or verify-merge.
There's also an optional hosted GitHub App (QAFind CI) that runs QA on pull requests and comments the report on the PR. Public repositories are free with no daily quota.
Try it and tell me where it's wrong
bash
npx qafind attach .
⭐ Repo: github.com/katesaikishore/QAFind
🌐 Site: qafind.si
If you run it on a real suite, the most useful thing you can send me is a test it labelled wrong. Open an issue or drop it in the comments.
What's the flakiest test in your suite right now, and what's its excuse?`
