A flaky Playwright test is one that passes and fails on the same code. The fix is almost never a longer timeout or more retries. It is finding which of four things is wrong: timing, selectors, shared state, or the environment. Then you fix that one thing.

Here is the process I use, in order. Each step either finds the cause or rules one out.

1. Make the flake happen on demand

You can't fix what you can't reproduce. Run the failing spec many times in a row:

npx playwright test tests/checkout.spec.ts --repeat-each=20 --project=chromium

Then run it again with a single worker:

npx playwright test tests/checkout.spec.ts --repeat-each=20 --project=chromium --workers=1

Compare the two runs:

  • Fails only with many workers: tests share something. Usually a user account, a database row, a file, or a port.
  • Fails even with one worker: the problem is inside the test. Usually timing or a selector.
  • Never fails locally, fails in CI: the environment is different. Look at server startup, CPU, browser version, and test data.

Write down which case you're in. It decides where you look next.

2. Turn on traces for failures

Guessing from a stack trace wastes hours. A trace shows every action, the DOM at each step, and the network calls.

// playwright.config.ts import { defineConfig } from '@playwright/test'; export default defineConfig({ retries: process.env.CI ? 1 : 0, use: { trace: 'on-first-retry', screenshot: 'only-on-failure', }, });

Open a trace with:

npx playwright show-trace test-results/<test-folder>/trace.zip

Look at the step that failed and the one right before it. Ask one question: what did the page look like when the test acted? Most of the time you'll see a spinner still running, a modal on top of the button, or an old list that hadn't refreshed yet.

One retry in CI is fine here. Its job is to capture a trace, not to hide the failure. Playwright marks a test that passed on retry as "flaky" in the report, so keep an eye on that list.

3. Replace fixed waits with waits for real state

This is the most common cause I see. The test waits a fixed time and hopes the app is ready.

// Flaky: sometimes the save takes 300 ms, sometimes 1.5 s await page.getByRole('button', { name: 'Save' }).click(); await page.waitForTimeout(1000); expect(await page.getByText('Saved').isVisible()).toBe(true);

isVisible() checks once and returns. If the toast shows up 50 ms later, the test fails.

Use a web-first assertion instead. It retries until the condition is true or the timeout runs out:

// Stable: waits for what the user would see await page.getByRole('button', { name: 'Save' }).click(); await expect(page.getByText('Saved')).toBeVisible();

When the next step depends on a server response and there's nothing visible to wait for, wait for that response. Start listening before you click:

const saved = page.waitForResponse( (res) => res.url().endsWith('/api/profile') && res.ok() ); await page.getByRole('button', { name: 'Save' }).click(); await saved;

A quick way to find these problems: search your test folder for waitForTimeout, isVisible(), and textContent() inside expect(...). Each one is a candidate.

4. Use locators that survive UI changes

Selectors like this break when someone moves a div:

await page.locator('div.toolbar > div:nth-child(2) > button').click();

They also cause flakes when the list renders in a different order, or a second matching element appears for a moment during a re-render.

Prefer locators that describe what the user sees:

await page.getByRole('button', { name: 'Create project' }).click(); await page.getByLabel('Email').fill('test@example.com');

Order of preference: getByRole with a name, then getByLabel, then getByText, then getByTestId when nothing visible is stable enough.

If you see "strict mode violation" errors now and then, the locator matches more than one element some of the time. Narrow it to a region instead of grabbing .first():

const dialog = page.getByRole('dialog', { name: 'Invite member' }); await dialog.getByRole('button', { name: 'Send invite' }).click();

5. Give every test its own data

If step 1 showed failures only with parallel workers, this is your step.

Typical cases:

  • Two tests log in as the same user and change its settings.
  • One test deletes a record another test expects to exist.
  • A test counts rows in a table that other tests are adding to.

The fix is that each test creates what it needs, with a unique value, and doesn't rely on order:

import { test, expect } from '@playwright/test'; test('user can rename a project', async ({ page, request }, testInfo) => { const name = `proj-${testInfo.workerIndex}-${Date.now()}`; await request.post('/api/projects', { data: { name } }); await page.goto('/projects'); await page.getByRole('link', { name }).click(); // ... });

Creating data through the API is faster and less flaky than clicking through the UI to set it up. Save UI clicks for the flow you're actually testing.

Avoid test.describe.configure({ mode: 'serial' }) as a permanent fix. It hides the shared state instead of removing it.

6. Check the environment last

If the test only fails in CI, compare these between your machine and the CI runner:

  • Is the app ready before tests start? Use the webServer option with a url so Playwright waits until the server answers.
  • Same browser version? Pin Playwright and install browsers with npx playwright install --with-deps in CI.
  • Slower machine? CI runners are often slower. That exposes timing bugs from step 3 that your laptop hides. Fix the wait, don't just raise the timeout.
  • Third-party calls? Payment widgets, analytics and auth providers can be slow or rate-limited in CI. Mock them with page.route() when the test isn't about that integration, and keep one separate test that hits the real thing.

7. Keep a short flake log

Once a week, look at the tests Playwright marked as flaky. For each one, write one line: the test name, the cause you found, and the fix. After a few weeks you'll see your team's pattern. It's often one or two causes repeating, like a shared login user or a toast assertion with isVisible().

Quarantine is OK for a day or two with test.fixme() and a linked issue. Leaving a test skipped for months means that flow has no coverage, and nobody notices.

Quick checklist

  1. Reproduce with --repeat-each=20, with and without --workers=1.
  2. Turn on traces and read the step before the failure.
  3. Replace waitForTimeout and one-shot checks with web-first assertions.
  4. Switch to role, label and text locators.
  5. Give each test unique data and no order dependence.
  6. Compare the CI environment with your machine.
  7. Log each flake's cause and fix it within days.

Retries and longer timeouts make the red go away. They don't make the test trustworthy. Fix the cause and the suite becomes something you can rely on before a release.

I'm building AlwaysQA, which checks login, checkout and other critical flows in a real browser after your coding agent ships a change. It sits next to a Playwright suite rather than replacing it.