Debugging and Fixing Flaky Tests

At a previous job we had an end-to-end suite that was red about one run in five. Everyone knew which tests “always do that”, so everyone reran the pipeline. Then one Friday a real checkout bug shipped, because its failing test was one of the ones that “always do that”. A flaky test isn’t a minor annoyance; it trains the whole team to ignore the alarm.

This lesson has two halves: the tools for seeing why a test failed, and the short list of causes behind almost every flaky test.

Seeing what happened

UI mode is where I spend most of my local debugging time:

npx playwright test --ui

It opens a window listing every test. Run one and you get a timeline of its actions; click any step to see a DOM snapshot of the page at that moment, plus console messages, network requests and the test source. It watches files and reruns on save. There’s a locator picker too: hover the snapshot and it suggests a locator.

The inspector steps through a test one action at a time in a headed browser:

npx playwright test e2e/settle-up.spec.ts --debug

Add await page.pause() in a test to stop at a specific point.

Traces are for failures you didn’t see happen, usually in CI. With trace: 'on-first-retry' in the config, Playwright records a trace on the first retry of a failed test: every action, a DOM snapshot before and after it, console output, network calls and the source line. Download the report from CI and open the trace:

npx playwright show-report            # HTML report, with traces attached
npx playwright show-trace trace.zip   # a single trace

Ninety percent of the time, the snapshot at the failing step tells you the answer immediately: a spinner covering the button, an error toast, a different page than expected.

The usual suspects

Almost every flaky end-to-end test I’ve deleted or fixed had one of these causes.

1. Racing the page. Fixed sleeps (page.waitForTimeout(2000)), values read once and asserted with a non-retrying matcher (expect(await locator.textContent()).toBe(...), expect(await locator.count()).toBe(3)), or if (await locator.isVisible()) branches that check at one arbitrary instant. Fix: web-first assertions (toHaveText, toHaveCount, toBeVisible) and actions that auto-wait.

2. Shared data. Tests that use the same group, the same user or the same seed records, and pass only in a particular order or with one worker. Fix: each test creates its own data, with unique names, through fixtures.

3. Uncontrolled time and randomness. “This week” logic that fails on Mondays, time zones that differ between your laptop and the CI runner, animations still running, random ids in assertions. Fix: page.clock to set the time (await page.clock.setFixedTime(new Date('2026-03-04T10:00:00'))), a fixed timezoneId in the config’s use block, and assertions that don’t depend on generated values.

4. Missing await. page.getByRole('button', { name: 'Add expense' }).click() without await returns immediately; the next line races the click. Fix: the @typescript-eslint/no-floating-promises lint rule catches these at write time.

Third-party scripts (analytics, chat widgets, payment iframes) are a fifth, external cause: block them with page.route in tests unless the test is about them.

Proving the fix

A flaky test that passed once proves nothing. Reproduce first, then prove the fix under the same pressure:

npx playwright test e2e/settle-up.spec.ts --repeat-each=30 --workers=4

If it fails a few times in 30 before the fix and zero after, you’re done. In CI, --fail-on-flaky-tests turns the flaky label into a failure, which some teams enable once the suite is clean so new flakiness can’t creep back in quietly.

The exercise shows a test from a real-looking suite with several of these problems. Find them all. That closes the end-to-end section; section 5 steps back to ask how you judge a suite as a whole.