Automated Testing with axe

Samir has a saying: “The robot finds the typos, a person finds the plot holes.” Automated accessibility tools are fast, cheap and tireless, and they find a large share of the problems on a typical page in seconds. They are also completely unable to tell whether your alt text is true, your focus order makes sense, or your dialog is usable. This lesson shows how to get the most from them, and where they stop.

axe and its family

axe-core, maintained by Deque, is an open-source JavaScript library of accessibility rules. It runs inside the page, inspects the DOM and computed styles, and reports violations with the WCAG criterion, the failing element and a fix suggestion. It is unusually careful about false positives: when it isn’t sure, it reports an item as “needs review” rather than a violation.

You will meet it in several places:

• axe DevTools, a browser extension, for on-demand scans while you develop.
• Lighthouse’s accessibility category in Chrome DevTools, which runs a subset of axe rules and produces a score.
• Test integrations: @axe-core/playwright for end-to-end tests, and wrappers such as vitest-axe for component tests.

WebAIM’s WAVE extension and Microsoft’s Accessibility Insights are good alternatives or companions; Accessibility Insights also adds guided manual checks.

What automation catches, and what it can’t

Automated rules are good at objective, structural facts:

How much is that overall? Studies disagree because they count differently. When the UK Government Digital Service planted 142 known barriers in a test page in 2017, the best tool found 40% of them, and some found far fewer. Deque, counting issues found in thousands of real audits, reports that automated tests caught about 57% of issues by volume, mostly because a few automatable problems, especially contrast, occur so often. A fair summary: automation finds the most frequent problems, but only covers a minority of what WCAG asks.

axe in your Playwright tests

Scanning in CI turns accessibility into a regression test: once a page is clean, a new missing label fails the build. With Playwright and @axe-core/playwright:

import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

const WCAG_AA = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'];

test('event page has no detectable WCAG A/AA violations', async ({ page }) => {
  await page.goto('/events/friday');
  const results = await new AxeBuilder({ page }).withTags(WCAG_AA).analyze();
  expect(results.violations).toEqual([]);
});

test('booking dialog has no detectable violations when open', async ({ page }) => {
  await page.goto('/events/friday');
  await page.getByRole('button', { name: 'Book now Friday pass' }).click();
  await expect(page.getByRole('dialog', { name: 'Book Friday pass' })).toBeVisible();

  const results = await new AxeBuilder({ page })
    .include('#booking')
    .withTags(WCAG_AA)
    .analyze();
  expect(results.violations).toEqual([]);
});

The second test matters as much as the first. axe only sees what is rendered at the moment of the scan, so closed dialogs, collapsed menus, error states and the dark theme are untested unless your test puts the page into those states first.

Notice that the test also finds elements with getByRole and an accessible name. Locating by role and name is Playwright’s recommended approach anyway, and it gives you a small accessibility check for free: if the dialog loses its name, the locator stops matching.

Linting catches problems even earlier

In React projects, eslint-plugin-jsx-a11y flags problems as you type: an img without alt, a div with onClick but no role or key handler, an aria-* attribute that doesn’t exist. It catches less than axe because it only sees source code, not the rendered page, but it catches it minutes after the mistake instead of days.

A layered setup looks like this: the linter in the editor, axe in component and end-to-end tests, and a person with a keyboard and a screen reader before every release. The next lesson is that person.