{"id":"l-a11y-5-1","courseId":"web-accessibility","lang":"en","kind":"lesson","summary":"Run axe in the browser and in Playwright tests to catch the common machine-detectable failures early, and know exactly which problems automation will never find.","html":"

Samir has a saying: “The robot finds the typos, a person finds the plot holes.” Automated accessibility tools are fast, cheap and tireless, and they find a large share of the problems on a typical page in seconds. They are also completely unable to tell whether your alt text is true, your focus order makes sense, or your dialog is usable. This lesson shows how to get the most from them, and where they stop.

\n

axe and its family

\n

axe-core, maintained by Deque, is an open-source JavaScript library of accessibility rules. It runs inside the page, inspects the DOM and computed styles, and reports violations with the WCAG criterion, the failing element and a fix suggestion. It is unusually careful about false positives: when it isn’t sure, it reports an item as “needs review” rather than a violation.

\n

You will meet it in several places:

\n\n

WebAIM’s WAVE extension and Microsoft’s Accessibility Insights are good alternatives or companions; Accessibility Insights also adds guided manual checks.

\n

What automation catches, and what it can’t

\n

Automated rules are good at objective, structural facts:

\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Automation reliably catchesNeeds a human
Missing alt attributeWhether alt text is accurate and useful
Inputs, buttons and links with no nameWhether the name makes sense, or matches the visible label
Text contrast below 4.5:1 or 3:1Contrast over images and gradients
Missing lang, duplicate ids, invalid ARIA valuesWhether the language is right for each part
aria-hidden on focusable elementsFocus order, keyboard traps, focus after a dialog closes
Some heading and landmark problemsWhether headings describe the content
Whether live updates are announced, and whether instructions are clear

How much is that overall? Studies disagree because they count differently. When the UK Government Digital Service planted 142 known barriers in a test page in 2017, the best tool found 40% of them, and some found far fewer. Deque, counting issues found in thousands of real audits, reports that automated tests caught about 57% of issues by volume, mostly because a few automatable problems, especially contrast, occur so often. A fair summary: automation finds the most frequent problems, but only covers a minority of what WCAG asks.

\n\n

axe in your Playwright tests

\n

Scanning in CI turns accessibility into a regression test: once a page is clean, a new missing label fails the build. With Playwright and @axe-core/playwright:

\n
tests/a11y.spec.js
import { test, expect } from '@playwright/test';\nimport AxeBuilder from '@axe-core/playwright';\n\nconst WCAG_AA = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'];\n\ntest('event page has no detectable WCAG A/AA violations', async ({ page }) => {\n  await page.goto('/events/friday');\n  const results = await new AxeBuilder({ page }).withTags(WCAG_AA).analyze();\n  expect(results.violations).toEqual([]);\n});\n\ntest('booking dialog has no detectable violations when open', async ({ page }) => {\n  await page.goto('/events/friday');\n  await page.getByRole('button', { name: 'Book now Friday pass' }).click();\n  await expect(page.getByRole('dialog', { name: 'Book Friday pass' })).toBeVisible();\n\n  const results = await new AxeBuilder({ page })\n    .include('#booking')\n    .withTags(WCAG_AA)\n    .analyze();\n  expect(results.violations).toEqual([]);\n});
\n

The second test matters as much as the first. axe only sees what is rendered at the moment of the scan, so closed dialogs, collapsed menus, error states and the dark theme are untested unless your test puts the page into those states first.

\n

Notice that the test also finds elements with getByRole and an accessible name. Locating by role and name is Playwright’s recommended approach anyway, and it gives you a small accessibility check for free: if the dialog loses its name, the locator stops matching.

\n\n

Linting catches problems even earlier

\n

In React projects, eslint-plugin-jsx-a11y flags problems as you type: an img without alt, a div with onClick but no role or key handler, an aria-* attribute that doesn’t exist. It catches less than axe because it only sees source code, not the rendered page, but it catches it minutes after the mistake instead of days.

\n

A layered setup looks like this: the linter in the editor, axe in component and end-to-end tests, and a person with a keyboard and a screen reader before every release. The next lesson is that person.

\n","toc":[{"id":"sec-1","text":"axe and its family","level":2},{"id":"sec-2","text":"What automation catches, and what it can’t","level":2},{"id":"sec-3","text":"axe in your Playwright tests","level":2},{"id":"sec-4","text":"Linting catches problems even earlier","level":2}],"takeaways":["axe-core is the open-source rules engine behind axe DevTools, Lighthouse’s accessibility audit and many test integrations.","Automated tools reliably catch missing alt, labels and names, contrast failures, invalid ARIA and missing page language.","They can’t judge meaning, so alt quality, focus order, keyboard behaviour, live announcements and label-in-name mismatches need a human.","Run axe in CI against key pages and states, such as an open dialog, so regressions fail the build, and treat a clean scan as the floor, not the finish."],"further":[{"title":"Playwright: Accessibility testing","url":"https://playwright.dev/docs/accessibility-testing"},{"title":"W3C WAI: Selecting Web Accessibility Evaluation Tools","url":"https://www.w3.org/WAI/test-evaluate/tools/selecting/"},{"title":"web.dev Learn Accessibility: Automated testing","url":"https://web.dev/learn/accessibility/test-automated"}],"quiz":[{"q":"

Ticketly’s home page scores 100 in Lighthouse’s accessibility audit. What can you conclude?

","options":[{"text":"The page meets WCAG 2.2 AA","why":"Lighthouse runs a subset of axe rules. A perfect score means none of those automated checks failed, not that the page conforms."},{"text":"None of the automated checks Lighthouse runs found a problem; keyboard, screen reader and content issues are still untested","why":"Correct. It is a good sign and a useful regression guard, but most of WCAG needs human judgment."},{"text":"Screen reader users will have no trouble with it","why":"Automated tools can’t tell whether names are meaningful, focus moves sensibly, or updates are announced. Only testing with a screen reader shows that."}],"answer":1},{"q":"

Which of these problems would axe flag by default?

","options":[{"text":"An image with alt="image"","why":"The alt attribute is present, so there is nothing for a rule to detect. Whether the text is meaningful needs a human."},{"text":"A modal that leaves focus behind it when it opens","why":"Focus management depends on interaction and intent; a static scan can’t know where focus should go."},{"text":"A status message in a plain paragraph that screen readers never announce","why":"axe can’t know which text changes dynamically or should be announced. This is found by testing with a screen reader."},{"text":"A text input with no label and no accessible name","why":"Correct. Missing form labels are exactly the kind of objective, structural failure automated rules detect reliably."}],"answer":3},{"q":"

Why might this Playwright test pass even though Ticketly’s booking dialog has contrast failures?

\n
JavaScript
await page.goto('/events/friday');\nconst results = await new AxeBuilder({ page }).analyze();\nexpect(results.violations).toEqual([]);
\n","options":[{"text":"AxeBuilder ignores color contrast by default","why":"color-contrast is one of axe’s default rules. The problem is what is on the page at scan time."},{"text":"The dialog is closed when the scan runs, so its content isn’t rendered and isn’t tested","why":"Correct. Open the dialog in the test (click Book now) and scan again, ideally with .include() on the dialog, to test that state."},{"text":"Playwright runs headless, and contrast can’t be measured without a visible window","why":"Headless browsers render pages fully, and axe computes contrast from styles, so headless mode isn’t the issue."}],"answer":1}],"exercise":{"kind":"spot-bug","language":"html","code":"
\n

Harbour Lights 2026

\n \"image\"\n

Tickets

\n \n \n \n

\n \n Festival FAQ\n
","bugLines":[3,4,8,9],"title":"Find what axe can't","prompt":"

This part of Ticketly passes an axe scan with zero violations. A screen reader user still runs into four real WCAG failures. Select the lines a human reviewer would flag. (.section-title is styled to look exactly like the other section headings, and addToBasket() writes “Ticket added” into #basket-status.)

\n","hints":["axe checks that things exist. Look for things that exist but mean the wrong thing.","Compare what a sighted user sees with what the accessibility tree says, line by line.","One problem is about text, one about structure, one about dynamic updates, and one about voice control."],"explanation":"

Line 3: alt="image" is present, so axe is satisfied, but it tells a screen reader user nothing (“image, image”). Describe the photo or, if it is decorative, use alt="".

\n

Line 4: the paragraph looks like a section heading but isn’t one, so it is missing from heading navigation (1.3.1 Info and Relationships). Use <h2>Tickets</h2>.

\n

Line 8: #basket-status changes after each click, but it isn’t a live region, so screen readers announce nothing (4.1.3 Status Messages). Add role="status".

\n

Line 9: the visible text is “Reserve seats” but the accessible name is “Book”, so a voice control user saying “click Reserve seats” gets nothing (2.5.3 Label in Name). Remove the aria-label. axe has a rule for this, but it is experimental and off by default.

\n

The other lines are fine: the input is labelled, the buttons and link have clear names.

\n","checkLabels":{}},"words":728,"readMinutes":3}