Coverage matrix
What we test by machine, what we test partly, and what needs a person
Every accessibility scanner finds only part of what an audit finds. Instead of leaving that vague, this page states it per WCAG 2.2 A/AA criterion: which failures our scanner finds itself, where it flags cases for a person to check, and where it cannot judge at all. It is based on what the scanner actually runs, not on what we would like it to do.
Version 1.0 · 2026-09-27 · WCAG 2.2 A/AA · axe-core 4.12.0
This is not a conformance claim
A clean scan does not mean a page conforms to WCAG. Automated testing covers roughly half of what an expert audit finds, and several criteria can only be judged by a person using the site. Treat this matrix as a description of the instrument, not as a certificate.
- automated
- 2 / 55
- The scanner finds the common failures itself and with certainty; what it misses is rare.
- partly automated
- 21 / 55
- The scanner finds part of the failures with certainty and sets doubtful cases aside for a person; the rest is human work.
- human judgement
- 32 / 55
- No automated test; only a person using the site can judge this.
- Rules run
- 70
- axe-core rules mapped to WCAG 2.2 A/AA that every scan executes.
How to read the table: the criterion (official WCAG title), its level, how we cover it, and in one sentence what the machine does and does not see, followed by the rule names so a specialist can check them against the axe-core documentation. “Fix available” marks criteria for which our Shopify app proposes a ready-made change in the theme code.
1. Perceivable
| Criterion | Level | Coverage | What the machine sees, and what it does not |
|---|---|---|---|
| 1.1.1 Non-text Content | A | partly automatedFix available in the Shopify app | Finds images, image buttons, SVGs and objects without a text alternative. Whether an existing alt text is accurate or an image is truly decorative needs a person. image-alt · input-image-alt · object-alt · role-img-alt · svg-img-alt · aria-meter-name · aria-progressbar-name |
| 1.2.1 Audio-only and Video-only (Prerecorded) | A | partly automated | Flags <audio> elements without a captions track. Transcripts placed elsewhere on the page are not detected. audio-caption |
| 1.2.2 Captions (Prerecorded) | A | partly automated | Flags <video> elements without a captions track. Embedded players (YouTube, Vimeo) and open captions burned into the picture need a person. video-caption |
| 1.2.3 Audio Description or Media Alternative (Prerecorded) | A | human judgement | No automated test exists for the presence or quality of audio description. |
| 1.2.4 Captions (Live) | AA | human judgement | Live media cannot be assessed by a page scan. |
| 1.2.5 Audio Description (Prerecorded) | AA | human judgement | No automated test exists for audio description. |
| 1.3.1 Info and Relationships | A | partly automatedFix available in the Shopify app | Finds broken lists, description lists, table header relationships and ARIA parent/child structures. Whether headings, landmarks and form groupings reflect the visual structure needs a person. list · listitem · definition-list · dlitem · aria-required-children · aria-required-parent · td-has-header · td-headers-attr · th-has-data-cells · table-fake-caption · p-as-heading · aria-hidden-body |
| 1.3.2 Meaningful Sequence | A | human judgement | Reading order versus visual order is not machine-decidable. |
| 1.3.3 Sensory Characteristics | A | human judgement | Instructions that rely on shape, colour or position ("click the round button") need a person to spot. |
| 1.3.4 Orientation | AA | partly automated | Finds CSS that locks the page to one orientation. Scripts that do the same are not detected. css-orientation-lock |
| 1.3.5 Identify Input Purpose | AA | partly automated | Checks that autocomplete values are valid. Whether a field that should carry one is missing it needs a person. autocomplete-valid |
| 1.4.1 Use of Color | A | partly automated | Finds links in running text that differ from the text by colour alone. Colour-only meaning in charts, forms and status needs a person. link-in-text-block |
| 1.4.2 Audio Control | A | partly automated | Finds media that autoplays with sound for more than three seconds without controls. no-autoplay-audio |
| 1.4.3 Contrast (Minimum) | AA | partly automated | Measures text contrast against solid backgrounds. Text over images, gradients or overlapping layers is set aside for a person to check. color-contrast |
| 1.4.4 Resize Text | AA | partly automatedFix available in the Shopify app | Finds the viewport setting that blocks zooming. Whether the page still works at 200% needs a person. meta-viewport |
| 1.4.5 Images of Text | AA | human judgement | Whether an image contains text is not machine-decidable in this scanner. |
| 1.4.10 Reflow | AA | human judgement | Requires viewing the page at 320 CSS pixels wide; not part of the scan. |
| 1.4.11 Non-text Contrast | AA | human judgement | Contrast of icons, borders and controls is not measured. |
| 1.4.12 Text Spacing | AA | partly automated | Finds inline styles that prevent users from overriding text spacing. Whether content breaks when spacing is increased needs a person. avoid-inline-spacing |
| 1.4.13 Content on Hover or Focus | AA | human judgement | Tooltip and popover behaviour needs interaction to assess. |
2. Operable
| Criterion | Level | Coverage | What the machine sees, and what it does not |
|---|---|---|---|
| 2.1.1 Keyboard | A | partly automated | Finds scroll areas and frames that keyboard users cannot reach. Custom controls that only work with a mouse need a person. scrollable-region-focusable · frame-focusable-content · server-side-image-map |
| 2.1.2 No Keyboard Trap | A | human judgement | Keyboard traps only show when tabbing through the page. |
| 2.1.4 Character Key Shortcuts | A | human judgement | Single-key shortcuts are not detectable from the page. |
| 2.2.1 Timing Adjustable | A | partly automated | Finds pages that refresh or redirect on a timer. Session time-outs and countdowns in scripts need a person. meta-refresh |
| 2.2.2 Pause, Stop, Hide | A | partly automated | Finds the blink and marquee elements. Carousels and animations need a person. blink · marquee |
| 2.3.1 Three Flashes or Below Threshold | A | human judgement | Flashing content cannot be measured by a page scan. |
| 2.4.1 Bypass Blocks | A | partly automated | Checks for a skip link, landmarks or headings. Whether the skip link actually works needs a person. bypass |
| 2.4.2 Page Titled | A | automated | Finds pages without a title. Whether the title is descriptive is a judgement, but a missing one is certain. document-title |
| 2.4.3 Focus Order | A | human judgement | Focus order needs tabbing through the page. |
| 2.4.4 Link Purpose (In Context) | A | partly automatedFix available in the Shopify app | Finds links and image-map areas without any accessible name. Whether "read more" makes sense in context needs a person. link-name · area-alt |
| 2.4.5 Multiple Ways | AA | human judgement | Whether there is a second way to reach pages (search, sitemap, navigation) is a site-level judgement. |
| 2.4.6 Headings and Labels | AA | human judgement | Descriptiveness of headings and labels is not machine-decidable. |
| 2.4.7 Focus Visible | AA | human judgement | Needs the page to be operated with a keyboard while watching the focus indicator. |
| 2.4.11 Focus Not Obscured (Minimum) | AA | human judgement | Sticky headers and cookie bars covering the focused element only show when tabbing. |
| 2.5.1 Pointer Gestures | A | human judgement | Multipoint and path gestures are not detectable from the page. |
| 2.5.2 Pointer Cancellation | A | human judgement | Down-event behaviour needs interaction to assess. |
| 2.5.3 Label in Name | A | partly automated | Finds controls whose accessible name does not contain their visible text. label-content-name-mismatch |
| 2.5.4 Motion Actuation | A | human judgement | Motion-triggered functions are not detectable from the page. |
| 2.5.7 Dragging Movements | AA | human judgement | Whether drag actions have a single-pointer alternative needs a person. |
| 2.5.8 Target Size (Minimum) | AA | partly automated | Measures whether targets meet 24×24 CSS pixels or have enough spacing. Inline links in text are exempt and skipped. target-size |
3. Understandable
| Criterion | Level | Coverage | What the machine sees, and what it does not |
|---|---|---|---|
| 3.1.1 Language of Page | A | automated | Finds a missing or invalid page language. Whether the declared language is the right one is a judgement, but a missing one is certain. html-has-lang · html-lang-valid · html-xml-lang-mismatch |
| 3.1.2 Language of Parts | A | partly automated | Checks that lang attributes on parts are valid. Passages in another language without a lang attribute need a person. valid-lang |
| 3.2.1 On Focus | A | human judgement | Context changes on focus need interaction to assess. |
| 3.2.2 On Input | A | human judgement | Context changes on input need interaction to assess. |
| 3.2.3 Consistent Navigation | AA | human judgement | Consistency across pages is a site-level judgement. |
| 3.2.4 Consistent Identification | AA | human judgement | Consistency of components across pages is a site-level judgement. |
| 3.2.6 Consistent Help | A | human judgement | Placement of help across pages is a site-level judgement. |
| 3.3.1 Error Identification | A | human judgement | Error messages only appear after submitting a form. |
| 3.3.2 Labels or Instructions | A | partly automated | Finds fields with conflicting labels; fields without any name are reported under 4.1.2 (label, select-name). Whether instructions are sufficient needs a person. form-field-multiple-labels |
| 3.3.3 Error Suggestion | AA | human judgement | Only observable after submitting invalid input. |
| 3.3.4 Error Prevention (Legal, Financial, Data) | AA | human judgement | Confirmation and reversal steps in checkout need a person. |
| 3.3.7 Redundant Entry | A | human judgement | Repeated data entry across a process needs a person. |
| 3.3.8 Accessible Authentication (Minimum) | AA | human judgement | Cognitive tests in login flows need a person. |
4. Robust
| Criterion | Level | Coverage | What the machine sees, and what it does not |
|---|---|---|---|
| 4.1.2 Name, Role, Value | A | partly automatedFix available in the Shopify app | The broadest machine coverage: names of buttons, links, fields and frames, valid ARIA roles and attributes, hidden-but-focusable content, nested controls. Whether states (expanded, selected) are kept up to date by scripts needs a person. button-name · link-name · label · select-name · input-button-name · input-image-alt · frame-title · frame-title-unique · summary-name · aria-allowed-attr · aria-required-attr · aria-roles · aria-valid-attr · aria-valid-attr-value · aria-prohibited-attr · aria-conditional-attr · aria-deprecated-role · aria-hidden-focus · aria-hidden-body · aria-command-name · aria-input-field-name · aria-tab-name · aria-toggle-field-name · aria-tooltip-name · aria-roledescription · aria-braille-equivalent · duplicate-id-aria · nested-interactive · area-alt |
| 4.1.3 Status Messages | AA | human judgement | Whether status messages are announced needs interaction to assess. |
Method and versioning
The scanner loads a page in a real browser the way a visitor does and runs axe-core with the WCAG 2.2 A and AA rule sets. The rule-to-criterion mapping in this table is taken directly from axe-core's own metadata; the classification into automated, partly automated and human judgement is our professional assessment and is stated as such.
Results that axe-core cannot decide (for example text over an image) are shown in every report as items to check by hand, not hidden and not counted as passes.
This matrix carries a version and a date. It changes when the axe-core version, the rule sets or our classification change, so that two editions can be compared.