FAQ

"My results changed after upgrading to core 1.16"

1.16 brings jsdom closer to the browser and words its messages for any standard. The changes most likely to show in a pipeline:

• Under jsdom, only a dialog opened with showModal() is modal, as in a browser. The page behind a <dialog open aria-modal="true"> is now judged, so a jsdom test that fakes a modal that way sees findings behind it; engineOptions.assumeModal: true keeps the old behaviour (see Known limitations; with test-matchers 1.6.0, give it to installLiveRegionHook() too, see test-matchers). Nothing behind a modal dialog takes focus, and of several opened with showModal() only the topmost is judged.
• jsdom hides what a browser hides. Rules inside @layer, @supports and @container hide elements, so Tailwind v4's hidden and sr-only work; clip: rect(0 0 0 0) counts as clipped; an image map's <area> is on the page. Findings on content that was hidden all along go away.
• Messages and steps describe the check instead of naming the WCAG criterion, in every locale, so a report for another standard reads right. Mappings and failWhen are unchanged; a test that compares message text needs updating.
• Deep pages are judged. An element 200 or more levels deep is judged with everything above it, up to 10,000 levels, where a fail that deep used to become a question noted in error; so failures can appear there. Past 10,000 it is a question, reported in ancestorWalkLimit.
• Pages that replace Array.from (Prototype.js, MooTools) are scanned, where the scan stopped with PAGE_BUILTINS_BROKEN (see the entry below).
• Some selectors are different. Control characters are escaped, a value over 256 characters gives way to the element's path, and page-title-present and page-title-patterns point at the title element itself, or at <head> when there is none. A finding's identity doesn't include its selector, so baselines still match. The contrast rules are notApplicable with no occurrence when no contrast could be computed.
• Mistakes are reported instead of read as nothing. An empty contextSelector still scans the whole page, but warns; something that isn't a baseline throws INVALID_BASELINE instead of making every failure new; and under strictOptions, an unknown profile name throws. The section508 profile is deprecated: use wcag20-aa, which runs the same rules.
• Faster scans, with the same findings: a few percent in Chromium on ordinary pages, 10 to 25% under jsdom, and much more where it was worst (thousands of identical links or fields, long tables, many style rules).

New, and changing nothing unless you use it: the WCAG 2.0 and 2.1 profiles (Engine options), and, experimental until 1.18.0, standards as data, tests under requirements, standards built on others and the laws a standard is assessed under (Packs). The full list is in core's CHANGELOG.md.

"runOnly threw: no rule or tag named …, or either rule ids or tags"

Since 1.10.0, runOnly given as a bare array or string is read as rule ids or as tags (see ENGINE_OPTIONS.md). It throws in two cases:

• A value that names no rule and no tag, usually a typo: runOnly: ['img-alt-presnt']. Check the id against getChecksCatalog(). Case is not the cause: rule ids and tags are matched in any case, so IMG-ALT-PRESENT selects img-alt-present.
• Rule ids and tags in one array: runOnly: ['img-alt-present', 'wcag2a']. Use the object form to combine them:

runOnly: {
  includeRuleIds: ['img-alt-present'],
  tags: ['wcag2a'],
  includeMode: 'or',
}

Before 1.10.0, a bare array was ignored and every rule ran.

The object form, { includeRuleIds }, { tags } or { type: 'tag', values }, and engineOptions.rules.include / .tags.include throw too, with code: 'INVALID_RUN_ONLY', when no value in the list names a rule or a tag (runOnly.tags: no tag named "wcag2.2aa".). They used to run no rule at all, which reads as a clean pass. A single unknown value beside known ones, or in an exclude list, only logs a console.warn.

"I selected tags and rule ids, and the scan ran no rule but looks clean"

Under the default includeMode: 'and', a selection with both tags and rule ids runs only the rules that match both. .withTags('wcag2a').withRules('region') runs nothing, since region is a best practice with no wcag2a tag, and a scan of no rules has no failures. The engine warns in the page's console, which a test run doesn't show, and throws INVALID_RUN_ONLY only under strictOptions. To run either, ask for it: .includeMode('or') in the bindings, includeMode: 'or' in runOnly (above), or --include-mode or in the CLI. From version 1.3.0, the CLI exits 2 with that hint when the two have no rule in common.

To catch a selection that picks nothing before it runs, check the builder's plannedChecks(), the rules the scan will run, worked out without a page (a chainable in Cypress):

const builder = new A11yCoreBuilder({ page })
  .withTags('wcag2a')
  .withRules('region')
  .includeMode('or'); // the wcag2a rules, plus region
 
if (builder.plannedChecks().length === 0) {
  throw new Error('This selection runs no rule.');
}

See Engine options for how the parts of a selection combine.

"runOnly.tags: no rules for WCAG 2.2 Level A ("wcag22a")"

The nine WCAG version/level tags, wcag2a to wcag22aaa, are always known, but three carry no rule: wcag21aaa, wcag22a and wcag22aaa, whose criteria have no automated check. Naming one beside other tags logs that line with console.info: the rest of the list runs, and those criteria need manual review. Pass engineOptions.logUntestedWcag: false to leave it out, or select by target instead, which names no tag: runOnly: { wcag: { version: '2.2', level: 'AA' } } (see Selecting by WCAG target). A list of only such tags would run nothing, so it throws INVALID_RUN_ONLY with the same explanation.

"runOnly.wcag.version must be …" or "runOnly.bestPractices must be true or false …"

runOnly.wcag takes { version, level }, with version one of '2.0', '2.1', '2.2' (a string: 2.2 as a number is refused) and level one of 'A', 'AA', 'AAA'; runOnly.bestPractices takes true or false. Anything else throws INVALID_RUN_ONLY naming the field and the value, rather than running a selection you did not mean.

"engineOptions: unknown option "lcoale" (did you mean "locale"?)", or a scan threw INVALID_ENGINE_OPTIONS

An option key one or two letters from a known one, in any case, is warned about and ignored: the scan runs as if it weren't there, so fix the spelling. Other unknown keys stay silent, since a custom rule can read settings of its own from ctx.engineOptions. With engineOptions.strictOptions: true, the scan instead throws, with code: 'INVALID_ENGINE_OPTIONS', on any key the engine doesn't read (nested ones included) and on a value of the wrong type or outside its set, naming each with the closest known key; the error's problems lists them. See Catching mistakes in the options.

"My custom rule fails, but the WCAG rollup doesn't"

A custom rule counts toward the WCAG rollup (rulesResults) of each criterion it maps to: give it meta.wcagSc: ['1.1.1'], or a WCAG entry in meta.normativeMappings. With neither, it is in no rollup. A custom rule that didn't run (listed in skippedCustomRules) doesn't count either. An override of a built-in counts only where its own mapping says, so one that declares none leaves the rollups its built-in counted toward, and a warning (customRules: "…" overrides a built-in rule but maps to none of the criteria of …) names them; give it the built-in's criteria to keep it there.

"My scoped scan reports everything as notApplicable", or "contextSelector threw: … is not a valid CSS selector"

Since 1.10.0, a contextSelector that matches no element scans nothing, and one the browser can't parse throws (see ENGINE_OPTIONS.md).

• Everything is notApplicable: look at result.contextMatch. elementCount: 0 means the selector matched nothing, so nothing was scanned: that is not a clean result. unmatchedSelectors names the selector. Check it for a typo, or whether the element is on the page yet when you scan (content rendered by a script after load, say).
• The scan threw with code: 'INVALID_CONTEXT_SELECTOR': the selector in the message, and in the error's selector, isn't valid CSS, for example '#main[' or a >>> combinator. Or the scope isn't a selector at all, such as an element or an { include, exclude } object, and the message says what was passed: pass a selector that matches the element, and exclusions in engineOptions.excludeSelectors.

Each of these used to scan the whole page instead: a selector that matched nothing or couldn't be parsed until 1.10.0, and a scope that isn't a selector in 1.10.0 too.

"A scan threw PAGE_BUILTINS_BROKEN"

The page's own scripts changed a JavaScript built-in the engine relies on (Array.prototype.filter, Map, JSON.stringify and others), so a scan of it could give a result that reads well and is wrong. The error's message and its broken list name each one. Scan the page before its scripts change them (inject the bundle earlier, or scan a build without the script that does it). See Known limitations.

When broken is only Array.from, the page replaced it (Prototype and MooTools do) and a pack's or custom rule in the scan uses it: core's own rules don't, since 1.16. Write that rule with spread ([...x], [...x].map(f)), or leave it out of the selection. The WAI-ARIA pack did this until its 1.0.2 release: update it.

"My scan never returns"

Something in the page blocks the scan: a getter or a script that never returns, or a pack's or custom rule's loop. Nothing inside the page can stop a scan once it has started, so put the time limit where you start it, and give up on the tab when it passes. Standalone shows how for Playwright and Puppeteer; in jsdom, run the scan in a worker thread or a child process.

"A scan takes longer after upgrading"

Most of a scan's time is style and layout work in the browser, so it depends on the page, the machine and the browser as much as on the engine. A release can still change it: a rule that now checks more of the page costs more, and core's CHANGELOG.md says when a release changes what a rule costs. PERFORMANCE.md, on GitHub, records how long a scan takes release by release, as a ratio to an earlier release on the same pages, and how that is measured. A rule that is slower and finds more may be paying for what it now finds; one that is slower with the same results is a regression worth reporting.

A scan of several profiles at once (profile: ['wcag22-aa', 'wai-aria-1.2']) scans the page once per profile, so it takes about as long as that many scans.

"A profile did not run", or engine.profilesNotRun is not empty

In a scan of several profiles, a profile that couldn't run is listed in engine.profilesNotRun with its reason, and the others still run: unknown-profile (no such profile: a pack's profile needs its pack in packs), pack-skipped (its pack didn't run, see the entry below) or overridden-by-selection (a runOnly include, or rule ids or tags in engineOptions, chose the rules instead, for every profile). Such a scan judged less than it was asked to, so SARIF and JUnit report an error unless allowProfilesNotRun: true, the CLI exits 2 unless --allow-profiles-not-run, the bindings report a 'profile-not-run' gap, and the test matchers fail the assertion. See Engine options.

"A pack did not run", or skippedPacks is not empty

A pack whose core range leaves out the installed core, that isn't valid, or that doesn't fit with core (a rule id outside its namespace, a dictionary key core has) is skipped: the scan runs without it, warns, and lists it in result.skippedPacks with the reason; under strictOptions it throws. In a page, a pack named in engineOptions.packs that no packScript call registered, or registered with another core version, is listed the same way. The bindings report it as a 'pack-skipped' gap, the test matchers fail the assertion, and the CLI exits 2. Install the pack beside @surea11y/core and the package that runs it, so all three share one copy of core. See Packs.

"My custom rule always returns cantTell with no clear reason"

Check the result's error field first — if it says "<something> is not defined", your runInPage references a variable from outside the function body (a module-scope const, an imported helper, anything not reached through ctx.*). This is a real, common footgun: runInPage is serialized to source text and re-evaluated later in the page context, so the build never catches this — only running the rule does, and the failure looks like a normal (if uninformative) result, not a crash. See RULE_AUTHORING.md §1.1 for the full explanation and the fix (move the value inside runInPage, or route it through ctx.rule/ctx.helpers).

"A geometry-dependent rule (e.g. target-size-minimum) always says notApplicable"

Plain jsdom (no real browser) doesn't implement CSS layout — getBoundingClientRect() always returns zero geometry. Rules that need real layout report notApplicable under jsdom rather than guess. Run through a real browser instead (Puppeteer/Playwright — see INTEGRATION.md Pattern 2) to get real findings from these rules. See LIMITATIONS.md.

"The title, language and landmark rules are notApplicable in a frame"

Since 1.15, a document shown in a frame is no page of its own: page-title-present, page-title-patterns, html-lang-attr-present, meta-viewport-zoom-enabled, meta-viewport-large, region, bypass-blocks-present, landmark-one-main and page-has-heading-one are notApplicable in it. The page around the frame is what has the title, language and landmarks people use, and a browser ignores a framed document's viewport <meta>. A test runner that shows the page under test in a frame of its own sets engineOptions.topLevelDocument: true; @surea11y/cypress 1.6.0 does it for you. See Engine options.

"A web component comes back cantTell (name_internals)"

A custom element that names itself, sets its states or gives itself a role through an ElementInternals kept in a private field (#internals) is exposed by the browser, but no page script can read that back. Since 1.12 the engine reads internals kept on a property (_internals, internals, …), and asks about any other defined custom element with no readable name or required state (name_internals, state_internals) instead of failing it.

To let the scan judge such elements, install the internals hook before the page's scripts run: installInternalsHook() in each binding (on the page or context in Playwright, the page in Puppeteer, the driver in Selenium, the browser in WebdriverIO, the support file or onBeforeLoad in Cypress) and in test-matchers' setup file, or core's internalsHookScript() for a page of your own. The result's engine.environment.internalsHook says whether it ran ('early' or 'late'). The hook lets any script on the page read every component's internals, so it is meant for test and audit runs. See Known limitations.

"I only see fail/cantTell occurrences — where's the list of elements that passed?"

By design, this engine never enumerates the elements a rule passed — only the ones it flagged. A rule's overall outcome: 'pass' means "applicable target(s) existed and none were flagged," and occurrences is []. The contrast rules are the exception: their pass carries one occurrence with an empty selector that describes the scan (how much text was eligible and computable), not an element. If you need to know which specific elements were checked and considered fine, that's not currently exposed — see OUTPUT_SCHEMA.md.

"A rule I expected to fire returned notApplicable / found nothing on a page I know has the issue"

Two common causes, in order of likelihood:

1. The element is excluded from the accessibility tree — aria-hidden="true", display: none, visibility: hidden, hidden, or an inert ancestor. Most rules skip content that's already invisible to assistive technology, since checking a hidden element would be meaningless and could produce a misleading fail on content no user encounters. Some rules explicitly opt out of this gating when it wouldn't make sense to (e.g. no-autoplay-audio — hidden audio still plays sound) — check the specific rule's file header comment (@applicability) in src/checks/.
2. excludeSelectors — if you've configured this (directly or inherited from a shared config), confirm the element in question isn't matched by it. Remember this can also be scoped to a single rule via engineOptions.rules[ruleId].excludeSelectors (see ENGINE_OPTIONS.md) — if a rule you expect to fire keeps coming back notApplicable/pass for one element only, check whether that rule specifically has its own exclude list configured, not just the global one.

"Every check passed, so why is each WCAG criterion cantTell?"

Since 1.14, a WCAG rollup in rulesResults passes only for a criterion whose rules check all it asks, and no criterion is listed as such yet (COMPLETE_CHECK in src/coverage/complete-applicability.js). Where every rule of a criterion passes, its rollup is cantTell with reasonCode composite.rollup.cantTell.partialCheck: the parts the rules check are met, and a person checks the rest. html-lang-attr-present finds that <html lang="fr"> names a language, not that the page is in French. Likewise, a criterion whose rules all found nothing is notApplicable only where they find every case it applies to, and otherwise cantTell with composite.rollup.cantTell.partialCoverage. A failing rule still fails its criterion. Show these as "checks passed, needs review" and "nothing found to check", never as passed; see WCAG conformance mapping.

"Does a clean scan (no failures) mean the page is WCAG conformant?"

No — see WCAG_CONFORMANCE.md. A rule's pass means the checks that ran came back clean on the part of the page they scanned. A scan scoped to part of a page, limited to some rules, or run under jsdom checks less than it could. A meaningful fraction of WCAG requires human judgment (accurate alt text, understandable error messages) or dynamic testing this engine's architecture can't do at all (keyboard traps, reflow at zoom) — see LIMITATIONS.md for the explicit, non-exhaustive-on-purpose list. What's left is a person's to check, and core's test procedures say how, criterion by criterion.

"Should I treat cantTell as a failure?"

Treat it as "needs a human to look" — it's neither pass nor fail by design. A common choice is to log cantTell findings without failing CI on them, since failing a build on something the engine explicitly couldn't determine tends to train people to ignore the gate. A team that wants every one reviewed before a merge can gate on them instead; either way, keep them apart from failures. In between, a build can gate only on the findings that lean fail, where the page's evidence points to a failure: leanFailAs in the JUnit and SARIF reporters, or leaningFailures(result) (see Leans). Each cantTell finding of a core rule says how to answer it in howToCheck. See POLICY.md if you want to reshape this behavior (e.g. via a custom policy contract), and INTEGRATION.md for a concrete CI-gating example.

"What happens if a locale is only partially translated?"

Missing keys fall back to English per-string (never a blank or broken result), so a partial locale degrades gracefully rather than failing outright. See I18N.md for the mechanism and current coverage. Key parity is enforced in core's own build rather than hoped for: every new or renamed English key is carried into every other locale file, holding the English text until someone translates it, so a locale can be behind on wording without ever being behind on keys.

"runDomRulesInPage vs runa11yCoreInPage — which one do I want?"

runDomRulesInPage if you're calling it directly in the same Node process (jsdom). runa11yCoreInPage if you're handing the function itself to a different JS realm — almost always page.evaluate in Puppeteer/Playwright, which serializes the function to source text and re-runs it inside the browser tab (which has no access to your Node module scope). See INTEGRATION.md for both patterns worked out in full.

"After upgrading, my baseline reports known findings as new and their entries as stale"

A finding's identity includes its reason code, and 1.14 gave codes to seventeen rules that had none and split a few codes by cause. A baseline written by 1.13 keeps matching those findings under 1.14.x, through src/baseline-moves.json, but not under 1.15, which dropped those moves: there an entry under an old code is stale and its finding is new. Rewrite the file's codes under 1.14.x with migrateBaselineEntries from @surea11y/core/baseline, then upgrade, or regenerate it under 1.15 and review the diff. 1.15 and 1.16 moved no code, so a baseline written by 1.14 keeps matching; a finding whose outcome changed with the upgrade is new or gone all the same, and the changelog says why. See Baseline / allowlist.

"css-orientation-lock asks about STYLESHEETS_NOT_READABLE"

A page's script can't read a style sheet another origin serves (a CDN, a fonts service), so the rules that read CSS can't tell what it holds. Give the scan its text in engineOptions.styleSheetTexts: preloadStyleSheets(), or runa11yCoreInPageAsync on the browser bundle, fetches those the server shares, and a tool driving the browser can fetch the others from outside the page. See "Style sheets from another origin" in INTEGRATION.md, the integration guide on GitHub.

"My results changed after upgrading to core 1.15"

1.15 fails a WCAG criterion only where WCAG fails, and reads frames as the browser shows them. The changes most likely to show in a pipeline:

• Fewer ARIA and HTML structure failures. Eight ARIA rules and the HTML structure rules no longer fail a criterion where only WAI-ARIA or the HTML content model is broken: those findings are cantTell, with uncertainty spec-only and steps to check. Affected: aria-role-name-present (grid, tree, radiogroup, meter, progressbar), dialog-name-present, tooltip-name-present, aria-allowed-attr, aria-valid-attr-value (except for a control's state, value or focus), aria-prohibited-attr, aria-roles-valid (except on a focusable element), aria-owns-valid (an ancestor; it maps to 1.3.1 only now), list-children-valid and listitem-parent-valid (except where the page draws the broken item), definition-list-children-valid, dlitem-parent-valid and nested-interactive-controls-absent (except inside a control with presentational children). Every finding keeps its reason code, so baselines still match, but a gate or a rollup that only these failed now passes or is cantTell. To judge the same findings by WAI-ARIA, where they fail, add the WAI-ARIA pack.
• A framed document is no page of its own. In a frame, page-title-present, html-lang-attr-present, the viewport, bypass and landmark rules are notApplicable (see the entry above). A frame under aria-hidden or inert is read as such, and one behind an open modal dialog is left out.
• Roles follow WAI-ARIA more closely. role="text" names no valid role; an image with a presentational role and a global ARIA attribute keeps its role, and fails img-alt-present without alt; the four globals WAI-ARIA 1.2 deprecates no longer give a presentational element its role back.
• A skipped heading level is a best practice issue, not a 1.3.1 failure, in the reviewed test procedures, which also give the "How to check" steps of six new questions.
• A baseline from 1.13 needs updating first. 1.15 moved no reason code, and dropped the moves from 1.13 to 1.14 (see the entry above).

New, and changing nothing unless you use it: scans of several profiles at once (Engine options), the wcag22-a and wcag22-aaa profiles, test procedures for assessing by hand, and the HTML report's "What's left for a person". The full list is in core's CHANGELOG.md.