Accessibility feature tests

When to add accessibility tests

Feature tests with axe-core-gem provide the most comprehensive accessibility testing approach, covering complete user journeys across all UI technologies (HAML, Vue, and JavaScript).

Prioritize accessibility tests for:

Mission-critical user journeys: core workflows that users depend on.
High-traffic pages: areas with significant user interaction.
New features: Ensure accessibility from the start.
Complex UI interactions: Multi-step processes, modals, dynamic content.

Strategic approach

Rather than testing every possible page combination, focus on deep coverage of key user scenarios. This approach provides better value than broad but shallow coverage across all views.

One of the advantages of testing in feature tests is that we can check different states and complete user flows, not only single components in isolation.

You can find some examples on how to approach accessibility checks below.

Empty state

Some views have an empty state that result in a page structure that’s different from the default view. They may also offer some actions, for example to create a first issue or to enable a feature. In this case, add assertions for both an empty state and a default view.

Ensure compliance before user interactions

Often we test against a number of steps we expect our users to perform. In this case, make sure to include the check early on, before any of them has been simulated. This way we ensure there are no barriers to what we expect of users.

Ensure compliance after changed page structure

User interactions may result in significant changes in page structure. For example, a modal is shown, or a new section is rendered. In that case, add an assertion after any such change. We want to make sure that users are able to interact with all available components.

How to add accessibility tests

Create new spec file

We want automated accessibility tests to follow already defined user journeys. To achieve this we are reusing test cases defined for E2E tests.

To add a new accessibility spec for your team, you can:

Browse the list of test cases for your team by either:

Looking through Test Cases page, which you can filter by your group’s label.
Navigating to the E2E browser_ui directory and ee/browser_ui directory, then selecting the director for your stage and a feature you want to cover.

Once you know the user journey you want to cover:

Navigate to spec/features/accessibility.
Create a new Ruby spec under the folder for your stage and the feature you are covering, for example create/repository/.
Name your file after E2E test case you are following, for example: add_new_branch_rule_spec.rb.

In this example, the result will be a dedicated feature spec under spec/features/accessibility/create/repository/add_new_branch_rule_spec.rb.

The next step is recreating the test cases with the Capybara feature tests syntax and setup.

Using axe methods

Axe provides the custom matcher be_axe_clean, which can be used like the following:

# spec/features/accessibility/create/repository/add_new_branch_rule_spec.rb
it 'passes axe automated accessibility testing', :js do
  visit_settings_page

  wait_for_requests # ensures page is fully loaded

  expect(page).to be_axe_clean
end

If needed, you can scope testing to a specific area of the page by using within.

Axe also provides specific clauses, for example:

expect(page).to be_axe_clean.within '[data-testid="element"]'

# run only WCAG 2.1 Level AA rules
expect(page).to be_axe_clean.according_to :wcag21aa

# specifies which rule to skip
expect(page).to be_axe_clean.skipping :'link-in-text-block'

# clauses can be chained
expect(page).to be_axe_clean.within('[data-testid="element"]')
                            .according_to(:wcag21aa)

Axe does not test hidden regions, such as inactive menus or modal windows. To test hidden regions for accessibility, write tests that activate or render the regions visible and run the matcher again.

You can run accessibility tests locally in the same way as you run any feature tests.

After adding accessibility tests, make sure to fix all possible errors. For help on how to do it, refer to this guide. You can also check accessibility sections in Pajamas components’ documentation. If any of the errors require global changes, create a follow-up issue and assign these labels: accessibility, WG::product accessibility.

Good practices

Adding accessibility checks in feature tests is easier if you have domain knowledge from the product area in question. However, there are a few things that can help you contribute to accessibility tests.

What parts of a page to add accessibility tests for

In most cases you do not want to test accessibility of a whole page. There are a couple of reasons:

We have elements that appear on every application view, such as breadcrumbs or main navigation. Including them in every feature spec takes up quite a lot of resources and multiplies something that can be done just once. These elements have their own feature specs and that’s where we want to test them.
If a feature spec covers a whole view, the best practice would be to scope it to <main id="content-body"> element. Here’s an example of such test case:
```
 it "passes axe automated accessibility testing" do
   expect(page).to be_axe_clean.within('#content-body')
 end
```
If a specific test case covers only a part of a page, like a section that includes some components, keep the test scoped to that section. Here’s an example of such test case:
```
 it 'passes axe automated accessibility testing for todo' do
   expect(page).to be_axe_clean.within(todo_selector)
 end
```

Test output not specific enough

When axe test case fails, it outputs the violation found and an element that it concerns. Because we often use Pajamas Components, it may happen that the element will be a <div> without any annotation that could help you identify it. However, we can take advantage of a fact that axe_core rules is used both for Ruby tests and Deque browser extension - axe devTools. They both provide the same output.

Make sure you have axe DevTools extension installed in a browser of your choice. See axe DevTools official website for more information.
Navigate to the view you’re testing with a feature test.
Open axe DevTools extension and run a scan of the page.
Expand found issues and use Highlight option to see the elements on the page for each violation.

Known accessibility violations

This section documents violations where a recommendation differs with the design system:

link-in-text-block: For now, use the skipping clause to skip :'link-in-text-block' rule to fix the violation. After this is fixed as part of issue 1444 and underline is added to the GlLink component, this clause can be removed.