Skip to content

Commit 9b320be

Browse files
jamiefolsomjamiefolsom
committed
Improve a11y test suite and add CI workflow
- Fix path detail URL bug (posts/ → paths/) in a11y.test.ts - Filter axe-core to WCAG 2.0/2.1/2.2 AA rules only - Use soft assertions so all violations surface per page - Add manual-trigger GitHub Actions workflow (.github/workflows/a11y.yml) - Add test results report (docs/a11y-test-results.md) - Update CLAUDE.md with multi-tenant env var and TinaCMS gotchas
1 parent 20fa2bc commit 9b320be

3 files changed

Lines changed: 143 additions & 3 deletions

File tree

‎.github/workflows/a11y.yml‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
name: Accessibility Tests
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
url:
7+
description: 'URL to test (e.g., https://uss-staging.netlify.app). Leave empty to build and test locally.'
8+
required: false
9+
type: string
10+
11+
jobs:
12+
a11y:
13+
runs-on: ubuntu-latest
14+
timeout-minutes: 30
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- uses: actions/setup-node@v4
20+
with:
21+
node-version: 22
22+
cache: npm
23+
24+
- name: Install dependencies
25+
run: npm ci
26+
27+
- name: Install Playwright chromium
28+
run: npx playwright install --with-deps chromium
29+
30+
# When testing a deployed URL, just fetch config for test scaffolding
31+
- name: Fetch config
32+
if: ${{ inputs.url != '' }}
33+
env:
34+
CONFIG_URL: ${{ secrets.CONFIG_URL }}
35+
run: curl -fsSL "$CONFIG_URL" -o public/config.json
36+
37+
# When no URL provided, do a full static build
38+
- name: Build static site
39+
if: ${{ inputs.url == '' }}
40+
env:
41+
STATIC_BUILD: 'true'
42+
CONFIG_URL: ${{ secrets.CONFIG_URL }}
43+
GITHUB_OWNER: ${{ secrets.CONTENT_GITHUB_OWNER }}
44+
GITHUB_REPO: ${{ secrets.CONTENT_GITHUB_REPO }}
45+
GITHUB_BRANCH: ${{ secrets.CONTENT_GITHUB_BRANCH }}
46+
GITHUB_PERSONAL_ACCESS_TOKEN: ${{ secrets.CONTENT_GITHUB_PERSONAL_ACCESS_TOKEN }}
47+
MONGODB_URI: ${{ secrets.MONGODB_URI }}
48+
MONGODB_NAME: ${{ secrets.MONGODB_NAME }}
49+
MONGODB_COLLECTION_NAME: ${{ secrets.MONGODB_COLLECTION_NAME }}
50+
NEXTAUTH_SECRET: ${{ secrets.NEXTAUTH_SECRET }}
51+
run: npm run build
52+
53+
- name: Start preview server
54+
if: ${{ inputs.url == '' }}
55+
run: npx astro preview &
56+
57+
- name: Wait for server
58+
if: ${{ inputs.url == '' }}
59+
run: |
60+
for i in $(seq 1 30); do
61+
curl -sf http://localhost:4321 > /dev/null 2>&1 && exit 0
62+
sleep 1
63+
done
64+
echo "Server failed to start" && exit 1
65+
66+
- name: Run accessibility tests
67+
env:
68+
A11Y_HOST: ${{ inputs.url || 'http://localhost:4321' }}
69+
STATIC_BUILD: ${{ inputs.url != '' && 'false' || 'true' }}
70+
run: npx playwright test --project=chromium
71+
72+
- name: Upload report
73+
if: always()
74+
uses: actions/upload-artifact@v4
75+
with:
76+
name: a11y-report
77+
path: |
78+
playwright-report/
79+
test-results/
80+
retention-days: 30

‎docs/a11y-test-results.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Accessibility Test Results
2+
3+
**Date:** 2026-02-19
4+
**Engine:** axe-core 4.11 via `@axe-core/playwright`
5+
**Ruleset:** WCAG 2.0/2.1/2.2 Level A and AA only
6+
**Browser:** Chromium
7+
**Target:** USS content deployment (localhost:4321)
8+
9+
## Summary
10+
11+
| Status | Count |
12+
|--------|-------|
13+
| Passed | 6 |
14+
| Failed | 1 |
15+
| Skipped | 7 |
16+
17+
**1 unique WCAG violation** found. The test suite filters to WCAG 2.2 AA criteria and uses soft assertions to report all violations per page without stopping early.
18+
19+
## WCAG Violations
20+
21+
### Critical: `image-alt` — Images must have alternative text
22+
23+
- **WCAG:** 1.1.1 Non-text Content (Level A)
24+
- **Issue:** [#553](https://github.com/performant-software/core-data-places/issues/553)
25+
- **Pages:** `/en/pages/Institutions`, `/en/pages/About`, `/en/` (Home, intermittent due to `server:defer` timing)
26+
- **Element:** `<img>` in Banner component (`src/apps/pages/Banner.astro` line 52)
27+
- **Root cause:** `Banner.astro` uses `alt={imageAlt}`, but when TinaCMS content omits the `imageAlt` field, the `alt` attribute is undefined (missing entirely).
28+
- **Fix:** Default to `alt={imageAlt || ''}` for decorative images, or require alt text in the CMS schema.
29+
30+
## Best Practice Issues (Not WCAG Criteria)
31+
32+
These were found during initial testing with all axe-core rules enabled. They are not WCAG violations but are worth addressing:
33+
34+
- **`page-has-heading-one`** — 6 pages lack a visible `<h1>` at scan time because the heading is inside `Header.astro` with `server:defer`
35+
- **`region`** — Header and footer use `<div>` instead of semantic landmarks (`<header>`, `<nav>`, `<footer>`)
36+
37+
## Skipped Tests
38+
39+
These were skipped because the USS config doesn't include the relevant features:
40+
41+
- Paths page / Path detail (no paths collection)
42+
- Search table view / filters panel / detail panel (search type is `list`, not `map`)
43+
- Gallery / Gallery item (no gallery URL configured)
44+
45+
## Test Changes Made
46+
47+
1. **Bug fix:** `test/browser/a11y.test.ts:59` — changed `posts/` to `paths/` for path detail page URLs
48+
2. **WCAG filter:** `checkPage` now uses `.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22a', 'wcag22aa'])` to scan only WCAG criteria
49+
3. **Soft assertions:** `expect.soft()` records all violations without stopping, so every page gets fully scanned
50+
51+
## CI Workflow
52+
53+
`.github/workflows/a11y.yml` — manual trigger (`workflow_dispatch`), chromium only, HTML report uploaded as artifact.
54+
55+
Two modes:
56+
- **With URL input:** tests any deployed URL (staging, production, PR preview)
57+
- **Without URL:** builds static site locally and serves with `astro preview`

‎test/browser/a11y.test.ts‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ test.describe('Accessibility testing', () => {
5656
const paths = await fetchPaths();
5757

5858
for (const path of paths) {
59-
await page.goto(`posts/${path._sys.filename}`);
59+
await page.goto(`paths/${path._sys.filename}`);
6060
await checkPage(page);
6161
}
6262
});
@@ -198,6 +198,9 @@ test.describe('Accessibility testing', () => {
198198
* @param message
199199
*/
200200
const checkPage = async (page: Page, message: string = '') => {
201-
const accessibilityScanResults = await new AxeBuilder({ page }).analyze();
202-
expect(accessibilityScanResults.violations, message || page.url()).toEqual([]);
201+
const accessibilityScanResults = await new AxeBuilder({ page })
202+
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22a', 'wcag22aa'])
203+
.analyze();
204+
205+
expect.soft(accessibilityScanResults.violations, message || page.url()).toEqual([]);
203206
};

0 commit comments

Comments
 (0)