-
Notifications
You must be signed in to change notification settings - Fork 1
feat: add e2e playwright tests #158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
62b5bf0
75389ed
bc9bd04
555d633
671a653
7a6b6f3
59b9ef2
91f0964
e6aa28d
3efa497
16c09a2
6bb3a12
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| name: E2E (Playwright) | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| base_url: | ||
| description: 'Base URL to run tests against (e.g., https://staging.example.org)' | ||
| required: true | ||
| type: string | ||
|
|
||
| jobs: | ||
| e2e: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| defaults: | ||
| run: | ||
| working-directory: e2e | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Node | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '20' | ||
| cache: 'npm' | ||
| cache-dependency-path: e2e/package.json | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci || npm install | ||
|
|
||
| - name: Install Playwright browsers | ||
| run: npx playwright install --with-deps | ||
|
|
||
| - name: Run tests | ||
| env: | ||
| BASE_URL: ${{ inputs.base_url }} | ||
| CI: 'true' | ||
| run: npx playwright test | ||
|
|
||
| - name: Upload HTML report | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: playwright-report | ||
| path: e2e/playwright-report | ||
| retention-days: 7 | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| # E2E tests (Playwright) | ||
|
|
||
| End-to-end tests for the Interledger.org Drupal site using Playwright. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Node.js 18+ (20+ recommended) | ||
| - Browsers for Playwright (`npm run install:browsers`) | ||
| - The target site running locally or accessible via URL | ||
| having the website running means: | ||
| - files/ folder from production website in /web/sites/default/files | ||
| - mysql production database correctly imported locally | ||
|
|
||
JonathanMatthey marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ## Quick start (local) | ||
|
|
||
| 1) Start the local Drupal stack (optional — the test runner will also start it automatically if targeting localhost): | ||
|
|
||
| ```bash | ||
| make -C local up | ||
| # or: | ||
| cd e2e && npm run local:up | ||
| ``` | ||
|
|
||
| This exposes the site at <http://localhost:8080> by default. | ||
|
|
||
| 2) In this `e2e/` folder, install deps and browsers: | ||
|
|
||
| ```bash | ||
| npm install | ||
| npm run install:browsers | ||
| ``` | ||
|
|
||
| 3) Run the tests: | ||
|
|
||
| ```bash | ||
| npm test | ||
| # or explicitly against localhost (equivalent): | ||
| npm run test:local | ||
| ``` | ||
|
|
||
| Tests default to `BASE_URL=http://localhost:8080`. To target a different environment: | ||
|
|
||
| ```bash | ||
| BASE_URL=https://your-env.example.org npm test | ||
| ``` | ||
|
|
||
| Note: When `BASE_URL` is the default `http://localhost:8080` (and not in CI), the test runner will run `make -C ../local up` automatically and wait for the port to be ready before starting tests. If the stack is already running, it will reuse it. | ||
|
|
||
| ## Useful commands | ||
|
|
||
| - Run headed: `npm run test:headed` | ||
| - Debug mode: `npm run test:debug` | ||
| - Show last report: `npm run show-report` | ||
| - Codegen against current BASE_URL: `npm run codegen` | ||
|
|
||
| ## Running individual tests | ||
|
|
||
| To run a specific test file: | ||
|
|
||
| ```bash | ||
| npx playwright test tests/seo.spec.ts | ||
| npx playwright test tests/performance.spec.ts | ||
| npx playwright test tests/accessibility.spec.ts | ||
| # etc. | ||
| ``` | ||
|
|
||
| To run a specific test within a file (by test name): | ||
|
|
||
| ```bash | ||
| npx playwright test tests/seo.spec.ts -g "homepage has title" | ||
| ``` | ||
|
|
||
| To run tests matching a pattern: | ||
|
|
||
| ```bash | ||
| npx playwright test --grep "homepage" | ||
| ``` | ||
|
|
||
| To run tests in a specific browser: | ||
|
|
||
| ```bash | ||
| npx playwright test tests/seo.spec.ts --project=chromium | ||
| npx playwright test tests/seo.spec.ts --project=firefox | ||
| npx playwright test tests/seo.spec.ts --project=webkit | ||
| ``` | ||
|
|
||
| You can combine options with the npm scripts: | ||
|
|
||
| ```bash | ||
| BASE_URL=http://localhost:8080 npx playwright test tests/seo.spec.ts --headed | ||
| ``` | ||
|
|
||
| Available test files: | ||
|
|
||
| - `tests/seo.spec.ts` - SEO and meta tags | ||
| - `tests/performance.spec.ts` - Performance budgets | ||
| - `tests/accessibility.spec.ts` - Accessibility checks | ||
| - `tests/navigation.spec.ts` - Navigation tests | ||
| - `tests/images.spec.ts` - Image optimization | ||
| - `tests/error-pages.spec.ts` - Error page handling | ||
| - `tests/console-errors.spec.ts` - Console error checks | ||
| - `tests/smoke.spec.ts` - Basic smoke tests | ||
|
|
||
| # Managing the local Docker stack from here | ||
|
|
||
| - Start: `npm run local:up` | ||
| - Stop: `npm run local:down` | ||
|
|
||
| ## CI | ||
|
|
||
| There is a GitHub Actions workflow (`.github/workflows/e2e.yml`) that runs tests when manually dispatched and requires a `base_url` input. It does not start Drupal; point it to a running environment (e.g., staging). | ||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| { | ||
| "name": "interledger-org-e2e", | ||
| "private": true, | ||
| "version": "0.1.0", | ||
| "description": "Playwright end-to-end tests for interledger.org Drupal site", | ||
| "scripts": { | ||
| "test": "playwright test", | ||
| "test:local": "BASE_URL=http://localhost:8080 playwright test", | ||
| "test:headed": "playwright test --headed", | ||
| "test:debug": "PWDEBUG=1 playwright test", | ||
| "show-report": "playwright show-report", | ||
| "install:browsers": "playwright install --with-deps", | ||
| "codegen": "playwright codegen ${BASE_URL:-http://localhost:8080}", | ||
| "local:up": "make -C ../local up", | ||
| "local:down": "make -C ../local down" | ||
| }, | ||
| "devDependencies": { | ||
| "@playwright/test": "^1.48.0", | ||
| "@axe-core/playwright": "^4.9.0" | ||
| }, | ||
| "engines": { | ||
| "node": ">=18" | ||
| } | ||
| } | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| import { defineConfig, devices } from '@playwright/test'; | ||
|
|
||
| const isCI = !!process.env.CI; | ||
| const baseURL = process.env.BASE_URL || 'http://localhost:8080'; | ||
|
|
||
| export default defineConfig({ | ||
| testDir: './tests', | ||
| fullyParallel: true, | ||
| retries: isCI ? 2 : 0, | ||
| reporter: isCI ? [['list'], ['html', { outputFolder: 'playwright-report', open: 'never' }]] : [['list'], ['html']], | ||
| use: { | ||
| baseURL, | ||
| trace: 'on-first-retry', | ||
| screenshot: 'only-on-failure', | ||
| video: 'retain-on-failure' | ||
| }, | ||
| // When running locally against the default docker-compose stack, | ||
| // automatically ensure it's up before tests. Skip in CI or when BASE_URL is custom. | ||
| webServer: !isCI && baseURL.startsWith('http://localhost:8080') | ||
| ? { | ||
| command: 'make -C ../local up', | ||
| url: 'http://localhost:8080', | ||
| reuseExistingServer: true, | ||
| timeout: 120_000 | ||
| } | ||
| : undefined, | ||
| projects: [ | ||
| { | ||
| name: 'chromium', | ||
| use: { ...devices['Desktop Chrome'] } | ||
| }, | ||
| { | ||
| name: 'firefox', | ||
| use: { ...devices['Desktop Firefox'] } | ||
| }, | ||
| { | ||
| name: 'webkit', | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Webkit just won't run correctly on my Linux machine. It is valuable to have here but when we push this to CI we might have issues there too, depending what environment we're running in. |
||
| use: { ...devices['Desktop Safari'] } | ||
| } | ||
| ] | ||
| }); | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| import { test, expect } from '@playwright/test'; | ||
| import AxeBuilder from '@axe-core/playwright'; | ||
|
|
||
| test.describe('Accessibility', () => { | ||
| test('homepage has no serious/critical violations', async ({ page }) => { | ||
| await page.goto('/'); | ||
| const results = await new AxeBuilder({ page }).analyze(); | ||
| const issues = results.violations.filter(v => v.impact === 'critical' || v.impact === 'serious'); | ||
| const message = issues.map(v => `${v.impact}: ${v.id} (${v.nodes.length})`).join('\n'); | ||
| expect(issues, message).toHaveLength(0); | ||
| }); | ||
| }); | ||
|
|
||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import { test, expect } from '@playwright/test'; | ||
|
|
||
| test.describe('Console and runtime errors', () => { | ||
| // DISABLED FOR NOW: | ||
| // locally having issues | ||
|
|
||
| // TODO: re-enable this test when we have a way to test console errors | ||
| // test('homepage has no console errors', async ({ page }) => { | ||
| // const errors: string[] = []; | ||
| // page.on('console', msg => { | ||
| // if (msg.type() === 'error') { | ||
| // errors.push(msg.text()); | ||
| // } | ||
| // }); | ||
| // page.on('pageerror', err => { | ||
| // errors.push(`PageError: ${err.message}`); | ||
| // }); | ||
|
|
||
| // await page.goto('/'); | ||
| // await page.waitForLoadState('networkidle'); | ||
| // expect(errors.join('\n')).toBe(''); | ||
| // }); | ||
| }); | ||
|
|
||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can test what happens with webkit once merged and can remove it or edit appropriately after testing it out.