Introduce Functional Tests using Playwright and Behave

.github/workflows/ci.yml

@@ -147,3 +147,53 @@ jobs:
       - uses: actions/checkout@v4
       - name: Build Docker Image
         run: docker build --target web -t ons .
+
+  functional-tests:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        browser:
+          - chromium
+    needs:
+      - lint
+      - lint-front-end
+      - compile_static
+
+    env:
+      DJANGO_SETTINGS_MODULE: cms.settings.functional_test
+      SECRET_KEY: fake_secret_key_to_run_tests # pragma: allowlist secret
+      PLAYWRIGHT_BROWSER: ${{ matrix.browser }}
+      PLAYWRIGHT_TRACES_DIR: playwright_traces
+      PLAYWRIGHT_TRACE: 'true'
+
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install Poetry
+        run: pipx install poetry==${{ env.POETRY_VERSION }}
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version-file: .python-version
+          cache: poetry
+
+      - name: Install dependencies
+        run: make install-dev
+
+      - name: Playwright Install
+        run: poetry run python -m playwright install --with-deps ${{ matrix.browser }}
+
+      - uses: actions/download-artifact@v4
+        with:
+          name: static
+          path: cms/static_compiled/
+
+      - name: Run Functional Tests
+        run: make functional-tests
+
+      - name: Upload Failure Traces
+        uses: actions/upload-artifact@v4
+        if: failure()
+        with:
+          name: playwright-traces-${{ matrix.browser }}
+          path: playwright_traces/

.gitignore

@@ -190,3 +190,6 @@ static/
 
 # Media files
 /media
+
+# Playwright Traces
+/tmp_traces

Makefile

@@ -152,3 +152,28 @@ runserver: ## Run the Django application locally
 
 .PHONY: dev-init
 dev-init: load-design-system-templates collectstatic makemigrations migrate createsuperuser ## Run the pre-run setup scripts
+
+.PHONY: functional-tests-up
+functional-tests-up:  ## Start the functional tests docker compose dependencies
+	docker compose -f functional_tests/docker-compose.yml up -d
+
+.PHONY: functional-tests-dev-up
+functional-tests-dev-up:  ## Start the functional tests docker compose dependencies and dev app
+	docker compose -f functional_tests/docker-compose-dev.yml up -d
+
+.PHONY: functional-tests-down
+functional-tests-down:  ## Stop the functional tests docker compose dependencies (and dev app if running)
+	docker compose -f functional_tests/docker-compose-dev.yml down
+
+.PHONY: functional-tests-run
+functional-tests-run: load-design-system-templates collectstatic ## Only run the functional tests (dependencies must be run separately)
+	# Run migrations to work around Django bug (#35967)
+	poetry run ./manage.py migrate --noinput --settings cms.settings.functional_test
+	poetry run behave functional_tests
+
+.PHONY: functional-tests
+functional-tests: functional-tests-up functional-tests-run functional-tests-down  ## Run the functional tests with dependencies (all in one)
+
+.PHONY: playwright-install
+playwright-install:  ## Install Playwright dependencies
+	poetry run playwright install --with-deps

README.md

@@ -28,6 +28,7 @@ The Wagtail CMS for managing and publishing content for the Office for National
     -   [Front-end tooling](#front-end-tooling)
     -   [Adding Python packages](#adding-python-packages)
     -   [Run Tests with Coverage](#run-tests-with-coverage)
+    -   [Functional Tests](#functional-tests)
     -   [Linting and Formatting](#linting-and-formatting)
         -   [Python](#python)
         -   [Front-end](#front-end)
@@ -36,6 +37,7 @@ The Wagtail CMS for managing and publishing content for the Office for National
     -   [Django Migrations](#django-migrations)
 -   [Contributing](#contributing)
 -   [License](#license)
+
 <!-- markdown-link-check-enable -->
 
 For further developer documentation see [docs](docs/index.md)
@@ -52,9 +54,12 @@ Ensure you have the following installed:
    managing Python versions.
 2. **[Poetry](https://python-poetry.org/)**: This is used to manage package dependencies and virtual
    environments.
-3. **[Colima](https://github.com/ONSdigital/dp-compose/blob/main/setting-up-colima-locally.md)** for running the project in Docker containers.
-4. **[PostgreSQL](https://www.postgresql.org/)** for the database. Provided as container via `docker-compose.yml` when using the Docker setup.
-5. **[Node](https://nodejs.org/en)** and **[`nvm` (Node Version Manager)](https://github.com/nvm-sh/nvm)** for front-end tooling.
+3. **[Colima](https://github.com/ONSdigital/dp-compose/blob/main/setting-up-colima-locally.md)** for running the project
+   in Docker containers.
+4. **[PostgreSQL](https://www.postgresql.org/)** for the database. Provided as container via `docker-compose.yml` when
+   using the Docker setup.
+5. **[Node](https://nodejs.org/en)** and **[`nvm` (Node Version Manager)](https://github.com/nvm-sh/nvm)** for front-end
+   tooling.
 6. **[JQ](https://jqlang.github.io/jq/)** for the step in the build that installs the design system templates
 7. **Operation System**: Ubuntu/MacOS
 
@@ -119,8 +124,8 @@ Follow these steps to set up and run the project using Docker.
 
     Once the containers are running, you need to manually start Django from within the web container. This allows for running both the Django server and any additional background services (e.g., schedulers).
 
-    > ⚠️ WARNING  
-    >  The `honcho` command will pick up your local mounted `.env` file when running via `docker-compose`. Ensure that you comment out any variables in the `.env` file which might cause clashes in the container context as they will take precedence when running `honcho start`.
+    > ⚠️ WARNING
+    > The `honcho` command will pick up your local mounted `.env` file when running via `docker-compose`. Ensure that you comment out any variables in the `.env` file which might cause clashes in the container context as they will take precedence when running `honcho start`.
 
     ```bash
     # Start both Django and the scheduler using Honcho
@@ -164,6 +169,7 @@ make compose-dev-up
 -   generating and applying database migrations,
 -   creating a superuser with:
     -   username: `admin`
+    <!-- pragma: allowlist nextline secret) -->
     -   password: `changeme`
 
 ```
@@ -225,7 +231,69 @@ To run the tests and check coverage, run:
 make test
 ```
 
-During tests, the `cms.settings.test` settings module is used. When running test without using `make test`, ensure this settings module is used.
+During tests, the `cms.settings.test` settings module is used. When running test without using `make test`, ensure this
+settings module is used.
+
+### Functional Tests
+
+Our suite of functional browser driven tests uses [Behave](https://behave.readthedocs.io/en/latest/),
+[Playwright](https://playwright.dev/python/docs/intro) and
+[Django Live Server Test Cases](https://docs.djangoproject.com/en/stable/topics/testing/tools/#liveservertestcase) to
+run BDD Cucumber feature tests against the app from a browser.
+
+#### Installation
+
+Install the Playwright dependencies (including its browser drivers) with:
+
+```shell
+make playwright-install
+```
+
+#### Run the Functional Tests
+
+You can run the tests as an all-in-one command with:
+
+```shell
+make functional-tests
+```
+
+This will start and stop the docker compose services with the relevant tests.
+
+To run the docker compose dependencies (database and redis) separately, e.g. if you want to run individual functional
+tests yourself for development, start the docker compose dependencies with:
+
+```shell
+make functional-tests-up
+```
+
+This will start the dependent services in the background, allowing you to then run the tests separately.
+
+Then once you are finished testing, stop the dependencies with:
+
+```shell
+make functional-tests-down
+```
+
+#### Showing the Tests Browser
+
+By default, the tests will run in headless mode with no visible browser window.
+
+To disable headless mode and show the browser, set `PLAYWRIGHT_HEADLESS=False` in the environment from which you are
+running the tests. In this circumstance, you will probably also find it helpful to enable "slow mo" mode, which slows
+down the automated browser interactions to make it possible to follow what the tests are doing. You can configure it
+using the `PLAYWRIGHT_SLOW_MO` environment variable, passing it a value of milliseconds by which to slow each
+interaction, e.g. `PLAYWRIGHT_SLOW_MO=1000` will cause each individual browser interaction from the tests to be delayed
+by 1 second.
+
+For example, you can run the tests with visible browser and each interaction slowed by a second by running:
+
+```shell
+PLAYWRIGHT_HEADLESS=False PLAYWRIGHT_SLOW_MO=1000 make functional-tests
+```
+
+#### Developing Functional Tests
+
+Refer to the detailed [functional tests development docs](./functional_tests/README.md)
 
 ### Linting and Formatting
 

cms/settings/functional_test.py

@@ -0,0 +1,10 @@
+import copy
+
+from .test import *  # noqa: F403  # pylint: disable=wildcard-import,unused-wildcard-import
+
+DATABASES = {
+    "default": dj_database_url.config(default="postgres://ons:ons@localhost:15432/ons"),  # noqa: F405
+}
+DATABASES["read_replica"] = copy.deepcopy(DATABASES["default"])
+
+REDIS_URL = "redis://localhost:16379"

cms/users/tests/factories.py

@@ -13,7 +13,7 @@ class Meta:
 
     @factory.post_generation
     def access_admin(self, create, extracted, **kwargs):
-        """Creates BundlePage instances for the bundle.
+        """Allows the group access to the Wagtail Admin interface.
 
         Usage:
             # Create a Django generic_user group
@@ -42,13 +42,13 @@ class Meta:
 
     @factory.post_generation
     def access_admin(self, create, extracted, **kwargs):
-        """Creates BundlePage instances for the bundle.
+        """Allows the user access to the Wagtail Admin interface.
 
         Usage:
-            # Create a Django generic_user group
+            # Create a Django generic_user user
             generic_user = UserFactory()
 
-            # Create a Django generic_user group with Wagtail admin access
+            # Create a Django generic_user user with Wagtail admin access
             generic_user = UserFactory(access_admin=True)
         """
         if not create: