Merge pull request #1624 from NASA-AMMOS/dev/gh-test-workflows

Run backend e2e tests with given gateway docker tag when provided in PR body, add dev docs

.github/workflows/test.yml

@@ -5,6 +5,9 @@ on:
     branches:
       - develop
       - dev-[0-9]+.[0-9]+.[0-9]+
+    types:
+      # defaults + 'edited', to handle changes to flags like ___REQUIRES_GATEWAY_PR___ in the PR body
+      [ opened, synchronize, reopened, edited ]
   push:
     branches:
       - develop
@@ -27,6 +30,7 @@ env:
   SCHEDULER_PASSWORD: "${{secrets.SCHEDULER_PASSWORD}}"
   SEQUENCING_USERNAME: "${{secrets.SEQUENCING_USERNAME}}"
   SEQUENCING_PASSWORD: "${{secrets.SEQUENCING_PASSWORD}}"
+  PR_BODY: "${{github.event.pull_request.body}}"
 
 jobs:
   unit-test:
@@ -62,6 +66,12 @@ jobs:
     steps:
       - name: Checkout Repo
         uses: actions/checkout@v4
+      - name: Extract Aerie gateway docker tag from PR body
+        # look in the PR body for the string ___REQUIRES_GATEWAY_PR___=9999, extract the number if so & save to env var
+        # if gateway PR is labeled correctly, it will publish a docker tag called 'pr-9999' to use in tests
+        if: ${{ contains(env.PR_BODY, '___REQUIRES_GATEWAY_PR___=') }}
+        run: |
+          echo "GATEWAY_IMAGE_TAG=pr-$(echo $PR_BODY | sed -n 's/.*___REQUIRES_GATEWAY_PR___=\"\([0-9]\+\)\".*/\1/p')" >> $GITHUB_ENV
       - name: Setup Java
         uses: actions/setup-java@v4
         with:
@@ -81,9 +91,10 @@ jobs:
         run: ./gradlew e2e-tests:buildAllSchedulingProcedureJars
       - name: Start Services
         run: |
-          docker compose -f ./e2e-tests/docker-compose-test.yml up -d --build
+          echo "GATEWAY_IMAGE_TAG: $GATEWAY_IMAGE_TAG"
+          docker compose -f ./e2e-tests/docker-compose-test.yml up -d --build --quiet-pull
           docker images
-          docker ps -a
+          docker ps -a --no-trunc
       - name: Sleep for 30 Seconds
         run: sleep 30s
         shell: bash

docs/CONTRIBUTING.md

@@ -107,3 +107,30 @@ After your pull request is merged, you can safely delete your branch and pull th
   ```shell
   git branch -D my-fix-branch
   ```
+
+## <a name="submit-pr"></a> Required Checks on PRs
+
+When you submit a PR, we use Github Workflows to automatically run a number of CI-type checks on it, all of which must pass before the PR can be merged. These vary slightly from one repo to another, but mostly consist of unit tests, as well as end-to-end (e2e) tests for the backend and frontend (`aerie` and `aerie-ui`) repos. If your PR adds new tests to our existing test suites, they will automatically be run & validated with your PR.
+
+You can see the status of these checks at the bottom of the PR on Github - it should say "All checks have passed" if all is well. If it says "Some checks are not successful", you'll need to troubleshoot the cause of the failure before your PR can be accepted. Click "show all checks", then "details" to get detailed server logs from every step of the GH action that failed. Note that some of these runs may also have *files attached* to the run, which contain more detailed logs.
+
+If you can't determine the cause of the failure from logs alone, it may be helpful to try and reproduce the problem locally. You can see the commands run by the Required Check actions by looking in each repo under `.github/workflows` - each `yml` file in this folder contains the setup & commands necessary to run the check locally yourself.
+
+## <a name="submit-pr"></a> Running E2E Tests with specific branches/images
+
+Both `aerie` and `aerie-ui` repos contain **end-to-end (E2E) tests** which depend on code from other Aerie repos. Specifically:
+- The `aerie` backend e2e test suite, which tests a fully running Aerie API, depends on `aerie-gateway` since it is part of the API
+- The `aerie-ui` e2e suite, which tests a full Aerie UI + backend, depends on `aerie` and `aerie-gateway` code since it is a full stack test
+
+Sometimes a new feature requires code changes in *multiple repos simultaneously*, eg. a UI PR may add a new UI element that displays a new database field, but relies on a corresponding backend PR to add the new field. In this case, the UI PR's e2e tests will fail unless it has a way to *specify* which backend PR(s)/branches should be used for the e2e test.
+
+To handle this situation, follow these steps with your PRs:
+1. When you submit an `aerie` or `aerie-gateway` PR, if your PR contains code that *other PRs depend on*, add the GH label **"publish"** to the PR. This will cause an additional workflow to run which *publishes* the code in that branch to a docker image called eg. `pr-9999` for use by other PRs.
+2. When you submit an `aerie-ui` or `aerie` PR, if it *depends on other PR(s)* (aerie or gateway) for its tests to pass, edit the **body of your PR** to contain one or both of the following lines:
+```
+___REQUIRES_AERIE_PR___="9999"
+___REQUIRES_GATEWAY_PR___="9999"
+```
+Replace "9999" with the **PR number(s)** of the relevant `aerie`/`gateway` PR(s). PRs to `aerie-ui` allow either/both of these fields, while `aerie` PRs only allow a `gateway` PR to be specified.
+
+For example, see [this PR](https://github.com/NASA-AMMOS/aerie-ui/pull/1420) or [this PR](https://github.com/NASA-AMMOS/aerie-ui/pull/1492).

e2e-tests/docker-compose-many-workers.yml

@@ -17,7 +17,7 @@ services:
       AERIE_DB_PORT: 5432
       GATEWAY_DB_USER: "${GATEWAY_USERNAME}"
       GATEWAY_DB_PASSWORD: "${GATEWAY_PASSWORD}"
-    image: "ghcr.io/nasa-ammos/aerie-gateway:develop"
+    image: 'ghcr.io/nasa-ammos/aerie-gateway:${GATEWAY_IMAGE_TAG:-develop}'
     ports: ["9000:9000"]
     restart: always
     volumes:

e2e-tests/docker-compose-test.yml

@@ -19,7 +19,7 @@ services:
       AERIE_DB_PORT: 5432
       GATEWAY_DB_USER: "${GATEWAY_USERNAME}"
       GATEWAY_DB_PASSWORD: "${GATEWAY_PASSWORD}"
-    image: 'ghcr.io/nasa-ammos/aerie-gateway:develop'
+    image: 'ghcr.io/nasa-ammos/aerie-gateway:${GATEWAY_IMAGE_TAG:-develop}'
     ports: ['9000:9000']
     restart: always
     volumes: