Reorganize CI jobs so they appear sensibly in the GitHub UI #3422
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| name: "Docs build test" | ||
| # ensure the docs build still works | ||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: [main, release] | ||
|
|
||
| jobs: | ||
| check-docs: | ||
| name: runner / check docs | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v6 | ||
| - name: Ensure docs build cleanly | ||
| # Setting `O` to pass extra options to the sphinx-build command. | ||
| run: O="-a -E -n -W --keep-going" make docs | ||
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,52 @@ | ||||||
| name: Test MacOS | ||||||
| # Unknown. MacOS functionality test i guess, probably incomplete | ||||||
|
There was a problem hiding this comment. The comment is vague and unprofessional. Consider providing a clear description of what this workflow does, such as: "Tests installation and basic setup on macOS to ensure compatibility" or something more specific based on the actual purpose of this test.
Suggested change
Collaborator
Author
There was a problem hiding this comment. @sgoggins do you know why this test is here, or if we still rely on it?
Collaborator
There was a problem hiding this comment. @JohnStrunk created most of these adn they have been working. AFAIK these tests are part of what is keeping augur stable and easy to deploy ... which I think you know so I think I may not understand the question :)
Collaborator
Author
There was a problem hiding this comment. given the definition of this test, im unsure what it is specifically testing - it seems to just install some python data processing toolkits and things. are we trying to ensure one of our dependencies can install on mac OS? it feels weird to be testing a dependency in one of our tests like this - IMO it would make more sense to make sure the unit tests run on MacOS (which I think its set up by this PR)
Collaborator
There was a problem hiding this comment. The "Go" and NLTK parts have been the most inconsistent historically. @JohnStrunk's approach, as I understand it, was to create this test in the CI and then validate that, when the test succeeds, Augur runs on OSX. Successful installation of the toolkits cannot be assumed across OSX distros, Go, and Python distros on OSX, based on our long experience. |
||||||
| on: | ||||||
| push: | ||||||
| branches: | ||||||
| - main | ||||||
| - release | ||||||
| pull_request: | ||||||
| release: | ||||||
| types: | ||||||
| - published | ||||||
| workflow_dispatch: | ||||||
|
|
||||||
| permissions: {} | ||||||
|
|
||||||
| jobs: | ||||||
| test-macos: | ||||||
| name: Test on macOS | ||||||
| runs-on: macos-latest | ||||||
| env: | ||||||
| UV_LOCKED: true # Assert that uv.lock is up-to-date | ||||||
| steps: | ||||||
| - name: Checkout repository | ||||||
| uses: actions/checkout@v4 | ||||||
|
|
||||||
| - name: Install uv | ||||||
| uses: astral-sh/setup-uv@v6 | ||||||
|
|
||||||
| - name: Set up Python | ||||||
| uses: actions/setup-python@v5 | ||||||
| with: | ||||||
| python-version-file: ".python-version" | ||||||
|
|
||||||
| - name: Install Go | ||||||
| uses: actions/setup-go@v5 | ||||||
| with: | ||||||
| go-version: "stable" | ||||||
|
|
||||||
| # We don't use `make install` because it requires user input | ||||||
| # Instead, we manually sync and run a subset of commands | ||||||
| - name: Install dependencies | ||||||
| run: uv sync --all-groups | ||||||
|
|
||||||
| - name: Install workers | ||||||
| run: uv run scripts/install/workers.sh dev | ||||||
|
|
||||||
| - name: Install nltk | ||||||
| run: | | ||||||
| uv run python -m nltk.downloader stopwords | ||||||
| uv run python -m nltk.downloader punkt | ||||||
| uv run python -m nltk.downloader popular | ||||||
| uv run python -m nltk.downloader universal_tagset | ||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| name: Publish Docker Images | ||
| # Publish docker images to our registry | ||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| - release | ||
| pull_request: | ||
| release: | ||
| types: | ||
| - published | ||
| workflow_dispatch: | ||
|
|
||
| permissions: {} | ||
|
|
||
| jobs: | ||
| push-image: | ||
| name: Push image | ||
| needs: test-e2e | ||
sgoggins marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # We don't push images on pull requests | ||
| if: github.event_name != 'pull_request' | ||
| permissions: | ||
| contents: read # to fetch code (actions/checkout) | ||
| packages: write # to push docker image | ||
| strategy: | ||
| matrix: | ||
| image: | ||
| - backend | ||
| - database | ||
| - keyman | ||
| - rabbitmq | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
| id: setup-buildx | ||
|
|
||
| - name: Login to GitHub Container Registry | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ github.repository_owner }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Set container metadata | ||
| uses: docker/metadata-action@v5 | ||
| id: meta | ||
| env: | ||
| DOCKER_METADATA_ANNOTATIONS_LEVELS: index,manifest | ||
| with: | ||
| annotations: | | ||
| org.opencontainers.image.title=augur_${{ matrix.image }} | ||
| labels: | | ||
| org.opencontainers.image.title=augur_${{ matrix.image }} | ||
| images: ghcr.io/${{ github.repository_owner }}/augur_${{ matrix.image }} | ||
| # Pushes to the main branch update the *:devel-latest tag | ||
| # Releases update the *:latest tag and the *:<version> tag | ||
| tags: | | ||
| type=raw,value=devel-latest,enable=${{ github.ref == 'refs/heads/main' }} | ||
| type=raw,value=latest,enable=${{ github.event_name == 'release' }} | ||
| type=raw,value=${{ github.event.release.tag_name }},enable=${{ github.event_name == 'release' }} | ||
|
|
||
| - name: Build and push | ||
| id: push | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| annotations: ${{ steps.meta.outputs.annotations }} | ||
| context: . | ||
| file: ./docker/${{ matrix.image }}/Dockerfile | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
| platforms: linux/amd64 | ||
| # Only push if we've tagged the image in the metadata step | ||
| push: ${{ steps.meta.outputs.tags != '' }} | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| # Use the same cache as the build step | ||
| cache-from: type=gha,scope=container-${{ matrix.image }} | ||
| cache-to: type=gha,scope=container-${{ matrix.image }},mode=min | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.