Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add docker layer caching to smoke tests #255

Closed
wants to merge 70 commits into from
Closed
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
a92b848
add docker layer caching to smoke tests
m-goggins Mar 18, 2025
fbfadc6
test trigger to see if python slim is still pulled
m-goggins Mar 19, 2025
43545d9
add build and cache workflow
m-goggins Mar 19, 2025
bd07f7d
test running smoke tests with cached image
m-goggins Mar 19, 2025
db73172
comment out other workflows for testing
m-goggins Mar 19, 2025
607da7f
no sneakiness allowed
m-goggins Mar 19, 2025
9f15012
return back to old "on" for smoke tests
m-goggins Mar 19, 2025
df274ca
get cache to run first
m-goggins Mar 19, 2025
931c3cd
only run on workflow run?
m-goggins Mar 19, 2025
1d7f23d
try just needs
m-goggins Mar 19, 2025
34e15a6
back to the original
m-goggins Mar 19, 2025
0dbc0e9
try mysql instead of postgres
m-goggins Mar 19, 2025
e3d59e2
try resetting back to original
m-goggins Mar 19, 2025
cd342c8
run with cached mysql & add logs to rl docker image
m-goggins Mar 19, 2025
57bb0bf
try replacing localhost
m-goggins Mar 19, 2025
1ebc886
change to bride network
m-goggins Mar 19, 2025
7be1dc0
try bridge + mysql instead of localhost
m-goggins Mar 19, 2025
b972c29
add back pull in the workflow
m-goggins Mar 20, 2025
c87ddf7
back to host and mysql in url
m-goggins Mar 20, 2025
bc5e9bc
only run postgres
m-goggins Mar 20, 2025
c61fcb1
revert to orginal
m-goggins Mar 20, 2025
e05c14e
add logging and wait for postgres to start before building RL
m-goggins Mar 20, 2025
8e1359c
list running containers
m-goggins Mar 20, 2025
618d7cc
just show running containers, no tests
m-goggins Mar 20, 2025
55584db
uncomment
m-goggins Mar 20, 2025
770201f
just run postgres normally
m-goggins Mar 20, 2025
973cba4
add workflow to test cache
m-goggins Mar 20, 2025
33ca31e
try using cached postgres for smoke test
m-goggins Mar 20, 2025
df9c3de
add inspection of postgres service
m-goggins Mar 20, 2025
48c98c1
remove cached test
m-goggins Mar 20, 2025
37b61ff
update image name
m-goggins Mar 20, 2025
5dc3f58
add additional checks for postgres
m-goggins Mar 20, 2025
7fe0f6b
trty just pushing and using cached rl-service
m-goggins Mar 20, 2025
e364e8c
log in to ghcr first
m-goggins Mar 20, 2025
e7c47ad
change to only build if no cached version is found
m-goggins Mar 20, 2025
6a5643f
check removing buildx
m-goggins Mar 20, 2025
7ef6d20
split out caching as separate job
m-goggins Mar 20, 2025
7aae7fa
revert
m-goggins Mar 20, 2025
ed6d909
move RL caching to its own job
m-goggins Mar 20, 2025
9ca6024
steps
m-goggins Mar 20, 2025
14aebd0
add runs-on for caching
m-goggins Mar 20, 2025
da77953
try adding -rm
m-goggins Mar 20, 2025
f797790
oops --rm
m-goggins Mar 20, 2025
1ed076a
add Package name in both jobs
m-goggins Mar 20, 2025
da5ebd7
try to pull postgres image for service
m-goggins Mar 20, 2025
50e98f2
try separating steps
m-goggins Mar 20, 2025
856e96a
revert to original
m-goggins Mar 20, 2025
bb1132e
fix mistake in reversion
m-goggins Mar 20, 2025
2a1be6e
add wait for postgres db to be ready
m-goggins Mar 20, 2025
b0befd2
add check for postgres to be ready
m-goggins Mar 20, 2025
59a9ece
try different url for postgres check
m-goggins Mar 20, 2025
6cff853
bump sleep time
m-goggins Mar 20, 2025
e126455
check if postgres ever appeared
m-goggins Mar 20, 2025
09d9847
check pf logs
m-goggins Mar 20, 2025
df24b5e
add DB_URI
m-goggins Mar 20, 2025
df5c63c
add user and update uri
m-goggins Mar 20, 2025
f67b167
manually start pg
m-goggins Mar 20, 2025
7b98cd4
test postgres health
m-goggins Mar 21, 2025
b0c314c
combine cache and smoke_tests jobs
m-goggins Mar 21, 2025
ef4f268
oops typo
m-goggins Mar 21, 2025
ff0c9a4
add back postgres env in services
m-goggins Mar 21, 2025
9f3e4de
try pinning postgres to 17.4-bookwarm
m-goggins Mar 21, 2025
f9e6ed0
back to postgres:17
m-goggins Mar 21, 2025
c87281c
check other build and cache strategy
m-goggins Mar 21, 2025
369fb39
add smoke tests
m-goggins Mar 21, 2025
a6316dc
try addition options --pull=missing
m-goggins Mar 21, 2025
c65a998
try pinning postgres to 17.4 on smoke tests
m-goggins Mar 21, 2025
cc94eb3
try pinning with SHA
m-goggins Mar 21, 2025
2ceedaf
pull=missing
m-goggins Mar 21, 2025
366b17d
only run smoke tests workflow
m-goggins Mar 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 34 additions & 12 deletions .github/workflows/check_smoke_tests.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: check smoke tests
name: check smoke tests test

on:
push:
Expand All @@ -16,11 +16,12 @@ jobs:

services:
postgres:
image: postgres:17
image: postgres@sha256:7f29c02ba9eeff4de9a9f414d803faa0e6fe5e8d15ebe217e3e418c82e652b35
ports:
- 5432:5432
env:
POSTGRES_PASSWORD: pw
options: --pull=missing

mysql:
image: mysql:8
Expand All @@ -38,16 +39,37 @@ jobs:
SA_PASSWORD: "YourStrong!Passw0rd"

steps:
- name: Convert repository name to lowercase
run: echo "PACKAGE_NAME=$(echo '${{ github.repository }}' | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV

- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Check if cached RecordLinker image exists in GitHub Container Registry
id: check_rl_image
run: |
docker pull ghcr.io/${{ env.PACKAGE_NAME }}/rl-service:latest || echo "No cached image found"

- name: Build and push RecordLinker Docker image if not cached
if: steps.check_rl_image.outcome == 'failure'
run: |
# Build the Docker image, tag it, & push to ghcr
echo "Building and pushing RecordLinker Docker image..."
docker build -t rl-service-image .
docker tag rl-service-image ghcr.io/${{ env.PACKAGE_NAME }}/rl-service:latest
docker push ghcr.io/${{ env.PACKAGE_NAME }}/rl-service:latest

- name: Check out repository
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build RecordLinker Docker Image

- name: Pull cached RecordLinker image from GitHub Container Registry
run: |
docker build -t rl-service-image .

docker pull ghcr.io/${{ env.PACKAGE_NAME }}/rl-service:latest || echo "No cached image found, skipping pull."
- name: Start RecordLinker Service and Run Smoke Tests
run: |
if [[ "${{ matrix.database }}" == "postgres" ]]; then
Expand All @@ -64,7 +86,7 @@ jobs:
docker run -d --name rl-service \
--network="host" \
-e DB_URI=$DB_URI \
rl-service-image
ghcr.io/${{ env.PACKAGE_NAME }}/rl-service:latest # Use the cached image

# Wait for the RL Service to be healthy
TRIES=5
Expand All @@ -80,7 +102,7 @@ jobs:
done

# Run smoke tests and print the response
JSON_BODY_1='{"record": {"birth_date": "2053-11-07", "sex": "M", "identifiers":[{"value": "123456789", "type": "MR"}], "name":[{"family":"Shepard", "given":["John"]}]}}'
JSON_BODY_1='{"record": {"birth_date": "2013-11-07", "sex": "M", "identifiers":[{"value": "123456789", "type": "MR"}], "name":[{"family":"Shepard", "given":["John"]}]}}'
JSON_BODY_2='{"algorithm": "dibbs-default", "record": {"birth_date": "2000-12-06", "sex": "M", "identifiers":[{"value": "9876543210", "type": "MR"}], "name":[{"family":"Smith", "given":["William"]}]}}'

#basic tests
Expand Down Expand Up @@ -125,4 +147,4 @@ jobs:
-H "Content-Type: application/json")

echo "Response: $RESPONSE_5"
echo "$RESPONSE_5" | grep -q "No algorithm found"
echo "$RESPONSE_5" | grep -q "No algorithm found"
Loading