-
-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ref(ci): Refactor self-hosted e2e and dockerhub steps to not use GCR image #6800
base: master
Are you sure you want to change the base?
Conversation
…image we have three builds: * snuba-ci GHCR (ci.yml) -- this includes test dependencies * snuba GHCR (image.yml) * snuba GCR (cloudbuild.yml) the latter two images are identical, but it's too much hassle to poll GCR and re-publish, or pull one image from another, so we just build it twice for now then: * snuba GCR is re-published to dockerhub (happens in ci.yml by polling GCR) * snuba GCR is used to run self-hosted e2e tests (this has now been changed) now, to the changes in this PR: * self-hosted e2e would like to run on the same image as dockerhub. since #6796 those two images have diverged since one is using snuba-ci (GHCR) and the other GCR. this re-unifies them on top of snuba (GHCR) * publish-to-dockerhub was polling GCR. instead, it could just wait for the job result of GHCR
Dupe? #6770 |
@untitaker just a quick question, do you also want to publish to Google Artifact Registry like they've done it on Symbolicator? getsentry/symbolicator#1595 |
@aldy505 oh, that would be nice, to not build the same image twice. we didn't think it was worth our time to figure out how to do that and were fine with building the same image twice. but I think cloudbuild.yml can become a noop and we can publish from image.yml. if you want to port that from relay go ahead. curious @Dav1dde did you have to do anything in terms of secret mgmt to make this work? |
Yes. (Internal PR) If you add snuba to that list, you can authenticate with the artifact registry and push the image from GHA, similar to what I updated in Symbolicator and Relay has been doing for a long time, making the cloudbuild obsolete. Both repos also contain two different ways to build a multiarch arm64/amd64 container, maybe one of these methods you can use and easily adapt for snuba as well. Relay builds two binaries and builds the multiarch image in a single step using the (cross-)compiled binaries. Symbolicator merges two independent images into a single multiarch image (easier to use if you have a complicated Dockerfile). There is also a third method described here, cross compiling in the build. |
we actually used to have arm64 images but we used QEMU to build two separate images, then merge them. cross-compilation didn't work out for some reason. we dropped the image because there was no demand and the builds took way too long as we started introducing rust code. nowadays github has actual linux ARM runners so we probably stand a better chance. |
Yeah, that's what symbolicator does now, I just took the existing Dockerfile and added a native ARM64 build, then merges the images together. Ideally I'd like to do it differently, but that was just the easiest way to make it work. |
we have three builds:
the latter two images are identical, but it's too much hassle to poll GCR and re-publish, or pull one image from another, so we just build it twice for now
then:
now, to the changes in this PR:
self-hosted e2e would like to run on the same image as dockerhub.
since ci: Fix self-hosted test runs #6796 those two images
have diverged since one is using snuba-ci (GHCR) and the other GCR. this
re-unifies them on top of snuba (GHCR)
publish-to-dockerhub was polling GCR. instead, it could just wait for
the job result of GHCR