Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref(ci): Refactor self-hosted e2e and dockerhub steps to not use GCR image #6800

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

untitaker
Copy link
Member

we have three builds:

  • snuba-ci GHCR (ci.yml) -- this includes test dependencies
  • snuba GHCR (image.yml)
  • snuba GCR (cloudbuild.yml)

the latter two images are identical, but it's too much hassle to poll GCR and re-publish, or pull one image from another, so we just build it twice for now

then:

  • snuba GCR is re-published to dockerhub (happens in ci.yml by polling GCR)
  • snuba GCR is used to run self-hosted e2e tests (this has now been changed)

now, to the changes in this PR:

  • self-hosted e2e would like to run on the same image as dockerhub.
    since ci: Fix self-hosted test runs #6796 those two images
    have diverged since one is using snuba-ci (GHCR) and the other GCR. this
    re-unifies them on top of snuba (GHCR)

  • publish-to-dockerhub was polling GCR. instead, it could just wait for
    the job result of GHCR

…image

we have three builds:

* snuba-ci GHCR (ci.yml) -- this includes test dependencies
* snuba GHCR (image.yml)
* snuba GCR (cloudbuild.yml)

the latter two images are identical, but it's too much hassle to poll GCR and re-publish, or pull one image from another, so we just build it twice for now

then:

* snuba GCR is re-published to dockerhub (happens in ci.yml by polling GCR)
* snuba GCR is used to run self-hosted e2e tests (this has now been changed)

now, to the changes in this PR:

* self-hosted e2e would like to run on the same image as dockerhub.
  since #6796 those two images
  have diverged since one is using snuba-ci (GHCR) and the other GCR. this
  re-unifies them on top of snuba (GHCR)

* publish-to-dockerhub was polling GCR. instead, it could just wait for
  the job result of GHCR
@untitaker untitaker requested a review from a team as a code owner January 21, 2025 18:04
@aldy505
Copy link
Contributor

aldy505 commented Jan 22, 2025

Dupe? #6770

@onkar
Copy link
Member

onkar commented Jan 22, 2025

Dupe? #6770

Closed #6770

@aldy505
Copy link
Contributor

aldy505 commented Jan 23, 2025

@untitaker just a quick question, do you also want to publish to Google Artifact Registry like they've done it on Symbolicator? getsentry/symbolicator#1595

@untitaker
Copy link
Member Author

untitaker commented Jan 23, 2025

@aldy505 oh, that would be nice, to not build the same image twice. we didn't think it was worth our time to figure out how to do that and were fine with building the same image twice. but I think cloudbuild.yml can become a noop and we can publish from image.yml. if you want to port that from relay go ahead.

curious @Dav1dde did you have to do anything in terms of secret mgmt to make this work?

@Dav1dde
Copy link
Member

Dav1dde commented Jan 23, 2025

curious @Dav1dde did you have to do anything in terms of secret mgmt to make this work?

Yes. (Internal PR)

If you add snuba to that list, you can authenticate with the artifact registry and push the image from GHA, similar to what I updated in Symbolicator and Relay has been doing for a long time, making the cloudbuild obsolete.

Both repos also contain two different ways to build a multiarch arm64/amd64 container, maybe one of these methods you can use and easily adapt for snuba as well.

Relay builds two binaries and builds the multiarch image in a single step using the (cross-)compiled binaries. Symbolicator merges two independent images into a single multiarch image (easier to use if you have a complicated Dockerfile).

There is also a third method described here, cross compiling in the build.

@untitaker
Copy link
Member Author

we actually used to have arm64 images but we used QEMU to build two separate images, then merge them. cross-compilation didn't work out for some reason. we dropped the image because there was no demand and the builds took way too long as we started introducing rust code. nowadays github has actual linux ARM runners so we probably stand a better chance.

@Dav1dde
Copy link
Member

Dav1dde commented Jan 23, 2025

Yeah, that's what symbolicator does now, I just took the existing Dockerfile and added a native ARM64 build, then merges the images together. Ideally I'd like to do it differently, but that was just the easiest way to make it work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants