Skip to content

Commit

Permalink
add aarch64 image
Browse files Browse the repository at this point in the history
- add CI to build, tag and push x86_64 and aarch64 images
- add ARCH arg to Dockerfile
- build ccache from source as no pre-built aarch64 binary available
  • Loading branch information
lkeegan committed Feb 4, 2025
1 parent 901ae96 commit 62eb5f1
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 32 deletions.
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: CI
on: push

jobs:
manylinux:
runs-on: ${{ matrix.platform.os }}
strategy:
fail-fast: false
matrix:
platform:
- os: "ubuntu-22.04"
arch: "x86_64"
- os: "ubuntu-22.04-arm"
arch: "aarch64"
steps:
- uses: actions/checkout@v4
- run: docker compose build --build-arg="ARCH=${{ matrix.platform.arch }}"
- if: github.repository == 'spatial-model-editor/sme_manylinux' && github.event_name == 'push' && (startsWith(github.event.ref, 'refs/tags/'))
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- if: github.repository == 'spatial-model-editor/sme_manylinux' && github.event_name == 'push' && (startsWith(github.event.ref, 'refs/tags/'))
run: |
docker compose build . --build-arg="ARCH=${{ARCH}}" -t ghcr.io/spatial-model-editor/manylinux_${{ matrix.platform.arch }}:${{ github.ref_name }}
docker compose push ghcr.io/spatial-model-editor/manylinux_${{ matrix.platform.arch }}:${{ github.ref_name }}
34 changes: 20 additions & 14 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
FROM quay.io/pypa/manylinux_2_28_x86_64:2025-01-05-75aeda9 as builder
ARG ARCH
FROM quay.io/pypa/manylinux_2_28_${ARCH}:2025.02.02-1 as builder

ARG ARCH
ARG NPROCS=24
ARG BUILD_DIR=/opt/smelibs
ARG TMP_DIR=/opt/tmpwd
Expand Down Expand Up @@ -56,7 +58,7 @@ RUN mkdir -p $TMP_DIR && cd $TMP_DIR \
&& ./configure \
--prefix=$BUILD_DIR \
--disable-shared \
--host=x86_64-unknown-linux-gnu \
--host=${ARCH}-unknown-linux-gnu \
--enable-static \
--with-pic \
--enable-cxx \
Expand All @@ -75,7 +77,7 @@ RUN mkdir -p $TMP_DIR && cd $TMP_DIR \
&& ./configure \
--prefix=$BUILD_DIR \
--disable-shared \
--host=x86_64-unknown-linux-gnu \
--host=${ARCH}-unknown-linux-gnu \
--enable-static \
--with-pic \
--with-gmp-lib=$BUILD_DIR/lib \
Expand Down Expand Up @@ -182,7 +184,7 @@ RUN mkdir -p $TMP_DIR && cd $TMP_DIR \
&& ninja install \
&& rm -rf $TMP_DIR

ARG LLVM_VERSION="19.1.6"
ARG LLVM_VERSION="19.1.7"
RUN mkdir -p $TMP_DIR && cd $TMP_DIR \
&& git clone \
-b llvmorg-$LLVM_VERSION \
Expand All @@ -196,7 +198,7 @@ RUN mkdir -p $TMP_DIR && cd $TMP_DIR \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=$BUILD_DIR \
-DPython3_EXECUTABLE:FILEPATH=/opt/python/cp312-cp312/bin/python \
-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu \
-DLLVM_DEFAULT_TARGET_TRIPLE=${ARCH}-unknown-linux-gnu \
-DLLVM_TARGETS_TO_BUILD="X86" \
-DLLVM_BUILD_TOOLS=OFF \
-DLLVM_INCLUDE_TOOLS=OFF \
Expand Down Expand Up @@ -724,27 +726,31 @@ RUN mkdir -p $TMP_DIR && cd $TMP_DIR \
&& ninja install \
&& rm -rf $TMP_DIR

FROM quay.io/pypa/manylinux_2_28_x86_64:2025-01-05-75aeda9
FROM quay.io/pypa/manylinux_2_28_${ARCH}:2025.02.02-1

LABEL org.opencontainers.image.source=https://github.com/spatial-model-editor/sme_manylinux_x86_64
LABEL org.opencontainers.image.description="manylinux x86_64 image for compiling Spatial Model Editor python wheels"
LABEL org.opencontainers.image.source=https://github.com/spatial-model-editor/sme_manylinux
LABEL org.opencontainers.image.description="manylinux ${ARCH} image for compiling Spatial Model Editor python wheels"
LABEL org.opencontainers.image.licenses=MIT

ARG BUILD_DIR=/opt/smelibs
ARG TMP_DIR=/opt/tmpwd

RUN /opt/python/cp312-cp312/bin/pip install ninja \
&& ln -fs /opt/python/cp312-cp312/bin/ninja /usr/bin/ninja

ARG CCACHE_VERSION="4.10.2"
RUN mkdir -p $TMP_DIR && cd $TMP_DIR \
&& curl \
-L https://github.com/ccache/ccache/releases/download/v${CCACHE_VERSION}/ccache-${CCACHE_VERSION}-linux-x86_64.tar.xz \
-L https://github.com/ccache/ccache/releases/download/v${CCACHE_VERSION}/ccache-${CCACHE_VERSION}.tar.xz \
--output ccache.tar.xz \
&& tar xJf ccache.tar.xz \
&& cd ccache-${CCACHE_VERSION}-linux-x86_64 \
&& make install \
&& cd ccache-${CCACHE_VERSION} \
&& mkdir build \
&& cd build \
&& cmake -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCUMENTATION=OFF -DHTTP_STORAGE_BACKEND=OFF -DENABLE_TESTING=OFF -DREDIS_STORAGE_BACKEND=OFF -GNinja .. \
&& ninja \
&& ninja install \
&& rm -rf $TMP_DIR

RUN /opt/python/cp312-cp312/bin/pip install ninja \
&& ln -fs /opt/python/cp312-cp312/bin/ninja /usr/bin/ninja

# SME static libs
COPY --from=builder $BUILD_DIR $BUILD_DIR
25 changes: 7 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,15 @@
# sme_manylinux_x86_64
# sme_manylinux

Docker container for compiling linux x86_64 python wheels for [sme](https://pypi.org/project/sme/)
Docker containers for compiling linux python wheels for [sme](https://pypi.org/project/sme/)

- Available from <https://ghcr.io/spatial-model-editor/manylinux_x86_64>
- Available from <https://ghcr.io/spatial-model-editor/sme_manylinux>

- Used by <https://github.com/spatial-model-editor/spatial-model-editor>

- Based on <https://quay.io/repository/pypa/manylinux_2_28_x86_64>
- Based on
- <https://quay.io/repository/pypa/manylinux_2_28_x86_64>
- <https://quay.io/repository/pypa/manylinux_2_28_aarch64>

## To update

Update the Dockerfile, tag the commit with `tagname`, git push, then build and push the docker container:

```bash
docker build . -t ghcr.io/spatial-model-editor/manylinux_x86_64:tagname
docker push ghcr.io/spatial-model-editor/manylinux_x86_64:tagname
```

where `tagname` is today's date in the form `YYYY.MM.DD`

## Note

Would be cleaner to have a github action that builds the container on each tagged commit, as we do for sme_deps etc.

Currently not doing this for convenience, as the docker build would take a long time to run on CI.
To release new images, tag the commit with `YYYY.MM.DD`

0 comments on commit 62eb5f1

Please sign in to comment.