Skip to content

Commit 1c86afa

Browse files
leofangleofang
authored
Full CI support for public builds + switch to use cibuildwheel (#267)
* switch to cibuildwheel + some cleanups * try setting up Python manually * comment out un-needed code + propagate python-version * fix: need to check out first * only build natively; add -v; ensure targeting manylinux * single quotes * restore env setup to get artifact dir * fix artifact dir * fix artifact name * restore & fix artifact name again * build on all platforms! * fix CIBW_BUILD for windows * fix typo * no quotes for wildcard matching * move CIBW_BUILD logic to script * fix win runner name * try to find where pwsh is * try to escape * continue hunting.. * try to overwrite shell * try to install ps * be explicit about shell (why?) * only build for win 64 bits * try to install msvc * install msvc ourselves * fix typo * skip custom cl ver check * install to standard location * try to locate Python include path * switch to public windows runner for now * windows image does not have sudo * pwd on Windows Bash does not use Windows path format * cover all Python versions! * add quotes * align the python version installed via GHA vs used at build time * fix constraint syntax * check if setup-python is causing interference * fix typo * apply a WAR on Linux * fix unbound var * detect Python path after it's installed (by CIBW) * try CIBW_BEFORE_ALL_LINUX * try to restore the pre-py-span setup... * reduce build matrix to experiment with cuda.bindings builds * fix parenthesis * use abs path * defer CUDA_PATH setting * use CIBW_ENVIRONMENT to pass env var * fetch cuda-profiler-api headers * only rely on redist * allow wheel repair to fix the triplet tags * restore full build matrix! * fix wget on Windows; pass PARALLEL_LEVEL to CIBW * switch from wget to curl * windows needs unzip not tar * mv -> rsync * git bash has no wget or rsync... * ensure win-style path on win * convert abs path * debug * another level down * check if it is a race condition on win * clean up unused (but still relevant) code * clean up unused (but still relevant) code - cont'd * consolidate with PYTHON_VERSION_FORMATTED
1 parent 60f9362 commit 1c86afa

File tree

5 files changed

+178
-89
lines changed

5 files changed

+178
-89
lines changed

‎.github/actions/build/action.yml

+57-51
Original file line numberDiff line numberDiff line change
@@ -13,73 +13,79 @@ inputs:
1313
host-platform:
1414
required: true
1515
type: string
16-
use-container:
17-
required: true
18-
type: boolean
19-
docker-image:
20-
type: string
21-
required: true
2216
upload-enabled:
2317
required: true
2418
type: boolean
25-
python-version:
26-
required: true
27-
type: string
2819

2920
runs:
3021
using: composite
3122
steps:
23+
- name: Build cuda.core wheel
24+
uses: pypa/[email protected]
25+
env:
26+
CIBW_BUILD: ${{ env.CIBW_BUILD }}
27+
CIBW_ARCHS_LINUX: "native"
28+
CIBW_BUILD_VERBOSITY: 1
29+
# # ensure Python.h & co can be found
30+
# CIBW_BEFORE_BUILD_WINDOWS: >
31+
# python -c "import sysconfig; print(sysconfig.get_path('include'))" >> $env:INCLUDE
32+
with:
33+
package-dir: ./cuda_core/
34+
output-dir: ${{ env.CUDA_CORE_ARTIFACTS_DIR }}
3235

33-
- if: ${{ inputs.use-container }}
34-
name: Build (in container)
35-
shell: bash --noprofile --norc -xeuo pipefail {0}
36-
run: |
37-
38-
docker run \
39-
-e AWS_REGION \
40-
-e AWS_SESSION_TOKEN \
41-
-e AWS_ACCESS_KEY_ID \
42-
-e AWS_SECRET_ACCESS_KEY \
43-
-e GITHUB_TOKEN \
44-
-e BINDINGS_ARTIFACTS_DIR="$BINDINGS_ARTIFACTS_DIR" \
45-
-e CORE_ARTIFACTS_DIR="$CORE_ARTIFACTS_DIR" \
46-
-e UPLOAD_ENABLED="$UPLOAD_ENABLED" \
47-
-e USE_CUDA="$USE_CUDA" \
48-
-e REPO_DIR="$REPO_DIR" \
49-
-e LEGATE_CORE_BUILD_MODE="$LEGATE_CORE_BUILD_MODE" \
50-
-e PYTHON_VERSION="$PYTHON_VERSION" \
51-
-v "${{ env.REPO_DIR }}:${{ env.REPO_DIR }}" \
52-
-v "${{ env.BINDINGS_ARTIFACTS_DIR }}:${{ env.BINDINGS_ARTIFACTS_DIR }}" \
53-
-v "${{ env.CORE_ARTIFACTS_DIR }}:${{ env.CORE_ARTIFACTS_DIR }}" \
54-
--rm "${{ inputs.docker-image }}" \
55-
/bin/bash -c "${{ env.REPO_DIR }}/continuous_integration/scripts/entrypoint ${{ env.REPO_DIR }}/continuous_integration/scripts/build ${{ inputs.build-type}} ${{ inputs.target-device }}"
56-
57-
- if: ${{ !inputs.use-container }}
58-
name: Build (without container)
59-
shell: bash --noprofile --norc -xeuo pipefail {0}
60-
run: |
61-
"${{ env.REPO_DIR }}/continuous_integration/scripts/entrypoint" "${{ env.REPO_DIR }}/continuous_integration/scripts/build" "${{ inputs.build-type}}" "${{ inputs.target-device }}"
62-
63-
- name: Display structure of the bindings artifacts folder (post build)
36+
- name: List the cuda.core artifacts directory
6437
shell: bash --noprofile --norc -xeuo pipefail {0}
6538
run: |
66-
sudo chown -R $(whoami) ${{ env.BINDINGS_ARTIFACTS_DIR }}
67-
ls -lahR ${{ env.BINDINGS_ARTIFACTS_DIR }}
39+
if [[ "${{ inputs.host-platform }}" == win* ]]; then
40+
export CHOWN=chown
41+
else
42+
export CHOWN="sudo chown"
43+
fi
44+
$CHOWN -R $(whoami) ${{ env.CUDA_CORE_ARTIFACTS_DIR }}
45+
ls -lahR ${{ env.CUDA_CORE_ARTIFACTS_DIR }}
6846
69-
- name: Upload bindings build artifacts
47+
- name: Upload cuda.core build artifacts
7048
uses: actions/upload-artifact@v4
7149
with:
72-
name: ${{ env.BINDINGS_ARTIFACT_NAME }}
73-
path: ${{ env.BINDINGS_ARTIFACTS_DIR }}
50+
name: ${{ env.CUDA_CORE_ARTIFACT_NAME }}
51+
path: ${{ env.CUDA_CORE_ARTIFACTS_DIR }}/*.whl
52+
if-no-files-found: error
53+
overwrite: 'true'
54+
55+
- name: Build cuda.bindings wheel
56+
uses: pypa/[email protected]
57+
env:
58+
CIBW_BUILD: ${{ env.CIBW_BUILD }}
59+
CIBW_ARCHS_LINUX: "native"
60+
CIBW_BUILD_VERBOSITY: 1
61+
CIBW_ENVIRONMENT_LINUX: >
62+
CUDA_PATH="$(realpath ./cuda_toolkit)"
63+
PARALLEL_LEVEL=${{ env.PARALLEL_LEVEL }}
64+
CIBW_ENVIRONMENT_WINDOWS: >
65+
CUDA_HOME="$(cygpath -w $(realpath ./cuda_toolkit))"
66+
# PARALLEL_LEVEL=${{ env.PARALLEL_LEVEL }}
67+
# # ensure Python.h & co can be found
68+
# CIBW_BEFORE_BUILD_WINDOWS: >
69+
# python -c "import sysconfig; print(sysconfig.get_path('include'))" >> $env:INCLUDE
70+
with:
71+
package-dir: ./cuda_bindings/
72+
output-dir: ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}
7473

75-
- name: Display structure of the core artifacts folder (post build)
74+
- name: List the cuda.bindings artifacts directory
7675
shell: bash --noprofile --norc -xeuo pipefail {0}
7776
run: |
78-
sudo chown -R $(whoami) ${{ env.CORE_ARTIFACTS_DIR }}
79-
ls -lahR ${{ env.CORE_ARTIFACTS_DIR }}
77+
if [[ "${{ inputs.host-platform }}" == win* ]]; then
78+
export CHOWN=chown
79+
else
80+
export CHOWN="sudo chown"
81+
fi
82+
$CHOWN -R $(whoami) ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}
83+
ls -lahR ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}
8084
81-
- name: Upload core build artifacts
85+
- name: Upload cuda.bindings build artifacts
8286
uses: actions/upload-artifact@v4
8387
with:
84-
name: ${{ env.CORE_ARTIFACT_NAME }}
85-
path: ${{ env.CORE_ARTIFACTS_DIR }}
88+
name: ${{ env.CUDA_BINDINGS_ARTIFACT_NAME }}
89+
path: ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}/*.whl
90+
if-no-files-found: error
91+
overwrite: 'true'

‎.github/actions/setup/action.yml

+91-21
Original file line numberDiff line numberDiff line change
@@ -22,51 +22,121 @@ inputs:
2222
python-version:
2323
required: true
2424
type: string
25+
cuda-version:
26+
required: true
27+
type: string
2528

2629
runs:
2730
using: composite
2831
steps:
29-
- name: Set REPO_DIR and Dump environment
32+
# WAR: setup-python is not relocatable...
33+
# see https://github.com/actions/setup-python/issues/871
34+
- name: Set up Python ${{ inputs.python-version }}
35+
if: ${{ startsWith(inputs.host-platform, 'linux') }}
36+
id: setup-python
37+
uses: actions/setup-python@v5
38+
with:
39+
python-version: "3.12"
40+
41+
- name: Set up MSVC
42+
if: ${{ startsWith(inputs.host-platform, 'win') }}
43+
uses: ilammy/msvc-dev-cmd@v1
44+
45+
- name: Dump environment
3046
shell: bash --noprofile --norc -xeuo pipefail {0}
3147
run: |
32-
echo "REPO_DIR=$(pwd)" >> $GITHUB_ENV
3348
env
3449
35-
- name: Set environment variables
50+
- name: Get CUDA components
3651
shell: bash --noprofile --norc -xeuo pipefail {0}
3752
run: |
53+
CUDA_PATH="./cuda_toolkit"
54+
mkdir $CUDA_PATH
3855
39-
WITH_TESTS_STR=''
40-
if [[ ("${{ inputs.upload-enabled }}" == "false") && ("${{ inputs.build-type }}" != "ci") ]]; then
41-
WITH_TESTS_STR='-with_tests'
56+
# The binary archives (redist) are guaranteed to be updated as part of the release posting.
57+
CTK_BASE_URL="https://developer.download.nvidia.com/compute/cuda/redist/"
58+
CTK_JSON_URL="$CTK_BASE_URL/redistrib_${{ inputs.cuda-version }}.json"
59+
if [[ "${{ inputs.host-platform }}" == linux* ]]; then
60+
if [[ "${{ inputs.host-platform }}" == "linux-x64" ]]; then
61+
CTK_SUBDIR="linux-x86_64"
62+
elif [[ "${{ inputs.host-platform }}" == "linux-aarch64" ]]; then
63+
CTK_SUBDIR="linux-sbsa"
64+
fi
65+
function extract() {
66+
tar -xvf $1 -C $CUDA_PATH --strip-components=1
67+
}
68+
elif [[ "${{ inputs.host-platform }}" == "win-x64" ]]; then
69+
CTK_SUBDIR="windows-x86_64"
70+
function extract() {
71+
_TEMP_DIR_=$(mktemp -d)
72+
unzip $1 -d $_TEMP_DIR_
73+
cp -r $_TEMP_DIR_/*/* $CUDA_PATH
74+
rm -rf $_TEMP_DIR_
75+
}
4276
fi
77+
function populate_cuda_path() {
78+
# take the component name as a argument
79+
function download() {
80+
curl -kLSs $1 -o $2
81+
}
82+
CTK_COMPONENT=$1
83+
CTK_COMPONENT_REL_PATH="$(curl -s $CTK_JSON_URL |
84+
python -c "import sys, json; print(json.load(sys.stdin)['${CTK_COMPONENT}']['${CTK_SUBDIR}']['relative_path'])")"
85+
CTK_COMPONENT_URL="${CTK_BASE_URL}/${CTK_COMPONENT_REL_PATH}"
86+
CTK_COMPONENT_COMPONENT_FILENAME="$(basename $CTK_COMPONENT_REL_PATH)"
87+
download $CTK_COMPONENT_URL $CTK_COMPONENT_COMPONENT_FILENAME
88+
extract $CTK_COMPONENT_COMPONENT_FILENAME
89+
rm $CTK_COMPONENT_COMPONENT_FILENAME
90+
}
4391
44-
TARGET_PLATFORM='linux-64'
45-
if [[ "${{ inputs.host-platform }}" == "linux-aarch64" ]]; then
92+
# Get headers and shared libraries in place
93+
populate_cuda_path cuda_nvcc
94+
populate_cuda_path cuda_cudart
95+
populate_cuda_path cuda_nvrtc
96+
populate_cuda_path cuda_profiler_api
97+
ls -l $CUDA_PATH
98+
99+
# Note: the headers will be copied into the cibuildwheel manylinux container,
100+
# so setting the CUDA_PATH env var here is meaningless.
101+
102+
- name: Set environment variables
103+
shell: bash --noprofile --norc -xeuo pipefail {0}
104+
run: |
105+
# TODO: just align host-platform names with TARGET_PLATFORM...
106+
if [[ "${{ inputs.host-platform }}" == "linux-x64" ]]; then
107+
TARGET_PLATFORM='linux-64'
108+
elif [[ "${{ inputs.host-platform }}" == "linux-aarch64" ]]; then
46109
TARGET_PLATFORM='linux-aarch64'
110+
elif [[ "${{ inputs.host-platform }}" == "win-x64" ]]; then
111+
TARGET_PLATFORM='win-64'
47112
fi
48113
49-
BUILD_MODE="${{ inputs.build-mode }}"
50-
BUILD_MODE_STR=""
51-
[ -n "${BUILD_MODE}" ] && BUILD_MODE_STR="-${BUILD_MODE}"
114+
PYTHON_VERSION_FORMATTED=$(echo '${{ inputs.python-version }}' | tr -d '.')
115+
if [[ "${{ inputs.host-platform }}" == linux* ]]; then
116+
CIBW_BUILD="cp${PYTHON_VERSION_FORMATTED}-manylinux*"
117+
REPO_DIR=$(pwd)
118+
elif [[ "${{ inputs.host-platform }}" == win* ]]; then
119+
CIBW_BUILD="cp${PYTHON_VERSION_FORMATTED}-win_amd64"
120+
PWD=$(pwd)
121+
REPO_DIR=$(cygpath -w $PWD)
122+
fi
52123
124+
BUILD_MODE="${{ inputs.build-mode }}"
53125
if [[ ("${BUILD_MODE}" == "") || ("${BUILD_MODE}" == "release") ]]; then
54126
# We upload release versions in the default folder.
55127
PKG_DIR="${TARGET_PLATFORM}"
56128
else
57129
PKG_DIR="${BUILD_MODE}/${TARGET_PLATFORM}"
58130
fi
59131
60-
PYTHON_VERSION_FORMATTED=$(echo '${{ inputs.python-version }}' | tr -d '.')
61-
62-
echo "BINDINGS_ARTIFACT_NAME=${{ inputs.host-platform }}-${{ inputs.build-type }}-cuda_bindings-python${PYTHON_VERSION_FORMATTED}-${{ inputs.target-device }}${BUILD_MODE_STR}${WITH_TESTS_STR}-${{ github.sha }}" >> $GITHUB_ENV
63-
echo "BINDINGS_ARTIFACTS_DIR=$(realpath "$(pwd)/cuda_bindings/dist")" >> $GITHUB_ENV
64-
echo "CORE_ARTIFACT_NAME=${{ inputs.host-platform }}-${{ inputs.build-type }}-cuda_core-python${PYTHON_VERSION_FORMATTED}-${{ inputs.target-device }}${BUILD_MODE_STR}${WITH_TESTS_STR}-${{ github.sha }}" >> $GITHUB_ENV
65-
echo "CORE_ARTIFACTS_DIR=$(realpath "$(pwd)/cuda_core/dist")" >> $GITHUB_ENV
66-
echo "USE_CUDA=${{ (inputs.target-device == 'cpu' && 'OFF') || 'ON' }}" >> $GITHUB_ENV
132+
echo "PARALLEL_LEVEL=$(nproc)" >> $GITHUB_ENV
133+
echo "REPO_DIR=$REPO_DIR" >> $GITHUB_ENV
134+
echo "PKG_DIR=${PKG_DIR}" >> $GITHUB_ENV
135+
echo "CUDA_CORE_ARTIFACT_NAME=cuda-core-python${PYTHON_VERSION_FORMATTED}-${{ inputs.host-platform }}-${{ inputs.build-type }}-${{ github.sha }}" >> $GITHUB_ENV
136+
echo "CUDA_CORE_ARTIFACTS_DIR=$(realpath "$REPO_DIR/cuda_core/dist")" >> $GITHUB_ENV
137+
echo "CUDA_BINDINGS_ARTIFACT_NAME=cuda-bindings-python${PYTHON_VERSION_FORMATTED}-cuda${{ inputs.cuda-version }}-${{ inputs.host-platform }}-${{ inputs.build-type }}-${{ github.sha }}" >> $GITHUB_ENV
138+
echo "CUDA_BINDINGS_ARTIFACTS_DIR=$(realpath "$REPO_DIR/cuda_bindings/dist")" >> $GITHUB_ENV
67139
echo "UPLOAD_ENABLED=${{ (inputs.upload-enabled == 'true' && 'ON') || 'OFF' }}" >> $GITHUB_ENV
68-
echo "LEGATE_CORE_BUILD_MODE=${BUILD_MODE}" >> $GITHUB_ENV
69140
echo "BUILD_DATE=$(date +%Y%m%d)" >> $GITHUB_ENV
70141
echo "TARGET_PLATFORM=${TARGET_PLATFORM}" >> $GITHUB_ENV
71-
echo "PKG_DIR=${PKG_DIR}" >> $GITHUB_ENV
72-
echo "PYTHON_VERSION=${{ inputs.python-version }}" >> $GITHUB_ENV
142+
echo "CIBW_BUILD=${CIBW_BUILD}" >> $GITHUB_ENV

‎.github/workflows/ci-gh.yml

+11-2
Original file line numberDiff line numberDiff line change
@@ -18,15 +18,23 @@ jobs:
1818
matrix:
1919
host-platform:
2020
- linux-x64
21+
- linux-aarch64
22+
- win-x64
2123
target-device:
2224
- gpu
2325
build-mode:
2426
- release
2527
upload-enabled:
2628
- false
2729
python-version:
28-
#TODO cover the whole python and cuda matrix
29-
- 3.12
30+
- "3.12"
31+
- "3.11"
32+
- "3.10"
33+
- "3.9"
34+
cuda-version:
35+
# Note: this is for build-time only; the test-time matrix needs to be
36+
# defined separately.
37+
- "12.6.2"
3038
uses:
3139
./.github/workflows/gh-build-and-test.yml
3240
with:
@@ -36,4 +44,5 @@ jobs:
3644
build-type: ci
3745
upload-enabled: ${{ matrix.upload-enabled }}
3846
python-version: ${{ matrix.python-version }}
47+
cuda-version: ${{ matrix.cuda-version }}
3948
secrets: inherit

‎.github/workflows/gh-build-and-test.yml

+13-8
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
on:
22
workflow_call:
33
inputs:
4-
host-platform:
4+
target-device:
55
type: string
66
required: true
7-
target-device:
7+
build-type:
88
type: string
99
required: true
10-
build-mode:
10+
host-platform:
1111
type: string
1212
required: true
13-
build-type:
13+
build-mode:
1414
type: string
1515
required: true
1616
upload-enabled:
@@ -19,6 +19,10 @@ on:
1919
python-version:
2020
type: string
2121
required: true
22+
cuda-version:
23+
type: string
24+
required: true
25+
2226
jobs:
2327
build:
2428
if: ${{ github.repository_owner == 'nvidia' }}
@@ -28,13 +32,14 @@ jobs:
2832
client-repo: ${{ github.event.repository.name }}
2933
target-device: ${{ inputs.target-device }}
3034
runs-on: ${{ (inputs.host-platform == 'linux-x64' && 'linux-amd64-cpu8') ||
31-
(inputs.host-platform == 'linux-aarch64' && 'linux-arm64-cpu8') }}
35+
(inputs.host-platform == 'linux-aarch64' && 'linux-arm64-cpu8') ||
36+
(inputs.host-platform == 'win-x64' && 'windows-2019') }}
37+
# (inputs.host-platform == 'win-x64' && 'windows-amd64-cpu8') }}
3238
build-type: ${{ inputs.build-type }}
33-
use-container: ${{ inputs.host-platform == 'linux-x64' ||
34-
inputs.host-platform == 'linux-aarch64'}}
3539
host-platform: ${{ inputs.host-platform }}
36-
dependencies-file: ""
3740
build-mode: ${{ inputs.build-mode }}
3841
upload-enabled: ${{ inputs.upload-enabled }}
3942
python-version: ${{ inputs.python-version }}
43+
cuda-version: ${{ inputs.cuda-version }}
44+
dependencies-file: ""
4045
secrets: inherit

0 commit comments

Comments
 (0)