-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
PyPI Release Setup for C++ Extension Module (#81)
* wip * debug recall * clean up * cleaning up * some refactoring * some cmake changes * revert back to CMake v3.14 * work in progress building library for pypi release * wip * add script for building wheels * workflow for pushing linux and macos wheels to pypi * remove unnecessary change * remove extra pyproject.toml * fix pyton library unit tests * fix small typo * increment version number * fix workflow file * fail fast set to false * fix wheel key logic * make sure xcode is installed in the macos wheel builder * fix build step for mac * remove -march=native flag from the macos setup * fix project setup * upload wheels individually * fix workflow file * Update docker-run.sh * Update run-benchmark.py --------- Co-authored-by: blaise-muhirwa <[email protected]> Co-authored-by: Vihan Lakshman <[email protected]>
- Loading branch information
3 people
authored
Jan 16, 2025
1 parent
231b423
commit 207dc6d
Showing
49 changed files
with
4,106 additions
and
276 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| name: Build and Publish Wheels | ||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| # Trigger on version tags (e.g., v1.0.0) | ||
| - 'v*' | ||
|
|
||
| env: | ||
| PYTHON_VERSION: 3.11 | ||
|
|
||
| jobs: | ||
| build_wheels: | ||
| name: Build wheels on ${{ matrix.os }} for Python ${{ matrix.python }} | ||
| runs-on: ${{ matrix.os }} | ||
| timeout-minutes: 60 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest] | ||
| python: ['3.8', '3.9', '3.10', '3.11', '3.12'] | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: 'recursive' | ||
|
|
||
| # Set up Python 3.11 specifically for running cibuildwheel | ||
| - name: Set up global python | ||
| uses: actions/setup-python@v4 | ||
| with: | ||
| python-version: ${{ env.PYTHON_VERSION }} | ||
|
|
||
| - name: Install macOS dependencies | ||
| if: runner.os == 'macOS' | ||
| run: | | ||
| xcode-select --install || true | ||
| brew install cmake libomp | ||
| - name: Install Dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install "cibuildwheel==2.22.0" | ||
| - name: Set up Docker (Linux only) | ||
| if: runner.os == 'Linux' | ||
| uses: docker/setup-buildx-action@v3 | ||
| with: | ||
| install: true | ||
|
|
||
| - name: Build wheel | ||
| env: | ||
| MACOSX_DEPLOYMENT_TARGET: '10.14' | ||
| run: | | ||
| set -ex | ||
| ./cibuild.sh --current-version ${{ matrix.python }} | ||
| - name: Upload wheels as artifacts | ||
| uses: actions/upload-artifact@v3 | ||
| if: always() | ||
| with: | ||
| name: wheels-${{ runner.os }}-py${{ matrix.python }} | ||
| path: wheelhouse/*.whl | ||
|
|
||
| publish: | ||
| needs: build_wheels | ||
| if: always() | ||
| runs-on: ubuntu-latest | ||
| defaults: | ||
| run: | ||
| working-directory: python-bindings | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: 'recursive' | ||
|
|
||
| - name: Install pipx | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| python -m pip install pipx | ||
| - name: Clear dist folder and build SDist | ||
| run: | | ||
| rm -rf dist/ | ||
| pipx run build --sdist | ||
| - name: Download wheels | ||
| uses: actions/download-artifact@v3 | ||
| with: | ||
| path: python-bindings/dist/ | ||
|
|
||
| - name: Prepare dist directory and upload wheels individually | ||
| env: | ||
| TWINE_USERNAME: __token__ | ||
| TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} | ||
| run: | | ||
| # Move wheels to dist directory | ||
| find dist -name "*.whl" -exec mv {} dist/ \; | ||
| rm -rf dist/wheels-* | ||
| # Verify we have wheels to publish | ||
| wheel_count=$(find dist -name "*.whl" | wc -l) | ||
| if [ "$wheel_count" -eq 0 ]; then | ||
| echo "No wheels found to publish" | ||
| exit 1 | ||
| fi | ||
| echo "Found $wheel_count wheels to publish:" | ||
| ls -la dist/*.whl | ||
| # Upload each wheel individually | ||
| failed_uploads=0 | ||
| for wheel in dist/*.whl; do | ||
| echo "Uploading $wheel..." | ||
| if ! pipx run twine upload "$wheel"; then | ||
| echo "Warning: Failed to upload $wheel" | ||
| failed_uploads=$((failed_uploads + 1)) | ||
| fi | ||
| done | ||
| # Upload sdist last | ||
| echo "Uploading sdist..." | ||
| if ! pipx run twine upload dist/*.tar.gz; then | ||
| echo "Warning: Failed to upload sdist" | ||
| failed_uploads=$((failed_uploads + 1)) | ||
| fi | ||
| # Report final status | ||
| echo "Upload complete. $failed_uploads files failed to upload." | ||
| if [ "$failed_uploads" -eq "$((wheel_count + 1))" ]; then | ||
| echo "All uploads failed" | ||
| exit 1 | ||
| fi | ||
| if [ "$failed_uploads" -gt 0 ]; then | ||
| echo "Some uploads failed but others succeeded" | ||
| exit 0 | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -34,3 +34,6 @@ docs/doxygen_output | |
| data/ | ||
| baselines/ | ||
| external/openmp | ||
|
|
||
| # Python wheels | ||
| wheelhouse | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.