Repository navigation
⚡ Optimize release wheels with SDK/Core PGO (Phase 2) - #2476
Draft
burgholzer wants to merge 12 commits into
Draft
burgholzer wants to merge 12 commits into
burgholzer wants to merge 12 commits into
Conversation
This was referenced Sep 8, 2026
Open
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
burgholzer
force-pushed
the
codex/optimized-release-toolchain
branch
from
September 12, 2026 13:58
706fd8f to
f1eeefe
Compare
7 of 10 tasks
burgholzer
force-pushed
the
codex/optimized-release-toolchain
branch
from
September 14, 2026 18:18
cc686e4 to
3443517
Compare
burgholzer
changed the base branch from
main
to
codex/fix-mlir-exception-boundaries
September 14, 2026 18:18
burgholzer
force-pushed
the
codex/fix-mlir-exception-boundaries
branch
5 times, most recently
from
September 25, 2026 18:40
bf8226b to
3b8d8db
Compare
simon1hofmann
added this pull request to stack #2677
October 5, 2026 11:53
10 tasks done
8 of 10 tasks
Keep cache launchers and project defaults in the source build rather than applying them during find_package. Load install directories only when an explicit device helper needs them, and retain the exported helpers and their targets. Check consumer settings and a device-helper build against the installed package. Assisted-by: GPT-6 via Codex
Replace node-constrained edge members with concrete vector and matrix function overloads so supported compilers agree on the exported symbols. Keep edge layouts, numerical behavior, and Python methods unchanged. Move traversal recursion into the implementation and migrate all C++ callers, including bindings, shared MLIR test helpers, and documentation. Assisted-by: GPT-6 via Codex
Document the new edge parameters required by Doxygen and record the completed DD API migration with its validation and remaining scope. Assisted-by: GPT-6 via Codex
The DD bindings use complex numbers but no declarations from cmath. Remove the two includes flagged by hosted clang-tidy 23.1.3. Assisted-by: GPT-6 via Codex
Use the manylinux Clang distribution and LLD, enable Apple Clang IPO, and retain speed optimization in nanobind bindings. Select portable CPU settings explicitly for wheels while keeping native tuning for source builds. Exercise the installed package and concrete DD API in wheel tests, including GCC consumers on Linux. Document compiler selection, measured tradeoffs, and the limits of LTO across native SDK inputs. Assisted-by: GPT-6 via Codex
burgholzer
force-pushed
the
codex/optimized-release-toolchain
branch
from
October 8, 2026 21:22
6793f09 to
f1846a2
Compare
Restore two-level symbol lookup after HandleLLVMOptions, following the upstream AddMLIRPython helper. Keep binding modules from resolving private symbols against another loaded package. Assisted-by: GPT-6 via Codex
Preserve combined SDK/Core PGO and Linux BOLT while using current public APIs for release training and installed consumers. Retain Phase 1's SDK selection and Windows cleanup, and pin the pending wheel cache workflow. Full optimized-wheel qualification remains pending. The baseline GCC consumer passes; its Clang 23 counterpart exposes a constrained-template ABI mismatch in the GCC 13-built baseline wheel. Assisted-by: GPT-6 via Codex
Record the constrained-template aliases dropped by GCC 14 LTO and the independent ABI checks needed by the Clang PGO/BOLT pipeline. Assisted-by: GPT-6 via Codex
Let CMake own Core IPO in both PGO stages and limit Linux ThinLTO workers through LLD. Keep SDK LTO disabled and preserve profile and BOLT flags. Update release documentation and the installed consumer for the concrete DD API prerequisite. Assisted-by: GPT-6 via Codex
Check repaired wheels with the shared package-policy and DD fixture under both Clang and GCC. Run its CTest checks and remove the duplicate release consumer and its lint-only target. Assisted-by: GPT-6 via Codex
Preserve profile configuration on Linux and macOS while retaining the matched Linux profiling tools and portable Windows settings. Reuse the shared installed consumer, retain the current workflow pins, and document the concrete DD prerequisite and remaining PGO/BOLT qualification. Assisted-by: GPT-6 via Codex
Reject Linux SDKs missing executable BOLT, profile merging, or strip tools before downloading LLVM sources and starting the PGO build. Record the pending SDK prerequisite for full Phase 2 qualification. Assisted-by: GPT-6 via Codex
burgholzer
force-pushed
the
codex/optimized-release-toolchain
branch
from
October 8, 2026 21:28
f1846a2 to
5e75e50
Compare
burgholzer
force-pushed
the
codex/release-wheel-lto
branch
from
October 9, 2026 10:27
8e308fd to
fcf206a
Compare
burgholzer
force-pushed
the
codex/release-wheel-lto
branch
from
October 9, 2026 10:50
fcf206a to
d6d02f3
Compare
simon1hofmann
added this pull request to stack #2721
October 9, 2026 11:49
burgholzer
force-pushed
the
codex/release-wheel-lto
branch
3 times, most recently
from
October 9, 2026 15:28
be7a8e2 to
23fcc22
Compare
burgholzer
force-pushed
the
codex/release-wheel-lto
branch
2 times, most recently
from
October 9, 2026 18:48
83be162 to
5938109
Compare
burgholzer
removed this pull request from stack #2721
October 9, 2026 22:52
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Description
Add combined SDK/Core profile-guided optimization (PGO) to Linux and macOS release wheels. CMake owns Core ThinLTO in the instrumented and final builds; the SDK rebuild uses PGO with LTO disabled. Linux applies BOLT to the final binaries and validates the repaired wheel with training workloads and the shared installed CMake consumer under Clang and GCC.
This draft is stacked on #2715, including #2717's CMake package isolation and #2718's concrete DD operations. It retains portable CPU settings on every wheel platform, the assertion-free LLVM/MLIR 23.1.2 SDK, and the current Workflows #464 compiler-cache and #491 native mold 3 pins. Linux BOLT tooling depends on SDK #94.
Linux uses matched distribution Clang, LLD, compiler-rt, and llvm-profdata packages from manylinux_2_28. The Phase 1 static Clang 22 bundle lacks the profiling tools required here. macOS uses Apple Clang.
scripts/prepare_release.pysupplies the generatedCMAKE_PROJECT_INCLUDE_BEFOREconfiguration for both platforms; Windows clears the profiling hooks and keeps its normal portable build. The rebuilt SDK retains its assertion, exception-handling, and RTTI settings.Validation on the restacked branch:
Linux qualification is currently blocked: the selected released SDK lacks the BOLT tools, and SDK #94 remains open. Provisioning now requires executable
llvm-bolt,merge-fdata, andllvm-stripbefore the LLVM source download or PGO build. The full SDK/Core PGO build, Linux BOLT processing, final optimized-wheel validation, and macOS qualification have not been run on this head. Baseline consumer checks do not establish this pipeline's correctness or performance. No C++ source changes are introduced relative to #2715. AI assistance covered the restack, configuration alignment, review, and focused validation.Checklist
If PR contains AI-assisted content:
🤖 *AI text below* 🤖(titles are exempt).