Repository navigation
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## rust-direct-mode #2943 +/- ##
=================================================
Coverage 75.53% 75.53%
=================================================
Files 74 74
Lines 44177 44177
=================================================
+ Hits 33369 33370 +1
+ Misses 10808 10807 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
sylvestre
force-pushed
the
pgo-build
branch
3 times, most recently
from
October 24, 2026 18:00
3758950 to
619521b
Compare
sylvestre
marked this pull request as ready for review
October 25, 2026 08:56
sylvestre
force-pushed
the
pgo-build
branch
2 times, most recently
from
October 25, 2026 22:08
684abd0 to
95379fd
Compare
scripts/build-pgo.sh builds an instrumented sccache with the same LTO and codegen-unit settings as the release build, trains it on the integration tests (system, sccache_cargo, sccache_rustc, cache_hit_rate, sccache_args) and a generated C/C++ corpus of cache misses and hits, merges the profile and builds the optimized binary. --train-only stops after the profile, for CI to pass -Cprofile-use itself; --bench also builds a plain binary and compares the two with scripts/bench-pgo.sh, which uses hyperfine. The test executables are built with `cargo test --no-run`, but the sccache they drive is then rebuilt with `cargo build`: under `cargo test`, dev-dependencies unify extra features into shared crates, which renames their symbols and sccache's, and the profile would not match the release build. The test harnesses are instrumented too, so only the profile written by sccache itself is kept. On Windows, std::process::exit skips the atexit handler the profiling runtime writes the counters from, so the instrumented build (--cfg pgo_training) writes them before exiting. On Linux x86_64, PGO cuts the server's instructions per direct-mode cache hit by 3-5%: most of its time is in blake3, libc and the kernel, which the profile does not change. It is not enabled in CI.
On tags and on main, the x86_64 Linux (musl), macOS (arm64 and x86_64) and x86_64 Windows sccache builds train a profile with scripts/build-pgo.sh, then build the uploaded binary with -Cprofile-use, after checking that it made it into RUSTFLAGS. Those are the released targets whose runner can execute the binary it builds, which training needs. Pull requests keep the plain build, as do the cross-compiled targets and sccache-dist. The musl container gets gcc and g++ so that the training covers C/C++ compiles too, and the PGO jobs get a longer timeout. build-pgo.sh now builds the integration test executables without LTO: they are not trained on, and the sccache they drive is rebuilt with the release settings anyway.
The profile format changes between LLVM major versions, and an llvm-profdata from another one either fails to merge what the instrumented binary wrote, or writes a profile rustc cannot use. Either way that only showed up after the instrumented build and the training. Compare the major LLVM versions reported by `rustc -vV` and `llvm-profdata --version` as soon as llvm-profdata is found, and stop with the version to use if they differ. That matters mostly without rustup, when llvm-profdata comes from the distribution through --llvm-profdata.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a script to build sccache with profile-guided optimization
scripts/build-pgo.sh builds an instrumented sccache with the same LTO and
codegen-unit settings as the release build, trains it on the integration
tests (system, sccache_cargo, sccache_rustc, cache_hit_rate, sccache_args)
and a generated C/C++ corpus of cache misses and hits, merges the profile
and builds the optimized binary. --train-only stops after the profile, for
CI to pass -Cprofile-use itself; --bench also builds a plain binary and
compares the two with scripts/bench-pgo.sh, which uses hyperfine.
The test executables are built with
cargo test --no-run, but the sccachethey drive is then rebuilt with
cargo build: undercargo test,dev-dependencies unify extra features into shared crates, which renames
their symbols and sccache's, and the profile would not match the release
build. The test harnesses are instrumented too, so only the profile written
by sccache itself is kept.
On Windows, std::process::exit skips the atexit handler the profiling
runtime writes the counters from, so the instrumented build
(--cfg pgo_training) writes them before exiting.
On Linux x86_64, PGO cuts the server's instructions per direct-mode cache
hit by 3-5%: most of its time is in blake3, libc and the kernel, which the
profile does not change. It is not enabled in CI.
ci: build the released sccache binaries with PGO
On tags and on main, the x86_64 Linux (musl), macOS (arm64 and x86_64) and
x86_64 Windows sccache builds train a profile with scripts/build-pgo.sh,
then build the uploaded binary with -Cprofile-use, after checking that it
made it into RUSTFLAGS. Those are the released targets whose runner can
execute the binary it builds, which training needs. Pull requests keep the
plain build, as do the cross-compiled targets and sccache-dist.
The musl container gets gcc and g++ so that the training covers C/C++
compiles too, and the PGO jobs get a longer timeout.
build-pgo.sh now builds the integration test executables without LTO: they
are not trained on, and the sccache they drive is rebuilt with the release
settings anyway.
build-pgo.sh: check that llvm-profdata matches the LLVM of rustc
The profile format changes between LLVM major versions, and an llvm-profdata
from another one either fails to merge what the instrumented binary wrote,
or writes a profile rustc cannot use. Either way that only showed up after
the instrumented build and the training.
Compare the major LLVM versions reported by
rustc -vVandllvm-profdata --versionas soon as llvm-profdata is found, and stop withthe version to use if they differ. That matters mostly without rustup, when
llvm-profdata comes from the distribution through --llvm-profdata.