Skip to content

Caching.md

Latest commit

 

History

History
61 lines (47 loc) · 2.43 KB

Caching.md

File metadata and controls

61 lines (47 loc) · 2.43 KB
Raw

How caching works

To know if the storage contains the artifact we need, we are computing some hashes to make sure the input is the same.

Because the configuration and environment matter, the hash computation takes a few parameters into account.

How hash keys are computed.

Rust

We generate a blake3 digest for each file compiled. In parallel, we also take into account in the hash:

  • Path to the rustc executable
  • Host triple for this rustc
  • Path to the rustc sysroot
  • digests of all the shared libraries in rustc's $sysroot/lib
  • A shared, caching reader for rlib dependencies (for dist-client)
  • Parsed arguments from the rustc invocation

See

sccache/src/compiler/rust.rs

Line 122 in 8567bbe

pub struct ParsedArguments {
for the full list

C/C++ compiler

For C/C++, the hash is generated with a blake3 digest of the preprocessed file (-E with gcc/clang). Compilations that specify several different -arch flags are only cached when the environment variable SCCACHE_CACHE_MULTIARCH is set. The file is then preprocessed once per architecture, with only that -arch flag, and every output goes into the hash, so that code only one of the targets sees (e.g. under __aarch64__) is taken into account.

We also take into account in the hash:

  • Hash of the compiler binary
  • Hash of the assembler binary the compiler hands the compilation off to, and the version it reports, when there is one: GCC always assembles that way, clang only with -fno-integrated-as
  • Programming language
  • Flag required to compile for the given language
  • File in which to generate dependencies.
  • Commandline arguments for dependency generation
  • Commandline arguments for the preprocessor
  • Commandline arguments specifying the architecture to compile for
  • Extra files that need to have their contents hashed
  • Whether the compilation is generating profiling or coverage data
  • Color mode
  • Environment variables

See

sccache/src/compiler/c.rs

Line 84 in 8567bbe

pub struct ParsedArguments {

C/C++ preprocessor

In "preprocessor cache mode", explained in the local doc, an extra key is computed to cache the preprocessor output itself. It is very close to the C/C++ compiler one, but with additional elements:

  • The path of the input file
  • The hash of the input file

Note that some compiler options can disable preprocessor cache mode. As of this writing, only -Xpreprocessor and -Wp,* do.