Describe how to use it with MSVC

Hello everyone.
I tried to find any information about coupling MSVC with sccache, but I failed.
It would be great to have a page describing how to setup it, debug it and etc.

Hi there! sccache works very similarly to ccache, in that you simply take a compile commandline and prepend sccache to it, like:
cl.exe -c -Fotest.obj test.c
becomes: sccache.exe cl.exe -c -Fotest.obj test.c

How you would do this in a specific project's build system depends on the build system in use.

@luser

@luser,
Have you used it with DevEnv or with Visual Studio?
This information can be helpful more than only for me.

I have not, sorry! I don't know if it's even possible to make Visual Studio wrap compilations like that.

I tried to find any information and I failed.
That's why I am asking about tutorial about setting sccache up for MSVC.

I found some information about it.
Right now I am testing it.
I will open a PR if it works.

@KalitaAlexey

Thanks @KalitaAlexey. I'd be curious to see what you're trying out, even if it doesn't end up working.

Absent a better solution, we could implement a "stand-in" mode like cclash does, replacing cl.exe with the sccache binary and forwarding calls to a renamed version of the actual compiler (e.g. cl_real.exe). It's an ugly hack, but it should work with Visual Studio even when nothing else does.

I'd also suggest changing the docs to differentiate MSVC support from Visual Studio support, since that's likely to confuse people.

Absent a better solution, we could implement a "stand-in" mode like cclash does, replacing cl.exe with the sccache binary and forwarding calls to a renamed version of the actual compiler (e.g. cl_real.exe). It's an ugly hack, but it should work with Visual Studio even when nothing else does.

sccache does support this already--if you copy the sccache.exe binary to cl.exe it will run as if you ran sccache.exe cl.exe, and use the next cl.exe available in your path.

I'd also suggest changing the docs to differentiate MSVC support from Visual Studio support, since that's likely to confuse people.

The docs currently say "Sccache (tries to) support gcc, clang and MSVC. If you don't specify otherwise, sccache will use a local disk cache." It's perhaps not incredibly clear. If you have a suggestion for making that clearer I'd be happy to take a PR to change it!

sccache does support this already--if you copy the sccache.exe binary to cl.exe it will run as if you ran sccache.exe cl.exe, and use the next cl.exe available in your path.

I'm not entirely sure how that would work, at least by default. If I understand the VS2017 instillation correctly, it installs four copies of cl.exe by default, none of which are added to the system path (x86, x64, x86_on_64, x64_on_86). Visual Studio calls these directly, not by searching the path. If you replace one of the cl.exe binaries with a renamed sccache.exe, it can't call the correct cl.exe because you've just replaced it.

cclash seems to get around the path problem by searching the current directory for a copy of the compiler that is not named cl.exe, to avoid naming conflicts. This is the hack I was referring to.

It might be possible to work around this for one of the four compilers by moving the real cl.exe to a subdirectory and adding that to the path, but I'm not sure what else that would break (e.g. the real compiler may be expecting to be in a certain directory).

If you have a suggestion for making that clearer I'd be happy to take a PR to change it!

Will do. :)

An option might be to install a separate MSBuild toolset platform, which would also let you specify the command-line to be something other than cl.exe. You'd want one for each MSVC toolset and platform you want to support, and it would be set up to simply import the normal toolset platform for all the settings and then override the compiler executable path. I think you can override the executable name too, but I'm not certain.

An installer could generate these as well pretty easily, based on the currently-installed toolsets. You could also then offer sccache-specific options in the UI.
If it's not clear what I mean, it's dropping files in (for VS2010-VS2015 x64 target):

C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\(V100|V110|V120|V140)\Platforms\x64\PlatformToolsets

or for VS2017 x64 target:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\VC\VCTargets\Platforms\x64\PlatformToolsets

Both Clang/C2 and Clang/LLVM (clang-cl) use this mechanism to insert their install dirs into the search path before the directory with MSVC cl.exe.

For CMake users, it's then the -T option at generation time. For direct Visual Studio users, there's a drop-down in the solution properties to select the desired toolset.

@TBBle

@TBBle That looks like the best solution, but I dug through the toolchain files and they're a bit of a nightmare, at least for someone who's never touched MSBuild scripting before. I'm finding plenty of documentation on using various toolchains, but nothing so far on writing a new one. Any ideas where to look?

It certainly seems like you can change the executable name, since VS is using clang at C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\ClangC2\14.10.25903\bin\*\clang.exe, with various c2.dll files added to the path depending on the target architecture. However, I haven't managed to find that string yet.

Yeah, I had the same problem trying to work out how it selects 'clang.exe'. The Clang/C2 scripts are going to be hard to read because they're a full stand-alone toolchain, with their own configuration arguments and properties.

As a simpler example of redirection of the toolchain platform toolsets, see Clang/LLVM's scripts at https://github.com/llvm-mirror/llvm/tree/master/tools/msbuild. They stamp out a .props file for each platform toolset, which inherits from the native toolset and then changes the executable path to find clang-cl's cl.exe instead.

It also adds a command-line parameter since clang-cl needs to know the version of cl.exe which it is emulating.

The basic procedure is:

for every platform toolset you want to wrap
** five MSBuild-based toolchains so far; v100, v110, v120, v140, v141; three platforms: Win32, x64 and ARM; three MS-provided variants: v1xx, v1xx_clang_c2 and v1xx_xp (not v100): 45 hits on the matrix, although not all of those exist or make sense to support here.
create a new platform toolset which includes the relevant targets and props of the toolset you're wrapping, and then just override/extend the one or two properties needed (e.g. prepend to the executable path).
Install sccache as cl.exe somewhere in prepended path directory using the existing argv[0] detection.
** If you can work out how to make MSVC use a different executable name, you can do that instead. (I'm sure Clang/LLVM's scripts would like that too. ^_^)
** Perhaps support for sccache.exe as clang.exe if not already present, to support both clang/c2 and non-MSBuild users of clang/LLVM (who also have a clang-cl.exe available if I recall correctly)

You could also do the same thing to wrap Clang/LLVM's platform toolsets if present.

As a less-intrusive approach, it might be possible to use the 'ImportAfter' directories to override any toolchain you like, but you lose the easy configuration of the toolchain selection.

There are new possibilities now: https://blogs.msdn.microsoft.com/vcblog/2017/06/14/cmake-support-in-visual-studio-whats-new-in-2017-15-3-preview-2

Visual Studio 2017 supports CMake and the version 15.3 (right now in preview 2) supports Ninja as a generator. CMake supports a setting to prefix the compiler invocation with a command. You can then just set it to sccache and it will just work.

Sure, this isn't perfect as the CMake experience is a bit different from the regular msbuild project experience and it doesn't work with older versions of Visual Studio. But it works quite well and it could definitely be a solution moving forward!

Here's a way to change the CL.exe tool path that MSBuild uses without messing with your Visual Studio installation:

msbuild blah.sln /m /p:Platform=x64 /p:Configuration=Debug /t:Rebuild ^
    /p:CLToolExe=cl.exe /p:CLToolPath=C:\tools

Where C:\tools contains cl.exe which is just the renamed sccache.exe.

Unfortunately, there are still more problems with using sccache with msbuild:

sccache doesn't handle response files (e.g., @blah.rsp). msbuild likes to use these behind the scenes.
sccache doesn't handle multiple source files on a single command line.
The sccache server must be started within a VS command prompt so it can find the dependent DLLs for cl.exe (e.g., mspdb140.dll). Alternatively, you can just fuddle with the PATH so it finds dependent DLLs correctly.

sccache doesn't handle response files (e.g., @blah.rsp). msbuild likes to use these behind the scenes.

FYI we do have support for response files for gcc, we could probably expand that to work with MSVC as well.

sccache doesn't handle multiple source files on a single command line.

This would probably be kind of hard to shoehorn into sccache's current model, unless we split it up into one compile job per source file internally and then recombined the results at the end.

The sccache server must be started within a VS command prompt so it can find the dependent DLLs for cl.exe (e.g., mspdb140.dll). Alternatively, you can just fuddle with the PATH so it finds dependent DLLs correctly.

Yeah, this is a bit unfortunate. If you don't run the sccache server manually the first compile invocation will spawn a server which should have the right environment. We actually might be able to fix this, since we do pass the environment from the compiler invocation to the server for use for compilation--I'd guess the only thing that's breaking here is the few compiler invocations the server does to detect the compiler type and showIncludes prefix.

FYI we do have support for response files for gcc, we could probably expand that to work with MSVC as well.

Cool. I'll take a shot at implementing this for MSVC.

sccache doesn't handle multiple source files on a single command line.

This would probably be kind of hard to shoehorn into sccache's current model, unless we split it up into one compile job per source file internally and then recombined the results at the end.

Maybe once I get more familiar with the code, I can take a shot at this too if such a change would be welcome. No promises though.

sccache does support this already--if you copy the sccache.exe binary to cl.exe it will run as if you ran sccache.exe cl.exe, and use the next cl.exe available in your path.

It might be possible to work around this for one of the four compilers by moving the real cl.exe to a subdirectory and adding that to the path, but I'm not sure what else that would break (e.g. the real compiler may be expecting to be in a certain directory).

I have done this. I have moved dir/cl.exe to dir/test/cl.exe where dir is C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin. Then I copied the sccache.exe to dir and renamed it to cl.exe. Then I added dir/test and dir to the PATH. The problem now is that when I try to build a project I get the error:

1>------ Build started: Project: lib, Configuration: Debug Win32 ------
1>CL : error : failed to execute compile
1>  caused by: Compiler not supported: "Failed to detect showIncludes prefix"
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

Does anyone know how to get this to work?

To sum up, is there an easy way to use sccache in Visual Studio without modifying every projects ?
You mentioned that we can replace cl.exe ; however is this compatible with building against multiple platforms ? (x64, x86, ...)
I would be very interested in having a clear procedure.

Implement Response File support in MSVC

One way to connect sccache with MSVC/MSBuild without tricks such as replacing cl.exe with sccache.exe is to use a props-file to customize the build. Place the props file in the root of the source directory.

Example contents:

<Project>
<PropertyGroup>
  <CLToolExe>compiler.bat</CLToolExe>
  <CLToolPath>d:\mytools\</CLToolPath>
</PropertyGroup>
</Project>

And the bat file? It could contain

sccache.exe cl.exe %*

Note, for this to work, both exe files has to be in PATH.

Documentation for props files: https://learn.microsoft.com/en-us/visualstudio/msbuild/customize-your-build?view=vs-2022

@UsernameLinde

@UsernameLinde tried your method and I am getting the same old

sccache : error : failed to execute compile
sccache: caused by: Compiler not supported: "Failed to detect showIncludes prefix"

Also, using a batch script seems to have a considerable downside. When attempting to cancel a built it will prompt to terminate the batch job. A prompt that can never be acted upon and so effectively you cannot cancel the build.

May be suitable for quick small builds, but this is really nasty when you have huge builds with many many big files involved.

So I suppose the proper method would be to copy (or hardlink) sccache into place under the name of the respective tool(s) and prepend to PATH (e.g. using the SetEnv task).

Alas, there's one piece missing currently: while one can detect whether sccache is in the PATH by attempting to execute it from MSBuild (e.g. sccache --version), ignoring any odd exit code, then acting upon it, there is no nice way of determining the path of the invoked sccache.exe from within MSBuild.

I would strongly suggest to introduce something along the lines of git --exec-path, but returning the full file path to sccache.exe so that could be used to determine the path to sccache. If done properly, this would have to be done only once and one could save the sccache --version invocation altogether.

@exoosh

@exoosh one addition I can make to the short props file sample above is this line:
<UseMultiToolTask>true</UseMultiToolTask>

With that VS/MSBuild should compile one file at a time and hopefully it will be faster due to parallelism. Another thing it can help is that it is easier to cancel builds.

Regarding the error with showIncludes prefix, sorry, don't know why that is happening or what is wrong.

@exoosh

@exoosh one addition I can make to the short props file sample above is this line: <UseMultiToolTask>true</UseMultiToolTask>

Thanks. I'll give it a try. But I'm afraid this won't change the Ctrl+C behavior at all. It doesn't really matter if a batched job of a dozen or more files or a single one cannot be canceled.

With that VS/MSBuild should compile one file at a time and hopefully it will be faster due to parallelism. Another thing it can help is that it is easier to cancel builds.

Yep, I was aware of the tool, but it has other issues in relation to tools which some colleagues are using and need to work with.

Regarding the error with showIncludes prefix, sorry, don't know why that is happening or what is wrong.

Thought as much. Thanks for your response nevertheless!

Perhaps this one-liner at the top of the batch script will fix the "Terminate Y/N" blockage?

@TBBle

@TBBle Thanks (personal account of @exoosh here), but that's also somewhat suboptimal. I like the style, but not for this sort of task. After all it ends up calling the script itself and forcing standard input to be wired to NUL (to prevent the prompt), that's all it does. Then it simply prevents the called script from executing itself again based on an environment variable being set (i.e. limiting to one level of recursion). It doesn't seem very robust to me.

However, I left a comment in #107 on which I will extend later. This seems more promising to me and it has the flexibility I was hoping for.

I managed to have something working for MSVC + MSBUILD with the following Directory.Build.props at the root of my source folder:

<?xml version="1.0" encoding="utf-8"?>
<Project>
    <PropertyGroup>
        <CLToolExe>cl.exe</CLToolExe>
        <CLToolPath>$(MSBuildThisFileDirectory)\sccache</CLToolPath>
        <UseMultiToolTask>true</UseMultiToolTask>
        <EnforceProcessCountAcrossBuilds>true</EnforceProcessCountAcrossBuilds>
    </PropertyGroup>
</Project>

(my sccache executable is relative to this file, but it can also be an absolute path)

sccache/cl.exe is a renamed sccache.exe. The real cl.exe need to be in PATH (it will be the case if MSBuild is run within VS).
As mentioned above, UseMultiToolTask is mandatory for caching (otherwise VS sends the whole project to cl.exe, leading to multiple input files error on sccache side).

However I encountered several issues, for which I proposed a fix: #1834 #1835 #1836 (including a fix that works for me for the mentioned /showIncludes issue).

@AudranDoublet

@AudranDoublet thanks. In terms of MSBuild this of course suppresses MSBuild batching, which is this exact thing where it passes multiple files at once into one cl.exe invocation.

Perhaps this one-liner at the top of the batch script will fix the "Terminate Y/N" blockage?

As a side-note: this has potentially undesired side effects. For example the timeout command -- which ships with Windows -- expects stdin to be functional and errors out if it is not.

Is this still the best known method to use sccache with MSVC? Is there a fix for the /showIncludes issue?

I'm trying to speed up duckdb's windows build. Logs here

Hi!

trying sccache right now on a simple code:

int main(int, char**)
{
int retVal = 0;

return retVal;
}

From Developer Command Prompt inside VS 2022:

C:\repos\test>"C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.43.34808\bin\Hostx64\x64\cl.exe" /showIncludes /c C:\repos\test\fake\fake.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.43.34810 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

fake.cpp

C:\repos\test>"C:\repos\sccache\sccache.exe" "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.43.34808\bin\Hostx64\x64\cl.exe" /showIncludes /c C:\repos\test\fake\fake.cpp
sccache: error: failed to execute compile
sccache: caused by: Compiler not supported: "Failed to detect showIncludes prefix"

The same failure happens if I add an include in the .cpp file.

Is this a common issue still? Any way to solve it?

Thank you

UPDATE 1:
Solved downloading the main branch of the repo and building it on my own pc.

UPDATE 2:
After few builds it stopped and went back to the showInclude failure.

feat(transport): Change channel connect to be async (mozilla#107)

Navigation Menu

Repository navigation

Describe how to use it with MSVC #107

4 remaining items

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Describe how to use it with MSVC #107

Description

Activity

luser commented on Apr 19, 2017

KalitaAlexey commented on Apr 19, 2017

luser commented on Apr 19, 2017

KalitaAlexey commented on Apr 19, 2017

KalitaAlexey commented on Apr 21, 2017

kwaegel commented on May 8, 2017

luser commented on May 8, 2017

kwaegel commented on May 9, 2017

TBBle commented on May 18, 2017

kwaegel commented on May 24, 2017

TBBle commented on May 31, 2017

Orphis commented on Jun 15, 2017

jasonwhite commented on Oct 17, 2017

luser commented on Oct 17, 2017

jasonwhite commented on Oct 17, 2017

4 remaining items

autoantwort commented on Aug 27, 2020

nmoreaud commented on Oct 27, 2020

UsernameLinde commented on Mar 27, 2023

exoosh commented on Jun 12, 2023

UsernameLinde commented on Jun 13, 2023

exoosh commented on Jun 13, 2023

TBBle commented on Jun 13, 2023

assarbad commented on Jun 13, 2023

AudranDoublet commented on Jul 21, 2023

exoosh commented on Jul 24, 2023

exoosh commented on Jul 24, 2023

adsharma commented on Jun 2, 2025

Stefano-Bertozzi-OM commented on Jun 30, 2025

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions