Cargo creates unremovable files when run in a Windows container

[yodaldevoid]

If within a folder mounted from a host within a Windows container and Cargo is used to build a project with a build script or that has a dependency with a build script, the compiled build script is unremovable.

The following docker container was used for testing. It contains the bare minimum required to compile Rust on Windows using Rust 1.50.0 and Rustup 1.23.1.

Dockerfile

# escape=`

# IMAGES BUILT FROM THIS DOCKERFILE ARE NOT TO BE SHARED ON PUBLIC DOCKER HUBs.
#
# WARNING: NON-FREE AND DISTRIBUTION RESTRICTED ENCUMBERED SOFTWARE IN IMAGE.
#
# The VS Build Tools are very encumbered.
#
# IMAGE SHOULD ONLY BE BUILT BY USERS WHO CAN FOLLOW THESE TERMS:
# https://visualstudio.microsoft.com/license-terms/mlt553512/
#
# See the fourth paragraph of this blog post for a warning from a Microsoftie:
# https://blogs.msdn.microsoft.com/vcblog/2018/08/13/using-msvc-in-a-docker-container-for-your-c-projects/
#
# """
# The VS Build Tools are licensed as a supplement to your existing Visual Studio license.
# Any images built with these tools should be for your personal use or for use in your
# organization in accordance with your existing Visual Studio and Windows licenses.
# Please don’t share these images on a public Docker hub.
# """
#
# That said, this Dockerfile will be built in a CI system for validation and testing in the project
# but most definitely will skip deployment.

FROM mcr.microsoft.com/windows/servercore:1809

SHELL ["powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

RUN $url = 'https://download.visualstudio.microsoft.com/download/pr/20130c62-1bc8-43d6-b4f0-c20bb7c79113/7276a7355219f7988c480d198e23c2973bbb7ab971c4f0415c26cab2955344e5/vs_BuildTools.exe'; `
    $sha256 = '7276A7355219F7988C480D198E23C2973BBB7AB971C4F0415C26CAB2955344E5'; `
    Invoke-WebRequest -Uri $url -OutFile C:\vs_BuildTools.exe; `
    $actual256 = (Get-FileHash vs_BuildTools.exe -Algorithm sha256).Hash; `
    if ($actual256 -ne $sha256) { `
       Write-Host 'FAILED!'; `
       Write-Host ('expected: {0}' -f $sha256); `
       Write-Host ('got:      {0}' -f $actual256); `
       exit 1; `
    }; `
    Start-Process -filepath C:\vs_BuildTools.exe -wait -argumentlist ' `
        --quiet --wait --norestart --nocache `
        --installPath C:\BuildTools `
        --add Microsoft.Component.MSBuild `
        --add Microsoft.VisualStudio.Component.Windows10SDK.17763 `
        --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64'; `
    Remove-Item 'C:\\vs_BuildTools.exe'; `
    Remove-Item -Force -Recurse 'C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer'; `
    [Environment]::SetEnvironmentVariable('__VSCMD_ARG_NO_LOGO', '1', [EnvironmentVariableTarget]::Machine)

ENV RUSTUP_HOME=C:\rustup `
    CARGO_HOME=C:\cargo `
    RUST_VERSION=1.50.0

RUN [Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12; `
    $url = 'https://static.rust-lang.org/rustup/archive/1.23.1/x86_64-pc-windows-msvc/rustup-init.exe'; `
    $sha256 = 'a586cf9de3e4aa791fd5796b6a5f99ca05591ccef8bb94e53af5b69f0261fb03'; `
    Invoke-WebRequest -Uri $url -OutFile C:\rustup-init.exe; `
    $actual256 = (Get-FileHash rustup-init.exe -Algorithm sha256).Hash; `
    if ($actual256 -ne $sha256) { `
       Write-Host 'FAILED!'; `
       Write-Host ('expected: {0}' -f $sha256); `
       Write-Host ('got:      {0}' -f $actual256); `
       exit 1; `
    }; `
    New-Item ${env:CARGO_HOME}\bin -type directory | Out-Null; `
    $newPath = ('{0}\bin;{1}' -f ${env:CARGO_HOME}, ${env:PATH}); `
    [Environment]::SetEnvironmentVariable('PATH', $newPath, [EnvironmentVariableTarget]::Machine); `
    [Environment]::SetEnvironmentVariable('PATH', $newPath, [EnvironmentVariableTarget]::Process); `
    C:\rustup-init.exe -y -v --no-modify-path --default-toolchain ${env:RUST_VERSION} --default-host x86_64-pc-windows-msvc; `
    Remove-Item C:\rustup-init.exe; `
    rustup -V; `
    cargo -V; `
    rustc -V

The Dockerfile is built as such.

docker build -t rust_build_container .

Starting from a new binary project, we add the simplest build script possible.

cargo new rust_build_test

build.rs

fn main() {}

The container was then run with the following command. This mounts the newly created rust project within the container and then starts the shell within that mounted directory.

docker run -v "${PWD}:C:\rust_build_test" -w "C:\rust_build_test" --name rust_build_test -i rust_build_container

Within the container, cargo build runs without problem, but a subsequent cargo test fails to run because it cannot remove the compiled build script. Attempting to remove the compiled build script any other was also doesn't work for the same reason.

C:\rust_build_test>cargo build
cargo build
   Compiling rust_build_test v0.1.0 (C:\rust_build_test)
    Finished dev [unoptimized + debuginfo] target(s) in 0.97s

C:\rust_build_test>cargo test
cargo test
error: failed to remove file `C:\rust_build_test\target\debug\build\rust_build_test-0b2af87b09108863\build-script-build.exe`

Caused by:
  Access is denied. (os error 5)

C:\rust_build_test>del C:\rust_build_test\target\debug\build\rust_build_test-0b2af87b09108863\build-script-build.exe
del C:\rust_build_test\target\debug\build\rust_build_test-0b2af87b09108863\build-script-build.exe
C:\rust_build_test\target\debug\build\rust_build_test-0b2af87b09108863\build-script-build.exe
Access is denied.

C:\rust_build_test>

If the container is exited and then re-entered, the problematic files can be removed without issue. Similarly, the files can easily be removed from the host's side. Other files created by cargo during the build process can be removed without issue.

If the target folder is placed outside of a mounted location, the commands work as expected.

C:\rust_build_test>cargo build --target-dir C:\target
cargo build --target-dir C:\target
   Compiling rust_build_test v0.1.0 (C:\rust_build_test)
    Finished dev [unoptimized + debuginfo] target(s) in 0.61s

C:\rust_build_test>cargo test --target-dir C:\target
cargo test --target-dir C:\target
   Compiling rust_build_test v0.1.0 (C:\rust_build_test)
    Finished test [unoptimized + debuginfo] target(s) in 0.18s
     Running C:\target\debug\deps\rust_build_test-2ff26d71050a7530.exe

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s


C:\rust_build_test>

---

docker version

> docker version
Client: Docker Engine - Community
 Cloud integration: 1.0.7
 Version:           20.10.2
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        2291f61
 Built:             Mon Dec 28 16:14:16 2020
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.2
  API version:      1.41 (minimum version 1.24)
  Go version:       go1.13.15
  Git commit:       8891c58
  Built:            Mon Dec 28 16:26:48 2020
  OS/Arch:          windows/amd64
  Experimental:     false

docker info

> docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.5.0)

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 6
 Server Version: 20.10.2
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: hyperv
 Kernel Version: 10.0 19041 (19041.1.amd64fre.vb_release.191206-1406)
 Operating System: Windows 10 Pro Version 2004 (OS Build 19041.804)
 OSType: windows
 Architecture: x86_64
 CPUs: 16
 Total Memory: 31.95GiB
 Name: Monamo
 ID: ZSNU:YNGN:TX4M:3TMS:A3K6:HSNL:5NBW:OFI2:EX2A:PYJI:WEBV:FPMD
 Docker Root Dir: C:\ProgramData\Docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

[Evian-Zhang]

Same problem here. This issue may be related.

[NuSkooler]

Any update on this? We are running into the same issue. Here is an excerpt from the logs:

$ cargo test --workspace
 Downloading crates ...

...

error: failed to remove file `C:\builds\redacted\target\debug\build\proc-macro2-b0741c5a699756da\build-script-build.exe`
Caused by:
  Access is denied. (os error 5)