Skip to content

Add local_build_gems.ps1: run build_gems.ps1 in a Windows Server 2022 Core container - #379

Merged
tpowell-progress merged 8 commits into
18-Stablefrom
tp/local-build-gems-windows2022
Sep 15, 2026
Merged

tpowell-progress merged 8 commits into
18-Stablefrom
tp/local-build-gems-windows2022

Conversation

@tpowell-progress

Copy link
Copy Markdown
Contributor

Replaces #378 (branch renamed from copilot/local-build-gems-windows2022 to tp/local-build-gems-windows2022 to match repo convention; closing that one auto-closed the PR).

Summary

Adds a way to run the exact .expeditor/build_gems.ps1 CI build locally without installing Habitat/Chef-Client/etc. directly on your workstation.

  • docker/windows2022-core/Dockerfile — minimal Windows Server 2022 Core (mcr.microsoft.com/windows/servercore:ltsc2022) image with Chocolatey, Ruby, and Git. Everything else (Habitat, Chef-Client, Node.js) is installed by build_gems.ps1 itself at build time, same as the Buildkite rubydistros/windows-* CI agents.
  • docker/windows2022-core/Install-Ruby.ps1 — resolves RUBY_VERSION (a major.minor prefix like 3.4/3.1) to the newest matching Chocolatey ruby package version at image build time, since Chocolatey's package versions carry a packaging-revision suffix that drifts over time.
  • .expeditor/local_build_gems.ps1 — builds that image, then runs a container that:
    • mounts the repo checkout read-write at C:\workdir
    • mounts a host output directory (-OutputPath, default .\build-output) at C:\output
    • runs .expeditor\build_gems.ps1 inside the container
    • copies the resulting Habitat results/, built .gem, and compiled ruby_bin_folder DLLs into the mounted output directory (even on failure, for debugging)
    • takes a -HabAuthToken parameter (defaults to $env:HAB_AUTH_TOKEN) forwarded into the container, since hab pkg build needs a Habitat Builder token to install chef/hab-studio and warns early if it's missing
  • README updated with usage instructions.

Usage

.\.expeditor\local_build_gems.ps1
.\.expeditor\local_build_gems.ps1 -RubyVersion 3.1 -OutputPath C:\chef-powershell-shim-output

Requires Docker Desktop with Windows containers enabled, and a Habitat Builder personal access token ($env:HAB_AUTH_TOKEN or -HabAuthToken).

@tpowell-progress
tpowell-progress changed the base branch from jfm/18stable-net10-sync to 18-Stable September 11, 2026 15:40
@tpowell-progress
tpowell-progress force-pushed the tp/local-build-gems-windows2022 branch from aa64b70 to 5b582ff Compare September 11, 2026 15:44
@johnmccrae
johnmccrae self-requested a review September 11, 2026 16:58
tpowell-progress and others added 7 commits September 14, 2026 17:04
…22 Core container

Adds docker/windows2022-core/Dockerfile (Windows Server 2022 Core + Chocolatey/Ruby/Git baseline) and .expeditor/local_build_gems.ps1, which builds that image and runs .expeditor/build_gems.ps1 inside a container with the repo mounted read-write at C:\workdir and a host output directory mounted at C:\output, copying resulting Habitat/gem/DLL artifacts to the output directory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com>
Chocolatey's 'ruby' package versions carry an extra packaging revision
suffix (e.g. 3.4.10.1) that drifts over time, so an exact '3.4.8' isn't
guaranteed to exist (and '3.1.7' never did). Resolve RUBY_VERSION as a
major.minor prefix to the newest matching Chocolatey package version at
image build time instead of hardcoding a full version string.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com>
Docker's Windows shell-form RUN concatenates the SHELL directive's prefix
with the RUN command text into a single string before handing it to
'powershell -Command', which mangled the nested double/single quotes in
the inline Chocolatey OData query one-liner (URL literal + regex + string
interpolation). Move that logic into Install-Ruby.ps1, COPY'd into the
image and invoked with 'powershell -File', which sidesteps the quoting
collision entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com>
'hab pkg build' needs a Habitat Builder token to install chef/hab-studio
and this project's Habitat build dependencies, failing with '401
Unauthorized' without one. Add a -HabAuthToken parameter (defaulting to
\_Qk9YLTEKYmxkci0yMDE3MDkyNzAyMzcxNApibGRyLTIwMTcwOTI3MDIzNzE0Cnh4T1Y3WlhrSW9jWUkxdis1SGx1UnVWcXQyQkRKSU0rCkJTQUVQbU1OR3daejNEamhsM2k4SzQxTG05emRIV3c2Y0pDRmFwb1FZR05yQlhHcg==) that's forwarded into the container, plus an early
warning when it's missing so the failure is obvious before the build
proceeds. Documented in the README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com>
Resolution order is now: -HabAuthToken parameter, then $env:HAB_AUTH_TOKEN,
then a HAB_AUTH_TOKEN=... entry in ~/.env (a common place devs stash
tokens for local tooling). Documented in the parameter help and README.

Signed-off-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com>
The .NET Framework / Windows PowerShell side of the gem
(Chef.PowerShell.Wrapper.dll) is a mixed-mode C++/CLI assembly built with
the VS2022 (v145) toolset, which needs the matching Visual C++
Redistributable at load time. The minimal windows/servercore:ltsc2022
base image doesn't ship it, so ffi_lib failed with Win32 error 126
('The specified module could not be found') even though the DLL itself
was present on disk.

Also surface bundle update failures immediately instead of silently
continuing into gem_check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com>
…187/#196/#205)

PRs #187, #196, and #205 fixed a class of bug where the buffer the native
wrapper used to hand a result back to Ruby could be collected/reused
before Ruby finished reading it, silently corrupting or truncating the
payload -- even for values as trivial as a bare boolean. It mostly showed
up under memory pressure (e.g. constrained CI agents).

Add chef-powershell/memory_pressure_test.rb, which round-trips a matrix
of payload shapes (booleans, integers, floats, strings including
multi-byte Unicode, arrays, hashtables, a large string) through both the
:powershell and :pwsh interpreters many times, forcing a full GC before
every single round-trip (or, opt-in via CHEF_POWERSHELL_GC_STRESS=1, the
much slower GC.stress=true for the whole run), and asserts *exact*
equality every time.

Add .expeditor/local_low_memory_test.ps1, which runs that same script
inside a memory-constrained (default 512MB) Windows Docker container
-- reusing the windows2022-core build image -- against whatever DLLs are
currently under chef-powershell/bin/ruby_bin_folder/<ARCH> (i.e. the
output of a local build_gems.ps1/local_build_gems.ps1 run), so the
.NET/CLR side is put under genuine OS-level memory pressure too, not
just the Ruby side. 256MB was observed to reliably OOM before a
PowerShell runspace could even start, so 512MB is the floor used here.

Document both in 'Testing chef-powershell with Chef-18 and Chef-19.md'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com>
@tpowell-progress
tpowell-progress force-pushed the tp/local-build-gems-windows2022 branch from d0f3da0 to 562786c Compare September 14, 2026 21:05
The unicode payload row had extra alignment spacing that cookstyle
flagged as Layout/ExtraSpacing, failing the gem_check style task.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com>
@tpowell-progress
tpowell-progress merged commit 221f139 into 18-Stable Sep 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants