Skip to content

Commit 0757005

Browse files
Add native WinGet preindexed repository publishers in Rust and C# (#59)
## Summary - Add reusable Rust and .NET publishing libraries and matching `pinget repository build|diff|status|verify|publish|doctor` commands, independent of ordinary client/source initialization. - Generate native WinGet v2 SQLite indexes, merged manifests and MSZIP version-data sidecars from standard Windows manifests. Publish immutable metadata first and replace `source2.msix` last. - Provide Windows SDK MSIX packaging/PFX signing, explicit unsigned development mode, identity/version checks, destination locks, ownership inventories and rollback/retry protection. - Include review hardening for locale merging, schema string preservation, byte-level parity, correlation data, state/path verification, tool discovery and 64 MiB bounds. Artifact-format-v2 state rejects legacy artifacts without mutation. - Integrate packaging/release/version synchronization and static-hosting documentation. All publisher fixture, validation and comparison scripts are now **PowerShell**, not Python; no Python/PyYAML setup or dependency remains. - Preserve independent native SQLite/MSZIP inspection and exact manifest/sidecar comparison through PowerShell helpers using existing .NET build dependencies, rather than relying only on production verification. - Maintain meaningful non-Windows validation while gating native Windows generation tests. Both macOS packaging matrices use supported `macos-15` runners with unchanged architecture targets. ## Validation — all checks green - All **14 PR checks passed on final head `363e187`**: [CI](https://github.com/Devolutions/pinget/actions/runs/41314083678) and [CLI NuGet Packages](https://github.com/Devolutions/pinget/actions/runs/41314083606). Includes Rust lint/build/tests, .NET/PowerShell tests, all six Rust CLI targets, both Windows NativeAOT targets, pack/smoke and signed stock-client acceptance. - Hardened signed outputs from both publishers passed stock WinGet on a disposable Windows runner: source add/update, search/show, multi-block version data, SHA-256-verified harmless download, version bumps, additions/removals and cached metadata retention. Both Pinget readers consume each feed. [Latest signed gate](https://github.com/Devolutions/pinget/actions/runs/41314083678/job/124873958727). The documented tested hosted-client baseline is WinGet 1.11.510 / App Installer 1.26.510.0. - Runner-local, short-lived signing/TLS credentials and temporary source registrations are cleaned up and verified. No Sandbox, repository secrets or developer-host trust/source mutations. - Rust `initial_publish_post_copy_failure_cleans_destination_and_allows_retry` proves actual copied hashes before injected failure, complete cleanup, unchanged publication state, preserved snapshot and a successful normal retry. The public API and verification ordering remain unchanged. - .NET `Publish_InitialPostCopyMsixVerificationFailure_CleansArtifactsAndAllowsRetry` uses real MakeAppx behind a test-only identity-mutating wrapper to prove post-copy failure cleanup and successful retry. - PowerShell replacements passed local fixture schema/MSZIP/MSIX checks, independent validation of both snapshots, exact cross-language comparison, output-format checks and a negative corrupted-index-digest check. Managed production driver exits 0; NativeAOT snapshots validate and compare successfully. CI reran the new PowerShell production driver successfully. - Rust publisher suites: 40 Windows tests and 24 Ubuntu WSL tests; Ubuntu CI is green. .NET Core baseline: 274 passed plus 1 existing live-source skip; publisher baseline 57 plus focused follow-up. Formatting, Clippy, rustdoc, AOT and script/workflow syntax pass. ## Scope limits - Windows WinGet-compatible v2 publishing only. No REST server, cloud uploader, DSC/MCP integration, macOS execution, policy agent or new PowerShell cmdlets. - Compatibility covers tested clients/fixtures/toolchains, not every client or manifest field; supported manifest versions 1.0.0–1.10.0 receive structural/required-field checks, not exhaustive upstream JSON-schema validation. - The pre-existing SQLitePCLRaw.lib.e_sqlite3 2.1.11 NU1903 advisory is unchanged. - Release the matching updated Rust core before the publisher, as documented. --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 3aa2456 commit 0757005

58 files changed

Lines changed: 14330 additions & 30 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,12 @@ jobs:
7070
- name: Test pinget-cli
7171
run: cargo test -p pinget-cli --manifest-path rust/Cargo.toml
7272

73+
- name: Test pinget-publisher
74+
run: cargo test -p pinget-publisher --manifest-path rust/Cargo.toml
75+
76+
- name: Check publisher documentation
77+
run: cargo doc -p pinget-publisher --manifest-path rust/Cargo.toml --no-deps
78+
7379
- name: Build pinget-cli
7480
run: cargo build -p pinget-cli --manifest-path rust/Cargo.toml
7581

@@ -107,6 +113,27 @@ jobs:
107113
- name: Run core tests
108114
run: dotnet test dotnet/src/Devolutions.Pinget.Core.Tests/Devolutions.Pinget.Core.Tests.csproj -c Release --no-build
109115

116+
- name: Run publisher tests
117+
run: dotnet test dotnet/src/Devolutions.Pinget.Publisher.Tests/Devolutions.Pinget.Publisher.Tests.csproj -c Release
118+
119+
- name: Run Rust and .NET publisher interoperability
120+
shell: pwsh
121+
run: |
122+
cargo build --manifest-path rust/Cargo.toml -p pinget-cli --release
123+
if ($LASTEXITCODE -ne 0) { throw 'Rust publisher CLI build failed.' }
124+
$sdkRoot = Join-Path ${env:ProgramFiles(x86)} 'Windows Kits\10\bin'
125+
$makeAppx = Get-ChildItem -LiteralPath $sdkRoot -Directory |
126+
Sort-Object Name -Descending |
127+
ForEach-Object { Join-Path $_.FullName 'x64\makeappx.exe' } |
128+
Where-Object { Test-Path -LiteralPath $_ } |
129+
Select-Object -First 1
130+
if (-not $makeAppx) { throw 'Windows SDK MakeAppx was not found.' }
131+
& .\tests\publisher\run-production-conformance.ps1 `
132+
-RustCli .\rust\target\release\pinget.exe `
133+
-DotnetAssembly .\dotnet\src\Devolutions.Pinget.Cli\bin\Release\net10.0\pinget.dll `
134+
-MakeAppxPath $makeAppx
135+
if ($LASTEXITCODE -ne 0) { throw 'Publisher interoperability checks failed.' }
136+
110137
- name: Build native COM DLLs
111138
shell: pwsh
112139
env:
@@ -124,8 +151,77 @@ jobs:
124151
run: |
125152
New-Item -Path artifacts/ci-nuget -ItemType Directory -Force | Out-Null
126153
dotnet pack dotnet/src/Devolutions.Pinget.Core/Devolutions.Pinget.Core.csproj -c Release --no-build -o artifacts/ci-nuget
154+
dotnet pack dotnet/src/Devolutions.Pinget.Publisher/Devolutions.Pinget.Publisher.csproj -c Release --no-build -o artifacts/ci-nuget
127155
dotnet pack dotnet/src/Devolutions.Pinget.PowerShell.Engine/Devolutions.Pinget.PowerShell.Engine.csproj -c Release --no-build -o artifacts/ci-nuget
128156
129157
- name: Validate PowerShell module packaging
130158
shell: pwsh
131159
run: pwsh -NoLogo -NoProfile -File (Resolve-Path 'scripts/Build-PowerShellModule.ps1') -NoBuild -OutputRoot artifacts/ci-powershell -Clean
160+
161+
stock-winget-acceptance:
162+
name: Signed stock WinGet acceptance (ephemeral runner)
163+
runs-on: windows-latest
164+
needs:
165+
- dotnet
166+
timeout-minutes: 45
167+
168+
steps:
169+
- name: Checkout
170+
uses: actions/checkout@v6
171+
172+
- name: Setup .NET
173+
uses: actions/setup-dotnet@v5
174+
with:
175+
dotnet-version: 10.0.x
176+
177+
- name: Setup Rust
178+
shell: pwsh
179+
run: |
180+
rustup toolchain install stable --profile minimal --target x86_64-pc-windows-msvc
181+
rustup default stable
182+
183+
- name: Build disposable x64 publisher CLIs
184+
shell: pwsh
185+
run: |
186+
pwsh -NoLogo -NoProfile -File (Resolve-Path 'scripts/Build-CliNativeNuGetPackages.ps1') `
187+
-Package All `
188+
-RustRuntimeIdentifiers 'win-x64' `
189+
-DotNetRuntimeIdentifiers 'win-x64' `
190+
-NoPack `
191+
-Clean `
192+
-StagingRoot artifacts/ci-stock-winget `
193+
-OutputRoot artifacts/ci-stock-winget-packages
194+
if ($LASTEXITCODE -ne 0) { throw 'Building the disposable x64 publisher CLIs failed.' }
195+
196+
- name: Run signed stock WinGet acceptance with runner-local credentials
197+
shell: pwsh
198+
run: |
199+
$sdkRoot = Join-Path ${env:ProgramFiles(x86)} 'Windows Kits\10\bin'
200+
$sdkTools = Get-ChildItem -LiteralPath $sdkRoot -Directory |
201+
Sort-Object Name -Descending |
202+
ForEach-Object {
203+
[pscustomobject]@{
204+
MakeAppx = Join-Path $_.FullName 'x64\makeappx.exe'
205+
SignTool = Join-Path $_.FullName 'x64\signtool.exe'
206+
}
207+
} |
208+
Where-Object {
209+
(Test-Path -LiteralPath $_.MakeAppx) -and
210+
(Test-Path -LiteralPath $_.SignTool)
211+
} |
212+
Select-Object -First 1
213+
if (-not $sdkTools) { throw 'Windows SDK MakeAppx and SignTool were not found.' }
214+
215+
pwsh -NoLogo -NoProfile -File (Resolve-Path 'tests/publisher/run-ephemeral-host-publisher-gate.ps1') `
216+
-RustCliPath (Resolve-Path 'artifacts/ci-stock-winget/rust/win-x64/pinget.exe') `
217+
-DotnetCliPath (Resolve-Path 'artifacts/ci-stock-winget/dotnet-nativeaot/win-x64/pinget.exe') `
218+
-MakeAppxPath $sdkTools.MakeAppx `
219+
-SignToolPath $sdkTools.SignTool
220+
if ($LASTEXITCODE -ne 0) { throw 'Signed stock WinGet acceptance failed.' }
221+
222+
- name: Remove disposable build outputs
223+
if: always()
224+
shell: pwsh
225+
run: |
226+
Remove-Item -LiteralPath 'artifacts/ci-stock-winget' -Recurse -Force -ErrorAction SilentlyContinue
227+
Remove-Item -LiteralPath 'artifacts/ci-stock-winget-packages' -Recurse -Force -ErrorAction SilentlyContinue

‎.github/workflows/cli-packages.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,10 @@ jobs:
4141
os: windows-latest
4242
rust-target: aarch64-pc-windows-msvc
4343
- rid: osx-x64
44-
os: macos-14
44+
os: macos-15
4545
rust-target: x86_64-apple-darwin
4646
- rid: osx-arm64
47-
os: macos-14
47+
os: macos-15
4848
rust-target: aarch64-apple-darwin
4949

5050
steps:

‎.github/workflows/release.yml‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -120,18 +120,20 @@ jobs:
120120
if ([string]::IsNullOrWhiteSpace($Version)) {
121121
$coreVersion = Select-String -Path 'rust/crates/pinget-core/Cargo.toml' -Pattern '^version = "([^"]+)"$' | Select-Object -First 1
122122
$cliVersion = Select-String -Path 'rust/crates/pinget-cli/Cargo.toml' -Pattern '^version = "([^"]+)"$' | Select-Object -First 1
123+
$publisherVersion = Select-String -Path 'rust/crates/pinget-publisher/Cargo.toml' -Pattern '^version = "([^"]+)"$' | Select-Object -First 1
123124
$dotnetVersion = Select-String -Path 'dotnet/src/Devolutions.Pinget.Cli/Program.cs' -Pattern '^const string Version = "([^"]+)";$' | Select-Object -First 1
124125
125-
if (($null -eq $coreVersion) -or ($null -eq $cliVersion) -or ($null -eq $dotnetVersion)) {
126+
if (($null -eq $coreVersion) -or ($null -eq $cliVersion) -or ($null -eq $publisherVersion) -or ($null -eq $dotnetVersion)) {
126127
throw 'Unable to detect version from source files'
127128
}
128129
129130
$coreVersionValue = $coreVersion.Matches[0].Groups[1].Value
130131
$cliVersionValue = $cliVersion.Matches[0].Groups[1].Value
132+
$publisherVersionValue = $publisherVersion.Matches[0].Groups[1].Value
131133
$dotnetVersionValue = $dotnetVersion.Matches[0].Groups[1].Value
132134
133-
if (($coreVersionValue -ne $cliVersionValue) -or ($coreVersionValue -ne $dotnetVersionValue)) {
134-
throw "Version mismatch detected: pinget-core=$coreVersionValue pinget-cli=$cliVersionValue dotnet=$dotnetVersionValue"
135+
if (($coreVersionValue -ne $cliVersionValue) -or ($coreVersionValue -ne $publisherVersionValue) -or ($coreVersionValue -ne $dotnetVersionValue)) {
136+
throw "Version mismatch detected: pinget-core=$coreVersionValue pinget-cli=$cliVersionValue pinget-publisher=$publisherVersionValue dotnet=$dotnetVersionValue"
135137
}
136138
137139
$Version = $coreVersionValue
@@ -188,12 +190,12 @@ jobs:
188190
archive_name: pinget-windows-arm64.zip
189191
binary_name: pinget.exe
190192
- name: macos-x64
191-
os: macos-14
193+
os: macos-15
192194
target: x86_64-apple-darwin
193195
archive_name: pinget-macos-x64.zip
194196
binary_name: pinget
195197
- name: macos-arm64
196-
os: macos-14
198+
os: macos-15
197199
target: aarch64-apple-darwin
198200
archive_name: pinget-macos-arm64.zip
199201
binary_name: pinget
@@ -374,6 +376,9 @@ jobs:
374376
- name: Run core tests
375377
run: dotnet test dotnet/src/Devolutions.Pinget.Core.Tests/Devolutions.Pinget.Core.Tests.csproj -c Release --no-build
376378

379+
- name: Run publisher tests
380+
run: dotnet test dotnet/src/Devolutions.Pinget.Publisher.Tests/Devolutions.Pinget.Publisher.Tests.csproj -c Release
381+
377382
- name: Build native COM DLLs
378383
shell: pwsh
379384
env:
@@ -443,6 +448,7 @@ jobs:
443448
444449
$BinaryRoots = @(
445450
'dotnet/src/Devolutions.Pinget.Core/bin/Release',
451+
'dotnet/src/Devolutions.Pinget.Publisher/bin/Release',
446452
'dotnet/src/Devolutions.Pinget.PowerShell.Engine/bin/Release',
447453
'dotnet/src/Devolutions.Pinget.PowerShell.Cmdlets/bin/Release',
448454
'rust/target/x86_64-pc-windows-msvc/release',
@@ -477,6 +483,7 @@ jobs:
477483
New-Item -Path dist/nuget -ItemType Directory -Force | Out-Null
478484
$packArgs = @('-c', 'Release', '--no-build', '-p:ContinuousIntegrationBuild=true', "-p:Version=$env:PACKAGE_VERSION", '-p:IncludeSymbols=true', '-p:SymbolPackageFormat=snupkg', '-o', 'dist/nuget')
479485
dotnet pack dotnet/src/Devolutions.Pinget.Core/Devolutions.Pinget.Core.csproj @packArgs
486+
dotnet pack dotnet/src/Devolutions.Pinget.Publisher/Devolutions.Pinget.Publisher.csproj @packArgs
480487
dotnet pack dotnet/src/Devolutions.Pinget.PowerShell.Engine/Devolutions.Pinget.PowerShell.Engine.csproj @packArgs
481488
482489
- name: Package PowerShell module
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: Repository source build
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
manifests:
7+
description: Repository-relative directory containing WinGet manifests
8+
required: true
9+
type: string
10+
11+
permissions:
12+
contents: read
13+
14+
jobs:
15+
build:
16+
name: Build an unsigned source snapshot
17+
runs-on: windows-latest
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@v6
21+
22+
- name: Setup Rust
23+
shell: pwsh
24+
run: |
25+
rustup toolchain install stable --profile minimal
26+
rustup default stable
27+
28+
- name: Build publisher CLI
29+
run: cargo build --manifest-path rust/Cargo.toml --package pinget-cli --release
30+
31+
- name: Build and verify snapshot
32+
shell: pwsh
33+
env:
34+
MANIFEST_INPUT: ${{ inputs.manifests }}
35+
run: |
36+
$ErrorActionPreference = 'Stop'
37+
$inputRoot = [System.IO.Path]::GetFullPath((Join-Path $env:GITHUB_WORKSPACE $env:MANIFEST_INPUT))
38+
$checkoutPrefix = [System.IO.Path]::TrimEndingDirectorySeparator($env:GITHUB_WORKSPACE) + [System.IO.Path]::DirectorySeparatorChar
39+
if (-not $inputRoot.StartsWith($checkoutPrefix, [System.StringComparison]::OrdinalIgnoreCase)) {
40+
throw 'Manifest directory must be inside the checkout.'
41+
}
42+
$stateRoot = Join-Path $env:RUNNER_TEMP 'publisher-state'
43+
& .\rust\target\release\pinget.exe repository build --input $inputRoot --state $stateRoot
44+
if ($LASTEXITCODE -ne 0) { throw 'Source build failed.' }
45+
& .\rust\target\release\pinget.exe repository verify --state $stateRoot
46+
if ($LASTEXITCODE -ne 0) { throw 'Snapshot verification failed.' }
47+
48+
- name: Upload source snapshot
49+
uses: actions/upload-artifact@v7
50+
with:
51+
name: unsigned-source-snapshot
52+
path: ${{ runner.temp }}/publisher-state/builds
53+
if-no-files-found: error

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@ Structured manifest output is also supported:
2727
- `show --output json|yaml`
2828
- `search --manifests --output json|yaml`
2929

30+
For authoring a static Windows package source, see
31+
[Publishing a static WinGet source](docs/repository-publishing.md). The
32+
`repository` commands are separate from client-side `source` management.
33+
3034
The C# implementation also ships the **`Devolutions.Pinget.Client` PowerShell 7 module** that mirrors the upstream `Microsoft.WinGet.Client` cmdlet family with renamed `Pinget` nouns, backed by `Devolutions.Pinget.Core`.
3135

3236
## Pinget CLI

0 commit comments

Comments
 (0)