Skip to content

Commit 157cf1a

Browse files
Fix Windows TRUST_E_BAD_DIGEST on portable-signed MSIX/AppX; sign plain-ZIP packages (#46)
## Problem `psign-tool --mode portable sign` produced MSIX/AppX packages that psign's own `trust-verify-msix` accepted, but that Windows rejected: - `signtool verify /pa` → `0x80096010` (`TRUST_E_BAD_DIGEST`) - `Get-AuthenticodeSignature` → `HashMismatch` The same unsigned package signed by `signtool.exe` with the same self-signed PFX only fails on the untrusted root. ## Root cause The staged package and the final package are both rebuilt with the `zip` 0.6 crate. Its layout differs from AppxPackaging (makeappx/signtool) in three ways, and Windows AppxSip rejects a package that differs in **any** of them, even when the portable AXCD digest is self-consistent: | Field | `zip` crate | AppxPackaging | |---|---|---| | End records | classic EOCD only | ZIP64 EOCD (v45) + locator + all-`0xFF` classic EOCD | | CD "version made by" | `0x032e` (Unix, 4.6) | `0x002d` (45) | | CD external attributes | `0x81a40000` (`0o100644`) | `0` | I tested this on Windows 11 / SDK 10.0.26100. Each row was toggled on its own, and the only combination Windows accepted was all three AppxPackaging values. I also tried `version made by` 20 / `0x0314` / 46 and external attrs `0x20` (`FILE_ATTRIBUTE_ARCHIVE`): Windows rejected all of them. Hashing synthesized AppxPackaging values over a non-conforming archive doesn't match Windows either, so Windows isn't normalizing these fields. It requires them. ## Changes - **`psign-sip-digest`**: new `msix_digest::normalize_zip_for_appx_signing` rewrites a ZIP into the AppxPackaging layout. It sets "version made by" to 45 and external attrs to 0 on every CD entry, and replaces the end records with ZIP64 EOCD + locator + sentinel classic EOCD. Local headers and data stay untouched, so entry offsets don't move. - **`psign-portable-core`**: the flat and bundle staging (`stage_flat_msix_for_signature`, `stage_msix_bundle_for_signature`) and `replace_msix_signature_part` pass their output through it. The digest is computed over the same layout that gets shipped. - **Portable verify** (`hash_cd_and_eocd`): rejects packages Windows rejects, meaning those with no ZIP64 EOCD or with non-AppxPackaging "version made by" / external attributes. Packages written by psign ≤ 0.8.1 now fail `trust-verify-msix` with an explicit message instead of passing. - **Block map**: parses and validates the SDK 26100+ `b4:FileHash` element (`http://schemas.microsoft.com/appx/2021/blockmap`), which holds the digest of the whole uncompressed file. Before this, `verify-msix` failed on modern signtool output with ``unexpected AppxBlockMap.xml element `FileHash` ``. - Docs: `docs/rust-sip-gaps.md`. ## Signing packages not built by makeappx (pinget `source2.msix`) Pinget is dropping makeappx, so its input to psign will be a plain ZIP of the package layout. With only the layout fix above, signing such packages still failed or lost compression: - `[Content_Types].xml` from generic tools broke signing (``content type for `AppxBlockMap.xml` is `application/vnd.ms-appx.manifest+xml` ``), and a ZIP with no `[Content_Types].xml` was refused. - psign stored every payload uncompressed, so a package makeappx would write at ~145 KB came out around 1.4 MB. Changes: - **`psign-portable-core::appx_zip`**: a small ZIP writer that emits the AppxPackaging layout directly. It replaces the `zip`-crate writer for staging and for the final `AppxSignature.p7x` embed, which now copies the other entries' compressed bytes verbatim. Payloads are deflated as makeappx does: a full flush after every 64 KiB block, so each block decodes independently, followed by an empty final block, and each block's compressed size goes into `Block/@Size`. PNG/JPEG/ZIP/package/media payloads, payloads that compression wouldn't shrink, and bundle children are stored. - **`msix_digest::appx_content_types_for_signing`**: merges or generates `[Content_Types].xml`. It keeps the input's types, adds makeappx-style `Default`s for payload extensions, forces the (bundle) manifest type, and adds the block map and signature `Override`s. - Docs: `docs/linux-signing-pipelines.md` (MSIX without makeappx) and `docs/rust-sip-gaps.md`. Checked on Windows 11 / SDK 26100 with a pinget-shaped package (neutral `windows.appExtension` manifest, a 1.3 MB `Public/index.db`, PNG assets), signed on **Windows and on Linux (WSL Ubuntu 24.04)**, with and without RFC 3161 timestamping (DigiCert TSA): | Input | Signed size | WinVerifyTrust | Get-AuthenticodeSignature | makeappx unpack | trust-verify-msix | |---|---|---|---|---|---| | generic deflated ZIP + `[Content_Types].xml` | 148 KB | 0x800B0109 | UnknownError | ok | ok | | plain ZIP, no `[Content_Types].xml` | 148 KB | 0x800B0109 | UnknownError | ok | ok | | makeappx-built pinget layout (145 KB unsigned) | 148 KB | 0x800B0109 | UnknownError | ok | ok | | makeappx-built minimal package with a PE | 42 KB | 0x800B0109 | UnknownError | ok | ok | 0x800B0109 means only the self-signed root is untrusted. Timestamped signatures show `The signature is timestamped` in `signtool verify /pa /v`. ## Tests - `tests/msix_native_makeappx.rs::portable_signed_makeappx_msix_fails_windows_verify_only_on_untrusted_root` (Windows CI, existing "Portable MSIX/AppX native makeappx.exe validation" step): - builds an MSIX with `makeappx pack`; - creates a throwaway self-signed code-signing PFX in memory with `CertificateRequest` (never imported into any store); - portable-signs it with SHA256 and SHA384; - asserts that WinVerifyTrust (`psign-tool verify /pa`) reports `0x800B0109`, never `0x80096010`, that `Get-AuthenticodeSignature` reports `UnknownError` (not `HashMismatch`), and that `trust-verify-msix` passes. - `psign-portable-core::staged_and_signed_msix_use_appx_zip64_layout`: staged and signed `.msix` / `.appx` / `.msixbundle` outputs have ZIP64 end records and normalized CD entries, checked by an independent byte parser. - The Windows test also signs a plain ZIP of the same layout (no block map, no content types) and requires `makeappx unpack` to accept every signed output. - `psign-portable-core::signs_plain_zip_package_without_makeappx_metadata`: a plain deflated ZIP with no `[Content_Types].xml` signs into a compressed package with per-block `Size` attributes and generated content types. - `appx_zip` unit tests: each deflated block decodes independently, incompressible/precompressed data is stored, and output round-trips through the `zip` reader. `msix_digest::content_types_for_signing_fill_gaps_and_keep_existing_types` covers the content-types builder. - `psign-sip-digest`: unit tests for the normalizer (layout, idempotence, unchanged offsets), for the verify-side rejections, and for `FileHash` parse/validate (match, mismatch, wrong length, duplicate). ## Verification (Windows 11, SDK 10.0.26100) makeappx-built MSIX, portable-signed by this branch with a throwaway self-signed PFX: ```text signtool verify /pa -> SignTool Error: A certificate chain processed, but terminated in a root certificate which is not trusted by the trust provider. psign-tool verify /pa (WinVerifyTrust) -> 0x800B0109 (was 0x80096010 before this change) Get-AuthenticodeSignature -> UnknownError (was HashMismatch) psign-tool portable trust-verify-msix -> ok ``` The portable-signed `.msixbundle` (with a portable-signed child) also fails only on the untrusted root. All committed signed MSIX/AppX fixtures still pass `verify-msix`. `cargo fmt --all`, `cargo clippy --workspace --all-targets --locked`: clean. `cargo test --workspace --locked --no-fail-fast`: all MSIX tests pass. Locally, 5 `corpus_sign_verify` tests fail with `0x800B0109` on PE/ZIP-authenticode fixtures because this machine does not trust the Devolutions test root (CI bootstraps it via `bootstrap-devolutions-authenticode.ps1`). Those failures are unrelated to this change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent ea9b61f commit 157cf1a

8 files changed

Lines changed: 1310 additions & 115 deletions

File tree

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎crates/psign-portable-core/Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,4 +29,5 @@ serde_json = "1"
2929
sha1 = { version = "0.10", default-features = false }
3030
sha2 = "0.10"
3131
x509-cert = { version = "0.2.5", default-features = false, features = ["pem"] }
32+
flate2 = { version = "1.1", default-features = false, features = ["rust_backend"] }
3233
zip = { version = "0.6.6", default-features = false, features = ["deflate"] }
Lines changed: 326 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,326 @@
1+
//! Minimal ZIP writer for MSIX / AppX packages staged for portable signing.
2+
//!
3+
//! Windows AppxSip only accepts packages in the AppxPackaging (makeappx / signtool) ZIP
4+
//! layout, and AppxPackaging expects deflated payloads to be split into independently
5+
//! decodable 64 KiB blocks whose compressed sizes are listed in `AppxBlockMap.xml`.
6+
//! Generic ZIP writers produce neither, so packages are written here: no data
7+
//! descriptors or extra fields, "version made by" 45, zero external attributes, and
8+
//! ZIP64 end records (via [`msix_digest::normalize_zip_for_appx_signing`]).
9+
10+
use anyhow::{Result, bail};
11+
use flate2::{Compress, Compression, FlushCompress, Status};
12+
use psign_sip_digest::msix_digest;
13+
14+
/// AppxBlockMap block size (uncompressed bytes per `Block`).
15+
pub(crate) const APPX_BLOCK_SIZE: usize = 64 * 1024;
16+
17+
const METHOD_STORED: u16 = 0;
18+
const METHOD_DEFLATED: u16 = 8;
19+
const LOCAL_HEADER_SIG: u32 = 0x0403_4b50;
20+
const CENTRAL_HEADER_SIG: u32 = 0x0201_4b50;
21+
const EOCD_SIG: u32 = 0x0605_4b50;
22+
/// MS-DOS date for 1980-01-01 (time 00:00:00), matching the `zip` crate default.
23+
const DOS_DATE_1980: u16 = 0x21;
24+
25+
/// Extensions makeappx stores without compression because they are already compressed.
26+
const STORED_EXTENSIONS: &[&str] = &[
27+
"png",
28+
"jpg",
29+
"jpeg",
30+
"gif",
31+
"zip",
32+
"gz",
33+
"7z",
34+
"cab",
35+
"appx",
36+
"msix",
37+
"appxbundle",
38+
"msixbundle",
39+
"eappx",
40+
"emsix",
41+
"mp3",
42+
"mp4",
43+
"wma",
44+
"wmv",
45+
"m4a",
46+
];
47+
48+
pub(crate) struct AppxZipEntry {
49+
pub(crate) name: String,
50+
method: u16,
51+
crc32: u32,
52+
uncompressed_size: u64,
53+
data: Vec<u8>,
54+
/// Compressed size of each 64 KiB block of a deflated entry (`Block/@Size`).
55+
pub(crate) block_sizes: Option<Vec<u64>>,
56+
}
57+
58+
impl AppxZipEntry {
59+
/// An uncompressed entry.
60+
pub(crate) fn stored(name: impl Into<String>, data: Vec<u8>) -> Self {
61+
Self {
62+
name: name.into(),
63+
method: METHOD_STORED,
64+
crc32: crc32(&data),
65+
uncompressed_size: data.len() as u64,
66+
data,
67+
block_sizes: None,
68+
}
69+
}
70+
71+
/// A payload entry, deflated in AppX blocks unless that does not make it smaller.
72+
pub(crate) fn payload(name: impl Into<String>, data: &[u8]) -> Result<Self> {
73+
let name = name.into();
74+
let extension = name
75+
.rsplit('/')
76+
.next()
77+
.and_then(|leaf| leaf.rsplit_once('.'))
78+
.map(|(_, ext)| ext.to_ascii_lowercase())
79+
.unwrap_or_default();
80+
if data.is_empty() || STORED_EXTENSIONS.contains(&extension.as_str()) {
81+
return Ok(Self::stored(name, data.to_vec()));
82+
}
83+
let (compressed, block_sizes) = deflate_appx_blocks(data)?;
84+
if compressed.len() >= data.len() {
85+
return Ok(Self::stored(name, data.to_vec()));
86+
}
87+
Ok(Self {
88+
name,
89+
method: METHOD_DEFLATED,
90+
crc32: crc32(data),
91+
uncompressed_size: data.len() as u64,
92+
data: compressed,
93+
block_sizes: Some(block_sizes),
94+
})
95+
}
96+
97+
/// An entry copied byte-for-byte (already compressed with `method`) from a staged package.
98+
pub(crate) fn raw(
99+
name: impl Into<String>,
100+
method: u16,
101+
crc32: u32,
102+
uncompressed_size: u64,
103+
data: Vec<u8>,
104+
) -> Result<Self> {
105+
if !matches!(method, METHOD_STORED | METHOD_DEFLATED) {
106+
bail!("unsupported ZIP compression method {method} in staged MSIX package");
107+
}
108+
Ok(Self {
109+
name: name.into(),
110+
method,
111+
crc32,
112+
uncompressed_size,
113+
data,
114+
block_sizes: None,
115+
})
116+
}
117+
118+
/// Local file header size, as listed in `File/@LfhSize`.
119+
pub(crate) fn local_header_size(&self) -> usize {
120+
30 + self.name.len()
121+
}
122+
}
123+
124+
fn crc32(data: &[u8]) -> u32 {
125+
let mut crc = flate2::Crc::new();
126+
crc.update(data);
127+
crc.sum()
128+
}
129+
130+
/// Raw-deflate `data` as AppxPackaging does: every 64 KiB block ends with a full flush, so
131+
/// each block's compressed bytes decode on their own, followed by an empty final block.
132+
/// Returns the compressed stream and each block's compressed size.
133+
pub(crate) fn deflate_appx_blocks(data: &[u8]) -> Result<(Vec<u8>, Vec<u64>)> {
134+
let mut compress = Compress::new(Compression::default(), false);
135+
let mut out = Vec::with_capacity(data.len() / 2 + 64);
136+
let mut block_sizes = Vec::with_capacity(data.len().div_ceil(APPX_BLOCK_SIZE));
137+
for block in data.chunks(APPX_BLOCK_SIZE) {
138+
let start = out.len();
139+
deflate_into(&mut compress, block, FlushCompress::Full, &mut out)?;
140+
block_sizes.push((out.len() - start) as u64);
141+
}
142+
deflate_into(&mut compress, &[], FlushCompress::Finish, &mut out)?;
143+
Ok((out, block_sizes))
144+
}
145+
146+
fn deflate_into(
147+
compress: &mut Compress,
148+
mut input: &[u8],
149+
flush: FlushCompress,
150+
out: &mut Vec<u8>,
151+
) -> Result<()> {
152+
loop {
153+
out.reserve(input.len() + 1024);
154+
let before = compress.total_in();
155+
let status = compress.compress_vec(input, out, flush)?;
156+
let consumed = usize::try_from(compress.total_in() - before)?;
157+
input = &input[consumed..];
158+
// A flush is complete once the encoder stops filling the spare output capacity.
159+
if status == Status::StreamEnd
160+
|| (!matches!(flush, FlushCompress::Finish)
161+
&& input.is_empty()
162+
&& out.len() < out.capacity())
163+
{
164+
return Ok(());
165+
}
166+
}
167+
}
168+
169+
fn u32_field(value: u64, what: &str) -> Result<u32> {
170+
u32::try_from(value)
171+
.ok()
172+
.filter(|&v| v != u32::MAX)
173+
.ok_or_else(|| {
174+
anyhow::anyhow!(
175+
"{what} exceeds 4 GiB; portable MSIX signing does not write ZIP64 entries"
176+
)
177+
})
178+
}
179+
180+
/// Serialize `entries` in order into an AppxPackaging-layout ZIP.
181+
pub(crate) fn write_appx_zip(entries: &[AppxZipEntry]) -> Result<Vec<u8>> {
182+
let mut out = Vec::new();
183+
let mut central = Vec::new();
184+
for entry in entries {
185+
let name = entry.name.as_bytes();
186+
let name_len = u16::try_from(name.len())?;
187+
// Bit 11: entry name is UTF-8.
188+
let flags: u16 = if entry.name.is_ascii() { 0 } else { 1 << 11 };
189+
let offset = u32_field(out.len() as u64, "MSIX package size")?;
190+
let compressed_size = u32_field(entry.data.len() as u64, "MSIX entry size")?;
191+
let uncompressed_size = u32_field(entry.uncompressed_size, "MSIX entry size")?;
192+
193+
out.extend_from_slice(&LOCAL_HEADER_SIG.to_le_bytes());
194+
out.extend_from_slice(&20u16.to_le_bytes());
195+
out.extend_from_slice(&flags.to_le_bytes());
196+
out.extend_from_slice(&entry.method.to_le_bytes());
197+
out.extend_from_slice(&0u16.to_le_bytes());
198+
out.extend_from_slice(&DOS_DATE_1980.to_le_bytes());
199+
out.extend_from_slice(&entry.crc32.to_le_bytes());
200+
out.extend_from_slice(&compressed_size.to_le_bytes());
201+
out.extend_from_slice(&uncompressed_size.to_le_bytes());
202+
out.extend_from_slice(&name_len.to_le_bytes());
203+
out.extend_from_slice(&0u16.to_le_bytes());
204+
out.extend_from_slice(name);
205+
out.extend_from_slice(&entry.data);
206+
207+
central.extend_from_slice(&CENTRAL_HEADER_SIG.to_le_bytes());
208+
central.extend_from_slice(&45u16.to_le_bytes());
209+
central.extend_from_slice(&20u16.to_le_bytes());
210+
central.extend_from_slice(&flags.to_le_bytes());
211+
central.extend_from_slice(&entry.method.to_le_bytes());
212+
central.extend_from_slice(&0u16.to_le_bytes());
213+
central.extend_from_slice(&DOS_DATE_1980.to_le_bytes());
214+
central.extend_from_slice(&entry.crc32.to_le_bytes());
215+
central.extend_from_slice(&compressed_size.to_le_bytes());
216+
central.extend_from_slice(&uncompressed_size.to_le_bytes());
217+
central.extend_from_slice(&name_len.to_le_bytes());
218+
// Extra length, comment length, disk number start, internal attributes.
219+
central.extend_from_slice(&[0; 8]);
220+
central.extend_from_slice(&0u32.to_le_bytes());
221+
central.extend_from_slice(&offset.to_le_bytes());
222+
central.extend_from_slice(name);
223+
}
224+
let entry_count = u16::try_from(entries.len())
225+
.ok()
226+
.filter(|&n| n != u16::MAX)
227+
.ok_or_else(|| anyhow::anyhow!("too many MSIX package entries"))?;
228+
let cd_offset = u32_field(out.len() as u64, "MSIX package size")?;
229+
let cd_size = u32_field(central.len() as u64, "MSIX central directory size")?;
230+
out.extend_from_slice(&central);
231+
out.extend_from_slice(&EOCD_SIG.to_le_bytes());
232+
out.extend_from_slice(&[0; 4]);
233+
out.extend_from_slice(&entry_count.to_le_bytes());
234+
out.extend_from_slice(&entry_count.to_le_bytes());
235+
out.extend_from_slice(&cd_size.to_le_bytes());
236+
out.extend_from_slice(&cd_offset.to_le_bytes());
237+
out.extend_from_slice(&0u16.to_le_bytes());
238+
msix_digest::normalize_zip_for_appx_signing(out)
239+
}
240+
241+
#[cfg(test)]
242+
mod tests {
243+
use super::*;
244+
use flate2::{Decompress, FlushDecompress};
245+
246+
fn inflate_raw(data: &[u8], expected_len: usize) -> Vec<u8> {
247+
let mut out = Vec::with_capacity(expected_len + 16);
248+
Decompress::new(false)
249+
.decompress_vec(data, &mut out, FlushDecompress::Sync)
250+
.expect("inflate");
251+
out
252+
}
253+
254+
#[test]
255+
fn deflated_blocks_decode_independently() {
256+
let data: Vec<u8> = (0..200_000u32)
257+
.flat_map(|i| format!("row-{}|", i % 5000).into_bytes())
258+
.collect();
259+
let (compressed, sizes) = deflate_appx_blocks(&data).unwrap();
260+
assert_eq!(sizes.len(), data.len().div_ceil(APPX_BLOCK_SIZE));
261+
assert!(compressed.len() < data.len() / 2);
262+
263+
let mut offset = 0usize;
264+
for (block, size) in data.chunks(APPX_BLOCK_SIZE).zip(&sizes) {
265+
let size = usize::try_from(*size).unwrap();
266+
let decoded = inflate_raw(&compressed[offset..offset + size], block.len());
267+
assert_eq!(
268+
decoded, block,
269+
"each block must decode without prior blocks"
270+
);
271+
offset += size;
272+
}
273+
// Only the empty final block follows the last flushed block.
274+
assert!(compressed.len() - offset <= 8);
275+
276+
let mut whole = Vec::with_capacity(data.len() + 16);
277+
Decompress::new(false)
278+
.decompress_vec(&compressed, &mut whole, FlushDecompress::Finish)
279+
.unwrap();
280+
assert_eq!(whole, data);
281+
}
282+
283+
#[test]
284+
fn payload_stores_precompressed_and_incompressible_data() {
285+
assert!(
286+
AppxZipEntry::payload("Assets/logo.png", &[7; 4096])
287+
.unwrap()
288+
.block_sizes
289+
.is_none()
290+
);
291+
let mut state = 0x9e37_79b9_7f4a_7c15u64;
292+
let noise: Vec<u8> = (0..70_000)
293+
.map(|_| {
294+
state ^= state << 13;
295+
state ^= state >> 7;
296+
state ^= state << 17;
297+
(state >> 32) as u8
298+
})
299+
.collect();
300+
let entry = AppxZipEntry::payload("noise.bin", &noise).unwrap();
301+
assert_eq!(entry.method, METHOD_STORED);
302+
let text = AppxZipEntry::payload("index.db", &[b'a'; 70_000]).unwrap();
303+
assert_eq!(text.method, METHOD_DEFLATED);
304+
assert_eq!(text.block_sizes.as_ref().map(Vec::len), Some(2));
305+
}
306+
307+
#[test]
308+
fn written_zip_round_trips_through_zip_reader() {
309+
let entries = vec![
310+
AppxZipEntry::payload("AppxManifest.xml", &[b'x'; 100_000]).unwrap(),
311+
AppxZipEntry::stored("Public/empty.db", Vec::new()),
312+
AppxZipEntry::stored("AppxSignature.p7x", b"PKCX".to_vec()),
313+
];
314+
let zip = write_appx_zip(&entries).unwrap();
315+
let mut archive = zip::ZipArchive::new(std::io::Cursor::new(&zip)).unwrap();
316+
assert_eq!(archive.len(), 3);
317+
let mut manifest = Vec::new();
318+
std::io::Read::read_to_end(&mut archive.by_index(0).unwrap(), &mut manifest).unwrap();
319+
assert_eq!(manifest, vec![b'x'; 100_000]);
320+
let first = archive.by_index(0).unwrap();
321+
assert_eq!(
322+
first.data_start() - first.header_start(),
323+
entries[0].local_header_size() as u64
324+
);
325+
}
326+
}

0 commit comments

Comments
 (0)