Conformance vectors that carry real vendor evidence

[imran-siddique]

ROADMAP.md names this under Next, and it is the largest honesty gap in the conformance suite:

the quote vectors verify chain, signature and nonce binding against a synthetic PKI, which does not establish that an implementation can parse a real AMD, Intel or NVIDIA quote

The SDK has real captures (snp_quote_azure.json, tdx_quote_azure.json, tdx_quote_gcp.json, gpu_h100_attestation.json) and its own tests verify against them. The conformance suite does not: an independent implementation can pass wcm conformance today and still fail on the first real quote it sees.

What a proposal needs to settle before any vectors are written:

1. Vector shape. A conformance vector is language-neutral JSON with an expected outcome. A real capture drags in a certificate chain, a vendor root that must not come from the chain being verified, and a validity window. Does the vector carry the chain inline, or name a root the runner is expected to have staged?
2. Nonce binding, honestly. The Azure SEV-SNP capture binds the vTPM attestation key in REPORT_DATA, not a caller nonce, so "the report echoes the nonce" is not a property every real capture has. The vector needs to express which binding applies, per capture, rather than assuming one.
3. Expiry. Real leaf certificates expire. A vector that starts failing in a year is worse than no vector. Injected clock, or a documented refresh duty?
4. Refusals. The positive case proves little on its own. Wrong nonce, tampered body, stripped chain, and a root outside the chain are the cases that separate a parser from a verifier.

GPU-side is the same gap with one extra wrinkle: an H200 report has ranges that move between calls even under an identical nonce (see the note in nvidia.py), so a vector must not encourage pinning a report digest.

Design proposal welcome before implementation. Please open the discussion here rather than opening with a PR full of vectors.

[zohebk8s]

@imran-siddique Design proposal. Before answering I measured the corpus rather than reasoned about it, because two of your four questions turn out to have answers already sitting in tests/fixtures, and one of them is not the answer I expected.

What the corpus actually looks like today

The conformance suite ships 91 vectors. Eight carry certificates, 21 certificates in total, and every one of them has one of three subjects:

Subject	Role
CN=wcm-conformance-platform-root	self-signed root
CN=wcm-conformance-platform-intermediate	issued by that root
CN=attestation-key	the leaf

Zero issued by AMD, Intel or NVIDIA. So the gap is exactly as you describe it: accept-cryptographically-verified-quote verifies a chain, a signature and a REPORT_DATA binding, and an implementation can satisfy all three without ever having seen vendor bytes.

1. Vector shape

Your question is whether a vector carries the chain inline or names a root the runner must have staged. Your own captures already answer it both ways, and a third way:

Fixture	Evidence field	Root	Nonce field
snp_quote_azure.json	report_b64	root_pem inline	expected_nonce, null
tdx_quote_gcp.json	quote_b64	intel_sgx_root_ca_sha256	expected_nonce
tdx_quote_azure.json	quote_b64	intel_sgx_root_ca_sha256	expected_nonce, null
gpu_h100_attestation.json	report_b64	none, chain only	nonce

Four captures, three shapes, and the GPU one uses a different key name for the nonce. Whatever format you choose has to normalise these, so I would treat this less as an open design question and more as picking which of your existing conventions wins.

My recommendation: name the root, carry everything below it inline. The TDX shape, generalised.

The reason is the one that makes a capture worth anything. A chain that supplies its own anchor proves only that the vector is internally consistent, which the synthetic PKI already proves. Naming the root and requiring the runner to have staged it is what makes the test about the vendor rather than about the file. The SNP fixtures carrying root_pem are convenient and prove less.

Field	Content
leaf_pem, intermediates_pem	inline, ordered leaf to root
root	an identifier plus a digest of the DER
root_source	where a runner obtains it, as documentation, not as a fetch

A runner that cannot find the named root fails with "root not staged" rather than fetching it. A conformance suite that reaches the network mid-run fails differently on a plane, and staging a vendor root is a step every real implementer has to do anyway.

2. Nonce binding, declared per capture

Confirmed, and your own fixture says it out loud:

Genuine Azure SEV-SNP report via the vTPM (NV 0x01400001). REPORT_DATA is paravisor-bound to the vTPM AK, not a guest nonce, so expected_nonce is null.

The trap is worse than one capture behaving differently. A format with a nonce field invites an implementer to assume the echo property is universal, pass by checking the wrong thing, and never find out. So the vector should state which binding it asserts and a runner should have to handle the kind it is given:

kind	Meaning
nonce-digest	REPORT_DATA equals sha256(nonce)
nonce-and-transport	REPORT_DATA equals sha256(nonce || transport key), which is what stops relay
attestation-key	bound to a platform key, as in both Azure captures; no caller freshness
none	the capture proves signature and chain and nothing about freshness

An unrecognised kind should be a hard failure, not a skip. A runner that silently skips a binding it does not understand passes vectors it did not check.

none earning a place matters. Some real captures genuinely prove only that a chip signed something, and a format that cannot say so will have a nonce invented for it.

3. Expiry: injected clock, and the horizon is further out than it looks

I measured every certificate in the corpus, including the PCK chains embedded inside the TDX quotes.

Certificate	Expires	Days from today
AMD VCEK, snp_quote_azure.json	2032-01-26	1965
Intel PCK leaf, tdx_quote_azure.json	2033-02-01	2336
Intel PCK, remaining TDX captures	2033-05 to 2033-07	2445 to 2514
Intel SGX root	2049-12-31	8514
AMD SEV-Milan and ARK-Milan	2045-10-22	6983
NVIDIA chain, all five certificates	9999-12-31	never

Nothing in the corpus expires within a year. Nothing expires within five. The soonest is the AMD VCEK on 26 January 2032, and the NVIDIA chain never expires at all.

So the concern is right in principle and the deadline is not near. That changes what the answer should be. I would still use an injected clock, because determinism is worth having for its own sake and a conformance score that changes when a certificate ages is not a score. But I would not build a refresh process for a 2032 problem.

Tier	Clock	Scored
conformance	now from the vector, deterministic forever	yes
live	wall clock, expected to fail eventually	no, reported separately

The live tier is where "this capture aged out" becomes visible without degrading the score. And since the dates are known, I would record not_after of the shortest-lived certificate in each vector, so whoever inherits this can see the horizon without parsing chains.

4. Refusals, as a matrix over every capture

Agreed the positive case proves little. I would make refusals a matrix applied to every capture, so a new capture cannot be contributed with only a happy path:

Case	Expected
wrong nonce or binding	REPORT_DATA mismatch
tampered report body	signature fails
tampered signature	signature fails
stripped chain	no path to a trusted root
root outside the chain	no path to a trusted root
expired at the injected now	outside the validity window

Two things I got wrong myself while building the negatives for my own capture, offered so the format does not invite them.

Assert on a reason substring rather than an exact message, or the vectors become a change detector for wording.

And the untrusted-root case is easy to write and test nothing. My first attempt used the leaf's own issuer as the "untrusted" root, which of course verifies correctly, because that is a legitimate anchor. My second used a VCEK, which the certificate policy rejects for a non-positive serial before any chain logic runs, so it failed for the wrong reason and still looked like a pass. It has to be a root genuinely outside the chain, and the vector format should probably supply one rather than leave each runner to invent it.

The GPU wrinkle

Agreed. Those are the ranges I measured on the H200 and sent over, now in nvidia.py. Three ranges of a 4,129-byte report move between calls: the nonce at [4, 36), the signature at [4033, 4129), and 32 bytes at [3565, 3597) that change on every call even under an identical nonce, leaving 3,969 stable. The method matters for this issue as much as the numbers: they only appear if you fetch one report twice under the same nonce and once under a different one, which separates device freshness from nonce-derived change. Varying the nonce hides the middle range, so a pin built by diffing two reports with different nonces looks stable and is not.

My tooling re-proves those offsets at runtime rather than trusting the constants, because they are specific to that device and firmware, so I can rerun it on any H-series card I get access to.

So a vector must not pin a digest of the report, and must not pin the full report bytes either. What it can pin is the verification outcome, the certificate chain identity, and the binding check.

I would state that as a rule of the format rather than a note on one vector, because the failure mode is an implementer pinning a digest, watching it pass on their capture, and shipping something that breaks on the second call.

Two things this implies

Captures need provenance. Vendor, part, firmware or TCB level, and capture date. Without it a vector that starts failing is unattributable, and nobody can tell whether the implementation regressed or the platform moved.

Somebody has to own the refresh. An injected clock keeps vectors green forever, which means nothing forces a refresh, which means the corpus can drift from what implementers actually meet. The live tier is my answer, but it only works if someone reads it. Worth naming who, in the issue, rather than leaving it implied.

What I can contribute

A real SEV-SNP capture with the full binding: report, VCEK, the challenge nonce and the transport public key, so REPORT_DATA can be recomputed and the whole thing verifies away from the machine that produced it. Taken on a GCP N2D guest on 8 September, and it verifies end to end on 0.28.1 from PyPI. I also have a TDX capture at SEAM SVN 15, which is the one I offered in #117.

I would rather agree the format here first and then supply captures that fit it, than send captures and have the format bend around what I happen to have.

[zohebk8s]

Disclosure: Commercial Interest

I am the proposer in this repository, and I am recording a commercial interest so that it is on the record before it bears on any decision.

I build Custodian, a key broker for model weights that uses the Weight Custody Manifest SDK as a dependency rather than a fork.

• I will note this interest on any proposal where it applies, whether I am proposing it or taking part in its review.
• I will recuse where Custodian's own design is the question.
• I will not present Custodian as endorsed by this project or as the reference implementation. It is neither.
• I will update this notice if the interest changes materially.

Mohammed Zoheb Shaik

[imran-siddique]

Checked the measurements against the tree rather than reading them off the tables. The three shapes are there: root_pem inline on the SNP pair, intel_sgx_root_ca_sha256 on the TDX pair, and gpu_h100_attestation.json carrying cert_chain_pem with no root at all and its nonce field named nonce rather than expected_nonce. Expiry checks out too: VCEK 2032-01-26, ARK-Milan 2045-10-22, and all five NVIDIA certificates at 9999-12-31. Loading the VCEK raised the non-positive-serial warning you hit, so that trap is real and reproducible from this repository.

Design approved as proposed. Decisions on the parts that are mine.

Vector shape: named root, and the GPU capture is the test of it

Take it. Name the root, carry leaf and intermediates inline, fail with "root not staged" rather than fetching. Your reason is the right one: a chain carrying its own anchor proves internal consistency, which the synthetic PKI already proves, and the point of a vendor capture is to be about the vendor.

Two consequences to settle now rather than discover later.

The SNP fixtures move to the new shape and stop carrying root_pem. Do that as a separate PR after the format lands, so a format change and a fixture migration are not tangled in one diff and one review.

gpu_h100_attestation.json is the hard case and worth designing against rather than around. It has no root today, and its chain terminates in CN=NVIDIA Device Identity CA which is inside the file, so under this rule it must name an NVIDIA root the runner stages. If NVIDIA does not publish one in a form a runner can stage, say that in the vector as a stated limit rather than letting the chain anchor itself quietly. A format whose hardest case is handled by exception is not a format.

Binding kinds

Take it, including none and including the hard failure on an unrecognised kind. The silent skip is the failure mode that matters, because a runner that skips what it does not understand reports a pass it never performed.

Expiry

Take it. Injected clock for the scored tier, wall clock for a separately reported live tier, and record not_after of the shortest-lived certificate in each vector.

You have answered my concern rather than deferred it. The horizon is 2032 and a refresh process built now would be six years of maintenance for a problem the recorded not_after will surface on its own. The recorded date is what makes that a decision rather than an oversight, so treat it as required in the format, not optional.

Refusals

Take the matrix, applied to every capture, so a new capture cannot arrive with only a happy path.

The out-of-chain untrusted root should be one root shipped with the suite, not invented per vector. Your two failed attempts are the argument: an anchor that legitimately verifies, and a VCEK rejected on serial policy before chain logic runs, both look like passes. Every runner should fail that case for the same reason, which only happens if they are all given the same root.

Reason substrings rather than exact messages, agreed.

The GPU rule

State it as a rule of the format: a vector must not pin a digest of the report, nor the full report bytes. It may pin the verification outcome, the chain identity, and the binding check.

Put the reason next to it, because the rule without the reason will be relaxed by whoever finds it inconvenient: three ranges move between calls, and the 32 bytes at [3565, 3597) change even under an identical nonce, so a pin built by diffing two reports under different nonces looks stable and is not. That is a method note, not a numbers note, and it is the part that stops the next person repeating it.

Sequence

Format first, with the refusal matrix and the rules above. Then the SNP fixture migration. Then new captures. Open the format as a PR when ready.