Skip to content

fix(demos): demo 10 denied every call on a catalog drift CI could not see - #68

Merged
imran-siddique merged 1 commit into
mainfrom
fix/demo-10-catalog-drift-and-exit-code
Sep 10, 2026
Merged

fix(demos): demo 10 denied every call on a catalog drift CI could not see#68
imran-siddique merged 1 commit into
mainfrom
fix/demo-10-catalog-drift-and-exit-code

Conversation

@imran-siddique

Copy link
Copy Markdown
Member

The gateway logged UPSTREAM_CATALOG_DRIFT tool=model.chat_completion kind=rug_pull
policy=fail_closed and answered all five requests with 503. cMCP hashes description
plus input schema plus output schema on both sides of the boundary, so catalog.json
and model_server.py have to describe the same tool. They did not. The catalog declared
contains_identifiers and an output schema; the server advertised neither. Approved
digest f59f39e0, advertised dc448259.

contains_identifiers is not decoration. endpoint.py computes it and allow.cedar reads
it in identifiers-redacted-before-egress, so the one argument that carries the PII
decision was missing from the schema the gateway checks. PR #62 added definition_hash
without a check that the two sides still agreed, and nothing caught it.

The output schema now describes what model_server actually returns (model, region,
cloud, content, upstream) rather than a result field that never existed, and
definition_hash is recomputed with the runtime's own _compute_definition_hash.

Demo 10 now reads the way the README says: public allowed, PII allowed once redacted,
confidential denied out of region, PHI denied on shared infrastructure, PHI allowed on
dedicated. The signed claim carries all five, audit_chain.length 9.

Why CI stayed green: run.py called subprocess.call and dropped the return code, then
returned 0. CONTRIBUTING.md already sets the bar, exit non-zero on a failed assertion
so CI catches it, and this launcher was the exception. It now returns the client's
code, and client.py prints the 404 and names the drift instead of dumping a urllib
traceback.

tests/test_catalog_definition_drift.py locks the rule down statically, no ports and no
gateway: catalog against advertised tool, every argument endpoint.py sends against the
approved properties, and stored definition_hash against computed for every catalog in
the repo. Removing contains_identifiers again fails it in 0.2s. tests/ was never wired
into ci.yml, so it now runs before the demos.

web-console/run.py picked up the busy-port guard the terminal demos already carry. It
hit me during this check: 8443 was still held by a finishing demo, the console's own
gateway died on errno 10048 in cmcp.log, _wait_for_port answered against the stale one,
and the console reported a clean start. That is the failure mode the guard exists for.

Verified on a clean venv from requirements.txt (cmcp-runtime 0.5.0,
weight-custody-manifest 0.28.1): all ten demos pass, suite exits 0. Web console checked
too, three scenarios with the two breaches denying the write on
concentration-limit-breached and cdd-clearance-required, and the tampered bundle
failing policy_bundle.hash against the pinned verifier.

🤖 Generated with Claude Code

https://claude.ai/code/session_017WBSM72tRqvCfmKGRYkbHX

… see

The gateway logged UPSTREAM_CATALOG_DRIFT tool=model.chat_completion kind=rug_pull
policy=fail_closed and answered all five requests with 503. cMCP hashes description
plus input schema plus output schema on both sides of the boundary, so catalog.json
and model_server.py have to describe the same tool. They did not. The catalog declared
contains_identifiers and an output schema; the server advertised neither. Approved
digest f59f39e0, advertised dc448259.

contains_identifiers is not decoration. endpoint.py computes it and allow.cedar reads
it in identifiers-redacted-before-egress, so the one argument that carries the PII
decision was missing from the schema the gateway checks. PR #62 added definition_hash
without a check that the two sides still agreed, and nothing caught it.

The output schema now describes what model_server actually returns (model, region,
cloud, content, upstream) rather than a result field that never existed, and
definition_hash is recomputed with the runtime's own _compute_definition_hash.

Demo 10 now reads the way the README says: public allowed, PII allowed once redacted,
confidential denied out of region, PHI denied on shared infrastructure, PHI allowed on
dedicated. The signed claim carries all five, audit_chain.length 9.

Why CI stayed green: run.py called subprocess.call and dropped the return code, then
returned 0. CONTRIBUTING.md already sets the bar, exit non-zero on a failed assertion
so CI catches it, and this launcher was the exception. It now returns the client's
code, and client.py prints the 404 and names the drift instead of dumping a urllib
traceback.

tests/test_catalog_definition_drift.py locks the rule down statically, no ports and no
gateway: catalog against advertised tool, every argument endpoint.py sends against the
approved properties, and stored definition_hash against computed for every catalog in
the repo. Removing contains_identifiers again fails it in 0.2s. tests/ was never wired
into ci.yml, so it now runs before the demos.

web-console/run.py picked up the busy-port guard the terminal demos already carry. It
hit me during this check: 8443 was still held by a finishing demo, the console's own
gateway died on errno 10048 in cmcp.log, _wait_for_port answered against the stale one,
and the console reported a clean start. That is the failure mode the guard exists for.

Verified on a clean venv from requirements.txt (cmcp-runtime 0.5.0,
weight-custody-manifest 0.28.1): all ten demos pass, suite exits 0. Web console checked
too, three scenarios with the two breaches denying the write on
concentration-limit-breached and cdd-clearance-required, and the tampered bundle
failing policy_bundle.hash against the pinned verifier.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017WBSM72tRqvCfmKGRYkbHX
@imran-siddique
imran-siddique requested a review from a team as a code owner September 10, 2026 20:17
@imran-siddique
imran-siddique merged commit 6446968 into main Sep 10, 2026
5 checks passed
@imran-siddique
imran-siddique deleted the fix/demo-10-catalog-drift-and-exit-code branch September 10, 2026 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant