fix(demos): demo 10 denied every call on a catalog drift CI could not see - #68
Merged
Merged
Conversation
… see The gateway logged UPSTREAM_CATALOG_DRIFT tool=model.chat_completion kind=rug_pull policy=fail_closed and answered all five requests with 503. cMCP hashes description plus input schema plus output schema on both sides of the boundary, so catalog.json and model_server.py have to describe the same tool. They did not. The catalog declared contains_identifiers and an output schema; the server advertised neither. Approved digest f59f39e0, advertised dc448259. contains_identifiers is not decoration. endpoint.py computes it and allow.cedar reads it in identifiers-redacted-before-egress, so the one argument that carries the PII decision was missing from the schema the gateway checks. PR #62 added definition_hash without a check that the two sides still agreed, and nothing caught it. The output schema now describes what model_server actually returns (model, region, cloud, content, upstream) rather than a result field that never existed, and definition_hash is recomputed with the runtime's own _compute_definition_hash. Demo 10 now reads the way the README says: public allowed, PII allowed once redacted, confidential denied out of region, PHI denied on shared infrastructure, PHI allowed on dedicated. The signed claim carries all five, audit_chain.length 9. Why CI stayed green: run.py called subprocess.call and dropped the return code, then returned 0. CONTRIBUTING.md already sets the bar, exit non-zero on a failed assertion so CI catches it, and this launcher was the exception. It now returns the client's code, and client.py prints the 404 and names the drift instead of dumping a urllib traceback. tests/test_catalog_definition_drift.py locks the rule down statically, no ports and no gateway: catalog against advertised tool, every argument endpoint.py sends against the approved properties, and stored definition_hash against computed for every catalog in the repo. Removing contains_identifiers again fails it in 0.2s. tests/ was never wired into ci.yml, so it now runs before the demos. web-console/run.py picked up the busy-port guard the terminal demos already carry. It hit me during this check: 8443 was still held by a finishing demo, the console's own gateway died on errno 10048 in cmcp.log, _wait_for_port answered against the stale one, and the console reported a clean start. That is the failure mode the guard exists for. Verified on a clean venv from requirements.txt (cmcp-runtime 0.5.0, weight-custody-manifest 0.28.1): all ten demos pass, suite exits 0. Web console checked too, three scenarios with the two breaches denying the write on concentration-limit-breached and cdd-clearance-required, and the tampered bundle failing policy_bundle.hash against the pinned verifier. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017WBSM72tRqvCfmKGRYkbHX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gateway logged UPSTREAM_CATALOG_DRIFT tool=model.chat_completion kind=rug_pull
policy=fail_closed and answered all five requests with 503. cMCP hashes description
plus input schema plus output schema on both sides of the boundary, so catalog.json
and model_server.py have to describe the same tool. They did not. The catalog declared
contains_identifiers and an output schema; the server advertised neither. Approved
digest f59f39e0, advertised dc448259.
contains_identifiers is not decoration. endpoint.py computes it and allow.cedar reads
it in identifiers-redacted-before-egress, so the one argument that carries the PII
decision was missing from the schema the gateway checks. PR #62 added definition_hash
without a check that the two sides still agreed, and nothing caught it.
The output schema now describes what model_server actually returns (model, region,
cloud, content, upstream) rather than a result field that never existed, and
definition_hash is recomputed with the runtime's own _compute_definition_hash.
Demo 10 now reads the way the README says: public allowed, PII allowed once redacted,
confidential denied out of region, PHI denied on shared infrastructure, PHI allowed on
dedicated. The signed claim carries all five, audit_chain.length 9.
Why CI stayed green: run.py called subprocess.call and dropped the return code, then
returned 0. CONTRIBUTING.md already sets the bar, exit non-zero on a failed assertion
so CI catches it, and this launcher was the exception. It now returns the client's
code, and client.py prints the 404 and names the drift instead of dumping a urllib
traceback.
tests/test_catalog_definition_drift.py locks the rule down statically, no ports and no
gateway: catalog against advertised tool, every argument endpoint.py sends against the
approved properties, and stored definition_hash against computed for every catalog in
the repo. Removing contains_identifiers again fails it in 0.2s. tests/ was never wired
into ci.yml, so it now runs before the demos.
web-console/run.py picked up the busy-port guard the terminal demos already carry. It
hit me during this check: 8443 was still held by a finishing demo, the console's own
gateway died on errno 10048 in cmcp.log, _wait_for_port answered against the stale one,
and the console reported a clean start. That is the failure mode the guard exists for.
Verified on a clean venv from requirements.txt (cmcp-runtime 0.5.0,
weight-custody-manifest 0.28.1): all ten demos pass, suite exits 0. Web console checked
too, three scenarios with the two breaches denying the write on
concentration-limit-breached and cdd-clearance-required, and the tampered bundle
failing policy_bundle.hash against the pinned verifier.
🤖 Generated with Claude Code
https://claude.ai/code/session_017WBSM72tRqvCfmKGRYkbHX