diff --git a/examples/sb-runtime-governed/.gitignore b/examples/sb-runtime-governed/.gitignore new file mode 100644 index 000000000..557ad77e0 --- /dev/null +++ b/examples/sb-runtime-governed/.gitignore @@ -0,0 +1,3 @@ +receipts/ +__pycache__/ +*.pyc diff --git a/examples/sb-runtime-governed/README.md b/examples/sb-runtime-governed/README.md new file mode 100644 index 000000000..d82564122 --- /dev/null +++ b/examples/sb-runtime-governed/README.md @@ -0,0 +1,129 @@ +# sb-runtime Governed Example + +Demonstrates the architectural claim of the `sb-runtime` integration: + +> **The same Cedar policy produces semantically-equivalent signed receipts regardless of the sandbox layer that wraps the agent process. Auditors verify every receipt with one public key. The `sandbox_backend` field is inside the signature scope, so an operator cannot claim a hardened sandbox at verify time if the receipt was produced under `none`.** + +This is PR 3 in the three-PR sequence proposed on [#748](https://github.com/microsoft/agent-governance-toolkit/issues/748): + +| # | What | Status | +|---|---|---| +| 1 | Integration doc at [`docs/integrations/sb-runtime.md`](../../docs/integrations/sb-runtime.md) | Merged ([#1202](https://github.com/microsoft/agent-governance-toolkit/pull/1202)) | +| 2 | Provider shim at [`packages/agentmesh-integrations/sb-runtime-skill/`](../../packages/agentmesh-integrations/sb-runtime-skill/) | Merged ([#1203](https://github.com/microsoft/agent-governance-toolkit/pull/1203)) | +| 3 | Worked example (this directory) | This PR | + +## Quick start + +```bash +pip install -e packages/agentmesh-integrations/sb-runtime-skill/ +python examples/sb-runtime-governed/getting_started.py +``` + +Expected exit code: `0` (all 18 receipts verify, tamper test fails as designed). + +## What the demo does + +The demo runs the **same six actions** (three allowed, three denied) across **three sandbox-backend configurations**, using the **same Cedar policy** and the **same operator Ed25519 key**: + +| Scenario | `sandbox_backend` | Ring | Who owns the sandbox layer | +|---|---|:---:|---| +| `standalone` | `sb_runtime_builtin` | 3 | sb-runtime's own Landlock + seccomp | +| `nono` | `nono` | 2 | [nono](https://github.com/always-further/nono) capability set (recommended Linux path per [#1202](https://github.com/microsoft/agent-governance-toolkit/pull/1202)) | +| `openshell` | `openshell` | 2 | OpenShell container boundary | + +Each scenario produces six signed receipts in the [Veritas Acta receipt format](https://datatracker.ietf.org/doc/draft-farley-acta-signed-receipts/). The demo then: + +1. **Cross-verifies every receipt** (18 total) against the single operator public key, with no dependency on the `sb_runtime_agentmesh` skill at verify time. +2. **Demonstrates tamper-evidence** by flipping the `sandbox_backend` field on one receipt and confirming verification fails, proving the backend choice is covered by the Ed25519 signature and not sidecar metadata. +3. **Confirms chain linkage** by showing `receipt[1].previousReceiptHash == sha256(canonical(receipt[0]))` within a scenario. +4. **Writes receipts to disk** at `examples/sb-runtime-governed/receipts/` so the output can be inspected and re-verified with external tooling. + +## Expected output + +``` +======================================================================== + sb-runtime Governed Agent — Multi-backend Receipt Portability +======================================================================== + + Operator key: kid = UbV24JP2YJDzwYwX... + Policy: .../sandbox-policy.yaml + Agent DID: did:mesh:sb-runtime-demo-agent + +------------------------------------------------------------------------ + Scenario summaries (same policy, three sandbox backends) +------------------------------------------------------------------------ + + Scenario: standalone + sandbox_backend = "sb_runtime_builtin" ring=3 + receipts: 6 (3 allow, 3 deny) + policy digest: sha256:095c56b995768de62... + sample payload fields: type=sb-runtime:decision ring=3 sandbox_backend=sb_runtime_builtin + + Scenario: nono + sandbox_backend = "nono" ring=2 + receipts: 6 (3 allow, 3 deny) + policy digest: sha256:095c56b995768de62... + sample payload fields: type=sb-runtime:decision ring=2 sandbox_backend=nono + + Scenario: openshell + sandbox_backend = "openshell" ring=2 + receipts: 6 (3 allow, 3 deny) + policy digest: sha256:095c56b995768de62... + sample payload fields: type=sb-runtime:decision ring=2 sandbox_backend=openshell + +------------------------------------------------------------------------ + Cross-verification (single public key, all 18 receipts) +------------------------------------------------------------------------ + + Verified: 18 / 18 [ALL PASS] + +------------------------------------------------------------------------ + Tamper-evidence demonstration +------------------------------------------------------------------------ + + Tamper test: flip sandbox_backend on a receipt from the nono scenario + Before: 'nono' verifies = True + After: 'sb_runtime_builtin' verifies = False + -> sandbox_backend is inside the signature scope, not sidecar metadata. + + Chain linkage (scenario: standalone) + hash(receipt[0]) = 0NECPTsP6PFdM9VaNPFPX_a4cyk7GftN... + receipt[1].previousReceiptHash = 0NECPTsP6PFdM9VaNPFPX_a4cyk7GftN... + match = True +``` + +## What an auditor sees + +Every scenario's receipt contains: + +- `payload.decision` ∈ `{allow, deny, require_approval}` +- `payload.policy_digest` — identical across scenarios (same policy) +- `payload.ring` ∈ `{2, 3}` — runtime ring +- `payload.sandbox_backend` — which layer wrapped the process +- `payload.previousReceiptHash` — chain linkage +- `signature.alg` = `"EdDSA"` +- `signature.kid` — identical across scenarios (same operator key) +- `signature.sig` — Ed25519 signature over the JCS-canonicalized payload + +Verification path for any receipt, from a fresh machine with no AGT dependency: + +```bash +npx @veritasacta/verify examples/sb-runtime-governed/receipts/standalone/000.json \ + --key examples/sb-runtime-governed/receipts/operator-public.pem +``` + +The public key is written to disk alongside the receipts so the demo is self-contained. In production, operators publish the pubkey via an agent card extension, DID document service endpoint, or pinned JWKS URL; `@veritasacta/verify` resolves from any of those via `--jwks` or `--trust-anchor`. + +## Policy + +See [`policies/sandbox-policy.yaml`](./policies/sandbox-policy.yaml). Mirrors the style of `examples/openshell-governed/policies/sandbox-policy.yaml` so cross-example comparison is straightforward. + +## Related + +- **Integration doc:** [`docs/integrations/sb-runtime.md`](../../docs/integrations/sb-runtime.md) +- **Provider shim package:** [`packages/agentmesh-integrations/sb-runtime-skill/`](../../packages/agentmesh-integrations/sb-runtime-skill/) +- **Reference verifier:** [`@veritasacta/verify`](https://github.com/ScopeBlind/verify) (Apache-2.0, offline, zero runtime dependencies on AGT or sb-runtime) +- **Receipt format spec:** [draft-farley-acta-signed-receipts-02](https://datatracker.ietf.org/doc/draft-farley-acta-signed-receipts/) +- **Conformance profile:** [VeritasActa/agt-integration-profile](https://github.com/VeritasActa/agt-integration-profile) +- **Sibling example:** [`examples/openshell-governed/`](../openshell-governed/) (same policy contract, no receipts) +- **Sandbox primitive (recommended composition):** [nono](https://github.com/always-further/nono) (Always Further, Apache-2.0) diff --git a/examples/sb-runtime-governed/getting_started.py b/examples/sb-runtime-governed/getting_started.py new file mode 100644 index 000000000..e0c237825 --- /dev/null +++ b/examples/sb-runtime-governed/getting_started.py @@ -0,0 +1,327 @@ +#!/usr/bin/env python3 +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +""" +sb-runtime Governed Example — Getting Started + +Demonstrates the distinctive architectural claim of the sb-runtime integration: +a single Cedar policy produces semantically-equivalent signed receipts +regardless of the sandbox layer that wraps the agent process. The sandbox +backend is recorded in the signed payload so auditors can see which layer +ran (sb-runtime's own Landlock + seccomp, nono, OpenShell, or none), but +signature verification and policy-digest pinning are identical across all +backends. + +Three scenarios run against the same policy and same operator key: + + 1. Standalone sb-runtime (Ring 3) + sandbox_backend = "sb_runtime_builtin" + sb-runtime's own Landlock + seccomp owns the sandbox. + + 2. sb-runtime + nono (Ring 2) + sandbox_backend = "nono" + A nono capability set wraps the process; sb-runtime contributes + only Cedar evaluation + receipt signing. + + 3. sb-runtime + OpenShell (Ring 2) + sandbox_backend = "openshell" + An OpenShell container boundary; sb-runtime contributes only + Cedar + receipts. + +All 18 receipts (6 actions x 3 scenarios) verify against the same operator +public key. The demo then tampers with the `sandbox_backend` field in one +receipt to prove that the backend choice is covered by the Ed25519 +signature, not just sidecar metadata. + +Usage: + pip install -e packages/agentmesh-integrations/sb-runtime-skill/ + python examples/sb-runtime-governed/getting_started.py + +Background: + - Integration doc: docs/integrations/sb-runtime.md (merged as PR #1202) + - Provider shim: packages/agentmesh-integrations/sb-runtime-skill/ (merged as PR #1203) + - Receipt format: draft-farley-acta-signed-receipts-02 + - Offline verifier: npx @veritasacta/verify (Apache-2.0, zero dependencies) +""" + +from __future__ import annotations + +import json +import sys +from dataclasses import dataclass +from pathlib import Path + +# Import the skill from the in-tree package. Works without a pip install, +# matching the openshell-governed example's convention. +ROOT = Path(__file__).resolve().parent.parent.parent +SKILL_DIR = ROOT / "packages" / "agentmesh-integrations" / "sb-runtime-skill" +sys.path.insert(0, str(SKILL_DIR)) + +try: + from sb_runtime_agentmesh.skill import GovernanceSkill, SandboxBackend + from sb_runtime_agentmesh.receipts import ( + Signer, + receipt_hash, + verify_receipt, + ) +except ImportError as exc: + print( + "\n This example requires sb-runtime-skill to be importable.\n" + " Run: pip install -e packages/agentmesh-integrations/sb-runtime-skill/\n" + "\n or ensure the repository layout is intact; the example imports\n" + " from packages/agentmesh-integrations/sb-runtime-skill/ directly.\n" + f"\n Import error: {exc}\n" + ) + raise SystemExit(2) + +from cryptography.hazmat.primitives import serialization + +POLICY_DIR = Path(__file__).parent / "policies" +AGENT_DID = "did:mesh:sb-runtime-demo-agent" + +# One tool call series exercised under each sandbox backend. Three allowed, +# three denied, deliberately identical to the openshell-governed example's +# action list for cross-example comparison. +ACTIONS = [ + ("file:read:/workspace/main.py", "Read source file"), + ("shell:python", "Run Python interpreter"), + ("shell:git", "Git commit"), + ("shell:rm -rf /tmp", "Destructive shell"), + ("http:GET:169.254.169.254/metadata", "Cloud metadata exfiltration"), + ("file:write:/etc/shadow", "System-file write"), +] + + +@dataclass +class ScenarioResult: + name: str + backend: SandboxBackend + ring: int + receipts: list[dict] + + +def run_scenario( + name: str, + backend: SandboxBackend, + ring: int, + operator_signer: Signer, +) -> ScenarioResult: + """Run the same six actions through a ScopeBlindExtension... wait, wrong + class: through GovernanceSkill — configured for one specific sandbox + backend. Collect the emitted receipts. Return for cross-verification. + """ + skill = GovernanceSkill( + policy_dir=POLICY_DIR, + signer=operator_signer, + sandbox_backend=backend, + ring=ring, + ) + receipts = [] + for action, _desc in ACTIONS: + decision = skill.check_policy(action, context={"agent_did": AGENT_DID}) + assert decision.receipt is not None, "sign is on by default" + receipts.append(decision.receipt) + return ScenarioResult(name=name, backend=backend, ring=ring, receipts=receipts) + + +def print_scenario_summary(r: ScenarioResult) -> None: + allowed = sum(1 for rec in r.receipts if rec["payload"]["decision"] == "allow") + denied = len(r.receipts) - allowed + kid = r.receipts[0]["signature"]["kid"][:16] + "..." + digest = r.receipts[0]["payload"]["policy_digest"][:24] + "..." + print(f" Scenario: {r.name}") + print(f" sandbox_backend = \"{r.backend.value}\" ring={r.ring}") + print(f" receipts: {len(r.receipts)} ({allowed} allow, {denied} deny)") + print(f" operator kid: {kid}") + print(f" policy digest: {digest}") + # Show the sandbox_backend field in a sample payload so its presence + # is unambiguous to the reader of the demo output. + sample = r.receipts[0]["payload"] + print(f" sample payload fields: " + f"type={sample['type']} " + f"ring={sample['ring']} " + f"sandbox_backend={sample['sandbox_backend']}") + print() + + +def verify_all_receipts( + results: list[ScenarioResult], + operator_signer: Signer, +) -> int: + """Verify every receipt from every scenario against the same public key. + + Demonstrates the central claim: the verification path is identical + regardless of which sandbox backend produced the receipt. The operator + publishes one public key; auditors verify any receipt against it. + """ + pub = serialization.load_pem_public_key(operator_signer.public_pem()) + total = 0 + verified = 0 + for r in results: + for rec in r.receipts: + total += 1 + if verify_receipt(rec, pub): + verified += 1 + else: # pragma: no cover - defensive; shouldn't happen in a clean run + print(f" FAILED: {rec['payload']['action']} " + f"[{r.backend.value}]") + return total, verified + + +def demonstrate_tampering( + results: list[ScenarioResult], + operator_signer: Signer, +) -> None: + """Flip `sandbox_backend` in one receipt, confirm verification fails. + + This is the integrity claim the `sandbox_backend` field makes. The + field is inside the Ed25519 signature scope, not carried alongside it, + so an attacker claiming a Ring 3 hardened run actually used `none` + (no sandbox) would be detected at verify time. + """ + pub = serialization.load_pem_public_key(operator_signer.public_pem()) + # Pick a receipt from the nono scenario and flip it to claim sb_runtime_builtin + victim_scenario = next(r for r in results if r.backend is SandboxBackend.NONO) + original = victim_scenario.receipts[0] + tampered = { + "payload": {**original["payload"], "sandbox_backend": "sb_runtime_builtin"}, + "signature": original["signature"], + } + valid_before = verify_receipt(original, pub) + valid_after = verify_receipt(tampered, pub) + print(f" Tamper test: flip sandbox_backend on a receipt from the nono scenario") + print(f" Before: {original['payload']['sandbox_backend']!r} verifies = {valid_before}") + print(f" After: {tampered['payload']['sandbox_backend']!r} verifies = {valid_after}") + if valid_before and not valid_after: + print(f" -> sandbox_backend is inside the signature scope, not sidecar metadata.") + else: # pragma: no cover - defensive + print(f" !! unexpected verification outcome; see receipts.py _check_no_embedded_key") + print() + + +def demonstrate_chain_linkage(results: list[ScenarioResult]) -> None: + """Confirm successive receipts within a scenario link via previousReceiptHash. + + The chain is tamper-evident because each subsequent receipt binds to + the SHA-256 of the prior envelope. Modifying any earlier receipt + breaks the chain from that point forward at verify time. + """ + r = results[0] + first = r.receipts[0] + second = r.receipts[1] + expected = receipt_hash(first) + actual = second["payload"].get("previousReceiptHash") + print(f" Chain linkage (scenario: {r.name})") + print(f" hash(receipt[0]) = {expected[:32]}...") + print(f" receipt[1].previousReceiptHash = {actual[:32] if actual else '(missing)'}...") + print(f" match = {actual == expected}") + print() + + +def print_verify_instructions(operator_signer: Signer, out_dir: Path) -> None: + """Emit the exact commands an auditor would use to verify externally.""" + print(" Offline verification (no dependency on this skill or on AGT):") + print() + print(" # publish the operator public key out-of-band, e.g. as a JWKS URL") + print(" # or write it to a file and pin:") + print(f" cat {out_dir / 'operator-public.pem'}") + print() + print(" # verify any emitted receipt with the canonical verifier CLI:") + print(f" npx @veritasacta/verify {out_dir}/standalone/000.json \\") + print(f" --key {out_dir / 'operator-public.pem'}") + print() + print(" # receipt format reference: draft-farley-acta-signed-receipts-02") + print(" # https://datatracker.ietf.org/doc/draft-farley-acta-signed-receipts/") + print() + + +def write_receipts_to_disk( + results: list[ScenarioResult], + operator_signer: Signer, + out_dir: Path, +) -> None: + """Write every receipt plus the operator public key to disk under out_dir. + + Layout: + out_dir/ + operator-public.pem + standalone/{000.json, 001.json, ...} + nono/{000.json, ...} + openshell/{000.json, ...} + """ + out_dir.mkdir(parents=True, exist_ok=True) + (out_dir / "operator-public.pem").write_bytes(operator_signer.public_pem()) + for r in results: + sub = out_dir / r.name + sub.mkdir(exist_ok=True) + for i, rec in enumerate(r.receipts): + (sub / f"{i:03d}.json").write_text(json.dumps(rec, indent=2, sort_keys=True)) + + +def main() -> int: + print("=" * 72) + print(" sb-runtime Governed Agent — Multi-backend Receipt Portability") + print("=" * 72) + print() + + operator = Signer.generate() + print(f" Operator key: kid = {operator.kid}") + print(f" Policy: {POLICY_DIR}/sandbox-policy.yaml") + print(f" Agent DID: {AGENT_DID}") + print() + + scenarios = [ + ("standalone", SandboxBackend.SB_RUNTIME_BUILTIN, 3), + ("nono", SandboxBackend.NONO, 2), + ("openshell", SandboxBackend.OPENSHELL, 2), + ] + results = [ + run_scenario(name, backend, ring, operator) + for name, backend, ring in scenarios + ] + + print("-" * 72) + print(" Scenario summaries (same policy, three sandbox backends)") + print("-" * 72) + print() + for r in results: + print_scenario_summary(r) + + print("-" * 72) + print(" Cross-verification (single public key, all 18 receipts)") + print("-" * 72) + print() + total, verified = verify_all_receipts(results, operator) + status = "ALL PASS" if verified == total else f"FAIL ({total - verified} failed)" + print(f" Verified: {verified} / {total} [{status}]") + print() + + print("-" * 72) + print(" Tamper-evidence demonstration") + print("-" * 72) + print() + demonstrate_tampering(results, operator) + demonstrate_chain_linkage(results) + + out_dir = Path(__file__).parent / "receipts" + write_receipts_to_disk(results, operator, out_dir) + + print("-" * 72) + print(f" Receipts written to {out_dir.relative_to(Path.cwd()) if out_dir.is_absolute() else out_dir}") + print("-" * 72) + print() + print_verify_instructions(operator, out_dir.relative_to(Path.cwd()) if out_dir.is_absolute() else out_dir) + + print("=" * 72) + print(" The same signed receipt format across all three sandbox layers.") + print(" Auditors verify every receipt with one public key, zero runtime") + print(" dependencies on AGT or sb-runtime. The sandbox_backend field is") + print(" inside the signature, so an operator cannot claim 'Ring 3' at") + print(" verify time if the receipt was actually produced under 'none'.") + print("=" * 72) + return 0 if verified == total else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/examples/sb-runtime-governed/policies/sandbox-policy.yaml b/examples/sb-runtime-governed/policies/sandbox-policy.yaml new file mode 100644 index 000000000..7337aa36f --- /dev/null +++ b/examples/sb-runtime-governed/policies/sandbox-policy.yaml @@ -0,0 +1,40 @@ +apiVersion: governance.toolkit/v1 +# Shared policy exercised across all three sandbox-layer scenarios in this +# example. The claim this policy lets us demonstrate is architectural: +# the SAME policy produces SEMANTICALLY-EQUIVALENT receipts regardless of +# whether the sandbox is sb-runtime's own Landlock+seccomp, nono, or +# OpenShell. The receipt envelope carries the sandbox_backend choice so +# an auditor can see which layer wrapped the process, but verification +# of the signature + policy digest is identical across all three. +rules: + - name: block-destructive-shell + condition: {field: action, operator: matches, value: "shell:(rm|dd|mkfs|shutdown|reboot|kill)"} + action: deny + priority: 100 + message: Destructive shell command blocked + - name: block-metadata-endpoint + condition: {field: action, operator: contains, value: "169.254.169.254"} + action: deny + priority: 100 + message: Cloud metadata endpoint blocked + - name: block-system-writes + condition: {field: action, operator: matches, value: "file:write:/(etc|usr|root|var/log)"} + action: deny + priority: 95 + message: Writes to system directories blocked + - name: allow-workspace-read + condition: {field: action, operator: matches, value: "^file:read:/(workspace|tmp)"} + action: allow + priority: 90 + - name: allow-workspace-write + condition: {field: action, operator: matches, value: "^file:write:/(workspace|tmp)"} + action: allow + priority: 85 + - name: allow-safe-shell + condition: {field: action, operator: in, value: ["shell:ls","shell:cat","shell:grep","shell:python","shell:pip","shell:git","shell:curl"]} + action: allow + priority: 80 + - name: allow-github-api + condition: {field: action, operator: matches, value: "^http:(GET|POST):api\\.github\\.com"} + action: allow + priority: 80