examples: sb-runtime-governed — multi-backend receipt portability (PR 3 of 3 from #748)

[tomjwxf]

Summary

PR 3 (final) in the three-PR sequence proposed on #748. Adds a runnable worked example at examples/sb-runtime-governed/ that demonstrates the architectural claim of the sb-runtime integration:

The same Cedar policy produces semantically-equivalent signed receipts regardless of the sandbox layer that wraps the agent process. Auditors verify every receipt with one public key. The sandbox_backend field is inside the signature scope, so an operator cannot claim a hardened sandbox at verify time if the receipt was produced under none.

Completes the sequence proposed on #748:

#	What	Status
1	Integration doc at docs/integrations/sb-runtime.md	Merged (#1202)
2	Provider shim at packages/agentmesh-integrations/sb-runtime-skill/	Merged (#1203)
3	Worked example (this PR)	Open

What the demo does

Runs the same six actions (three allowed, three denied) across three sandbox-backend configurations using the same Cedar policy and the same operator Ed25519 key:

Scenario	sandbox_backend	Ring	Sandbox layer owner
standalone	sb_runtime_builtin	3	sb-runtime's own Landlock + seccomp
nono	nono	2	nono capability set
openshell	openshell	2	OpenShell container boundary

Then:

1. Cross-verifies all 18 receipts against the single operator public key (zero dependency on sb_runtime_agentmesh at verify time).
2. Demonstrates tamper-evidence by flipping sandbox_backend on a receipt and confirming verification fails (the field is inside the signature scope, not sidecar metadata).
3. Confirms chain linkage: receipt[1].previousReceiptHash == sha256(canonical(receipt[0])).
4. Writes the receipts + operator public key to disk under examples/sb-runtime-governed/receipts/ (gitignored) so the output is inspectable and re-verifiable with external tooling (npx @veritasacta/verify).

Files

examples/sb-runtime-governed/
├── README.md                     # walkthrough with expected output
├── getting_started.py            # ~280-line demo, three scenarios + verification + tamper
├── policies/sandbox-policy.yaml  # shared across all three scenarios
└── .gitignore                    # excludes receipts/ + __pycache__

No changes outside examples/sb-runtime-governed/.

Test plan

• python examples/sb-runtime-governed/getting_started.py exits 0
• All 18 receipts verify against the single operator public key
• Tamper test: flipping sandbox_backend fails verification
• Chain linkage: previousReceiptHash matches sha256(canonical(prior envelope))
• Receipts written to receipts/ subdirectory (gitignored), re-verifiable with npx @veritasacta/verify

Dependencies

• pyyaml>=6,<7 (same as openshell-skill)
• cryptography>=41 (for Ed25519 + JCS canonicalization)

Both are declared in the sb-runtime-skill pyproject.toml shipped in #1203. No new dependencies introduced at the AGT level.

Notes

• Mirrors the style of examples/openshell-governed/ (same action set, same three-allow-three-deny structure) so cross-example comparison is straightforward. Where openshell-governed demonstrates the policy contract alone, this example extends it with the signed-receipt layer (distinctive value-add) and the multi-backend portability claim.
• The nono scenario uses sandbox_backend="nono" as declared configuration; a future revision (coordinated with the nono Python library upstream) will wire in actual runtime enforcement. The current example demonstrates the receipt-layer portability claim, which is independent of whether a real nono runtime is doing the sandboxing.

Cc @imran-siddique (closing the three-PR sequence as committed on #748), @lukehinds (the nono scenario follows the composition pattern documented in #1202; happy to match nono's native CLI flag conventions or capability-file layout when the Python library is wired for actual runtime enforcement).

[github-actions]

Welcome to the Agent Governance Toolkit! Thanks for your first pull request.
Please ensure tests pass, code follows style (ruff check), and you have signed the CLA.
See our Contributing Guide.

[github-actions]

🤖 AI Agent: code-reviewer

Review Summary

This PR introduces a worked example demonstrating the architectural claim of sb-runtime integration: multi-backend receipt portability with signed receipts that are semantically equivalent across different sandbox layers. The example is well-documented, self-contained, and adheres to the repository's conventions. However, there are areas that require attention to ensure security, correctness, and backward compatibility.

---

🔴 CRITICAL: Security Issues

1. Sandbox Escape Vectors

   • The sandbox_backend field is included in the signed payload, which is good for tamper-evidence. However, the example does not validate whether the sandbox backend actually enforces the expected security guarantees. For example, the nono backend is currently a placeholder and does not enforce runtime sandboxing. This could lead to a false sense of security if operators rely solely on the receipt without verifying the actual sandbox behavior.
   • Actionable: Add runtime checks or attestations to ensure the sandbox backend is actively enforcing the expected security policies. Consider integrating runtime integrity checks or attestation mechanisms.

2. Cryptographic Operations

   • The Signer class is used for signing receipts, but the implementation details of Signer are not included in this PR. If Signer does not use a secure Ed25519 implementation or has vulnerabilities in its key management, the entire receipt verification process could be compromised.
   • Actionable: Audit the Signer implementation in sb-runtime-skill to ensure it uses a secure cryptographic library (e.g., cryptography) and follows best practices for key management.

3. Tampering Demonstration

   • While the tampering demonstration proves that the sandbox_backend field is inside the signature scope, it does not address potential replay attacks. An attacker could replay a valid receipt from a weaker sandbox backend (none) to bypass security checks.
   • Actionable: Include a nonce or timestamp in the receipt payload to prevent replay attacks.

---

🟡 WARNING: Potential Breaking Changes

1. Policy Format

   • The policy format in sandbox-policy.yaml uses a custom schema (governance.toolkit/v1). If this schema changes in the future, it could break backward compatibility for this example.
   • Actionable: Document the schema version explicitly in the example and provide migration instructions for future schema updates.

2. Dependency on sb-runtime-skill

   • This example depends on the sb-runtime-skill package, which is imported directly from the repository. If the package structure changes, the example will break.
   • Actionable: Pin the sb-runtime-skill version in the example's README and test plan.

---

💡 Suggestions for Improvement

1. Thread Safety

   • The example does not address thread safety in concurrent agent execution. While this is not directly relevant to the example, it is critical for real-world deployments where multiple agents may execute policies concurrently.
   • Actionable: Add a note in the README about thread safety considerations and recommend using thread-safe components in production.

2. Type Safety

   • The example does not use Pydantic models for receipt validation. This could lead to runtime errors if the receipt structure changes or contains unexpected fields.
   • Actionable: Use Pydantic models to validate receipt structures before signing and verifying.

3. OWASP Agentic Top 10 Compliance

   • The example does not explicitly address OWASP Agentic Top 10 risks, such as insufficient logging or insecure communication.
   • Actionable: Add logging for all critical operations (e.g., receipt signing, verification, tampering detection) and ensure that all communication is encrypted.

4. Documentation

   • The README is comprehensive but could benefit from a visual diagram illustrating the receipt flow across sandbox layers.
   • Actionable: Add a diagram showing the receipt lifecycle, including signing, verification, and tamper-evidence.

5. Testing

   • The test plan does not include edge cases, such as invalid policies, malformed receipts, or corrupted operator keys.
   • Actionable: Extend the test plan to include these edge cases and ensure the example handles them gracefully.

---

Final Recommendation

• Address the 🔴 CRITICAL issues before merging to ensure the example does not introduce security vulnerabilities.
• Resolve the 🟡 WARNING items to maintain backward compatibility and robustness.
• Implement the 💡 SUGGESTIONS to improve the example's usability, security, and compliance.

Once these issues are addressed, the example will be a valuable addition to the repository, demonstrating the architectural claim of sb-runtime integration effectively.

[github-actions]

🤖 AI Agent: security-scanner — Security Review for PR: `examples: sb-runtime-governed — multi-backend receipt portability`

Security Review for PR: examples: sb-runtime-governed — multi-backend receipt portability

This PR introduces a worked example demonstrating the multi-backend receipt portability of the sb-runtime integration. The example ensures that signed receipts are consistent across different sandbox backends (sb_runtime_builtin, nono, and openshell) and verifies their integrity and chain linkage. Below is a detailed security analysis based on the provided code and description.

---

Findings

1. Prompt Injection Defense Bypass

• Risk: 🔵 LOW
• Analysis: The example does not directly involve user-provided prompts or natural language inputs that could be exploited for prompt injection. The actions are predefined in the ACTIONS list, and there is no indication of user-supplied input being processed dynamically.
• Recommendation: No immediate action required. However, if user input is introduced in the future, ensure proper sanitization and validation to prevent injection attacks.

---

2. Policy Engine Circumvention

• Risk: 🟠 HIGH
• Analysis: The sandbox_backend field is included in the signed receipt payload, ensuring that the sandbox configuration cannot be tampered with post-signing. However, the nono sandbox is currently a placeholder and does not enforce runtime sandboxing. This could lead to a false sense of security for users who assume the nono sandbox is providing actual isolation.
• Recommendation:
  • Clearly document the limitations of the nono sandbox in the README and other relevant documentation.
  • Add a runtime enforcement mechanism for the nono sandbox or explicitly warn users that it is not yet implemented.

---

3. Trust Chain Weaknesses

• Risk: 🔵 LOW
• Analysis: The receipts include a previousReceiptHash field to ensure chain linkage, and the sandbox_backend field is within the signature scope. The use of Ed25519 for signing and verification is robust. The chain linkage mechanism appears sound, and there are no evident weaknesses in the trust chain.
• Recommendation: No immediate action required. Continue to ensure that the previousReceiptHash field is consistently validated during receipt verification.

---

4. Credential Exposure

• Risk: 🔵 LOW
• Analysis: The operator's private key is generated in-memory and not written to disk. The public key is written to disk for verification purposes, which is expected and safe. There is no evidence of sensitive credentials being logged or exposed.
• Recommendation: No immediate action required. Ensure that private keys are always handled securely and never exposed in logs or error messages.

---

5. Sandbox Escape

• Risk: 🟠 HIGH
• Analysis: The sandbox_backend field ensures that the sandbox configuration is auditable, but the actual sandboxing enforcement depends on the backend. The nono backend is currently a placeholder and does not enforce isolation, which could allow malicious actions to escape containment.
• Recommendation:
  • Implement runtime sandboxing for the nono backend or remove it from the example until it is functional.
  • Add automated tests to validate the isolation properties of each sandbox backend.

---

6. Deserialization Attacks

• Risk: 🟡 MEDIUM
• Analysis: The code uses json.loads and json.dumps for receipt handling, which is safe for JSON. However, the use of pyyaml for policy loading introduces a potential risk if untrusted YAML files are processed.
• Recommendation: Use yaml.safe_load instead of yaml.load to prevent arbitrary code execution during YAML deserialization.

---

7. Race Conditions

• Risk: 🔵 LOW
• Analysis: There is no evidence of time-of-check-to-time-of-use (TOCTOU) vulnerabilities in the policy checks or receipt verification. The operations appear to be atomic and deterministic.
• Recommendation: No immediate action required. Ensure that any future changes to the policy engine maintain atomicity.

---

8. Supply Chain

• Risk: 🟡 MEDIUM
• Analysis: The PR introduces dependencies on pyyaml and cryptography. While these are widely used and reputable libraries, they should be pinned to specific versions to avoid supply chain attacks or dependency confusion.
• Recommendation:
  • Pin dependencies to specific versions in the pyproject.toml or requirements.txt file.
  • Regularly audit dependencies for vulnerabilities using tools like pip-audit or safety.

---

Summary of Findings

Category	Risk	Description
Prompt Injection Defense	🔵 LOW	No user-provided input; no immediate risk.
Policy Engine Circumvention	🟠 HIGH	nono sandbox is a placeholder and does not enforce runtime isolation.
Trust Chain Weaknesses	🔵 LOW	Trust chain appears robust; no immediate issues.
Credential Exposure	🔵 LOW	No sensitive credentials are exposed.
Sandbox Escape	🟠 HIGH	Lack of runtime enforcement in nono sandbox could allow sandbox escape.
Deserialization Attacks	🟡 MEDIUM	Use of pyyaml introduces potential risks; switch to yaml.safe_load.
Race Conditions	🔵 LOW	No evidence of TOCTOU vulnerabilities.
Supply Chain	🟡 MEDIUM	Dependencies should be pinned to specific versions to mitigate supply chain risks.

---

Recommendations

1. Address nono Sandbox Placeholder:

   • Clearly document the lack of runtime enforcement in the nono sandbox.
   • Implement runtime sandboxing for nono or remove it from the example until it is functional.

2. Secure YAML Deserialization:

   • Replace yaml.load with yaml.safe_load in the policy loading code to prevent deserialization attacks.

3. Pin Dependencies:

   • Pin pyyaml and cryptography to specific versions in the dependency configuration files.
   • Regularly audit dependencies for vulnerabilities.

4. Enhance Sandbox Testing:

   • Add automated tests to validate the isolation properties of each sandbox backend to ensure they meet security expectations.

5. Monitor for Future Risks:

   • If user input is introduced in future iterations, ensure robust input validation and sanitization to prevent prompt injection attacks.

---

Final Rating: 🟠 HIGH

The primary concerns are the lack of runtime enforcement in the nono sandbox and the potential for deserialization attacks via pyyaml. Addressing these issues should be prioritized to ensure the security and reliability of the example.

[imran-siddique]

Nice work — this cleanly finishes the #748 series. One required change before merge:

License header in \getting_started.py\ — per repo convention, all new source files must use:
\\python

Copyright (c) Microsoft Corporation.

Licensed under the MIT License.

\\

Currently has # Copyright (c) 2026 Tom Farley (ScopeBlind). Please update to the Microsoft header. Everything else looks good — will merge once that's fixed.

[tomjwxf]

@imran-siddique fixed in ac669de, thanks.

[github-actions]

🤖 AI Agent: code-reviewer

Review Summary

This pull request introduces a worked example demonstrating the portability of signed receipts across multiple sandbox backends (sb_runtime_builtin, nono, and openshell) using the same Cedar policy. The example validates the architectural claim that the sandbox_backend field is included in the signature scope, ensuring tamper-evidence and consistent verification across all backends. The PR is well-documented, includes a comprehensive test plan, and does not introduce changes outside the examples/sb-runtime-governed/ directory.

Below is a detailed review of the PR, focusing on the specified areas of concern.

---

🔴 CRITICAL: Security Issues

1. Receipt Tampering Demonstration:

   • The demonstrate_tampering function modifies the sandbox_backend field in a receipt and verifies that the tampered receipt fails verification. While this is a useful demonstration, the code does not explicitly validate that the sandbox_backend field is always included in the signature scope. If this behavior is accidentally changed in the future, it could lead to a security vulnerability where the sandbox_backend field is no longer protected by the signature.
   • Recommendation: Add a unit test in the sb_runtime_agentmesh.receipts module to explicitly verify that the sandbox_backend field is always included in the signature scope. This test should fail if the field is ever excluded from the signature.

2. Receipt Chain Linkage:

   • The demonstrate_chain_linkage function verifies that the previousReceiptHash field matches the hash of the previous receipt. However, there is no explicit test to ensure that the previousReceiptHash field is always included in the signed payload.
   • Recommendation: Add a unit test in the sb_runtime_agentmesh.receipts module to ensure that the previousReceiptHash field is always included in the signature scope. This will prevent potential tampering with the receipt chain.

3. Key Management:

   • The Signer class generates a new operator key for each run of the example. While this is acceptable for a demo, it does not reflect real-world usage where operator keys are long-lived and securely stored.
   • Recommendation: Add a note in the README.md file emphasizing that in production, operator keys should be securely generated and stored using a hardware security module (HSM) or equivalent secure key management solution.

---

🟡 WARNING: Potential Breaking Changes

No breaking changes were identified in this PR. The changes are confined to the examples/sb-runtime-governed/ directory and do not affect the core functionality or public API of the library.

---

💡 Suggestions for Improvement

1. Code Readability:

   • The run_scenario function could benefit from additional comments explaining the purpose of the GovernanceSkill initialization parameters, especially for developers unfamiliar with the sb_runtime_agentmesh module.
   • Consider adding type hints for all function parameters and return types for better readability and type safety.

2. Error Handling:

   • The try block for importing sb_runtime_agentmesh.skill and sb_runtime_agentmesh.receipts raises a SystemExit on failure. While this is acceptable for a demo, it would be better to provide a more descriptive error message and suggest potential resolutions.
   • Example:

     except ImportError as exc:
         print(
             f"Error: Unable to import sb-runtime-skill. Please ensure the package is installed "
             f"and the repository layout is intact. Run:\n\n"
             f"    pip install -e packages/agentmesh-integrations/sb-runtime-skill/\n\n"
             f"Details: {exc}"
         )
         sys.exit(1)

3. Policy Validation:

   • The sandbox-policy.yaml file is well-structured, but there is no validation step in the example to ensure the policy conforms to the expected schema.
   • Recommendation: Use a Pydantic model or a JSON schema to validate the policy file before loading it into the GovernanceSkill. This will help catch any errors in the policy definition early.

4. Thread Safety:

   • The GovernanceSkill class is instantiated multiple times in the run_scenario function. If this class is used in a multithreaded context in the future, ensure that it is thread-safe, especially when accessing shared resources like the signer.

5. Documentation:

   • The README.md file is comprehensive, but it could benefit from a section explaining how to interpret the sandbox_backend field and its significance in the context of security and auditability.
   • Add a note about the importance of using a secure random number generator for key generation in the Signer class.

6. Test Coverage:

   • While the example includes a test plan, it does not integrate with the repository's pytest test suite. Consider adding automated tests to ensure the example's functionality is verified during CI/CD runs.

---

Final Assessment

The PR is well-structured and provides a valuable example of the sb-runtime integration's capabilities. However, the following critical issues must be addressed before merging:

1. Add explicit tests to ensure the sandbox_backend and previousReceiptHash fields are always included in the signature scope.
2. Emphasize secure key management practices in the documentation.

Once these issues are resolved, the PR can be safely merged. The suggested improvements can be addressed in subsequent iterations to further enhance the quality and maintainability of the code.

[github-actions]

🤖 AI Agent: code-reviewer

Review Summary

This pull request introduces a worked example demonstrating the portability of signed receipts across multiple sandbox backends using the sb-runtime integration. The example is well-documented, adheres to the architectural claims, and provides a clear demonstration of tamper-evidence, chain linkage, and cross-verification of receipts. The code is structured and aligns with the existing patterns in the repository.

Below is the detailed review, including critical issues, warnings, and suggestions.

---

🔴 CRITICAL

1. Tamper Test Validation

   • The tamper test in demonstrate_tampering relies on flipping the sandbox_backend field and verifying the signature. While the test demonstrates that the field is included in the signature scope, it does not test other potential tampering scenarios, such as modifying other fields in the payload or the signature itself. This could lead to a false sense of security.
     • Action: Extend the tamper test to include other fields in the payload (e.g., decision, policy_digest, ring) and the signature itself. Ensure that any modification invalidates the receipt.

2. Thread Safety of Signer

   • The Signer class is used to sign receipts, but its thread safety is not explicitly guaranteed. If this example is extended to a multi-threaded environment, concurrent access to the Signer instance could lead to race conditions or corrupted signatures.
     • Action: Ensure that the Signer class is thread-safe or document that it should not be shared across threads.

3. Receipt Verification

   • The verify_receipt function is used to validate receipts, but there is no explicit check for the integrity of the previousReceiptHash field during verification. This could allow an attacker to tamper with the chain linkage without detection.
     • Action: Ensure that the receipt verification process includes a check for the integrity of the previousReceiptHash field.

---

🟡 WARNING

1. Backward Compatibility

   • While this PR does not introduce changes to the core library, it relies on the sb-runtime-skill package introduced in #1203. If there are any breaking changes in the sb-runtime-skill package, it could affect this example.
     • Action: Ensure that the sb-runtime-skill package maintains backward compatibility or document the required version explicitly in the example's README.

2. Policy File Compatibility

   • The sandbox-policy.yaml file uses specific syntax and fields (e.g., field, operator, value). If the policy engine's schema changes in the future, this example may break.
     • Action: Add a note in the README to specify the version of the policy engine that this example is compatible with.

---

💡 SUGGESTIONS

1. Improved Documentation

   • The README is comprehensive, but it could benefit from a brief explanation of the sandbox_backend field and its significance in the context of the architectural claim.
     • Action: Add a short section in the README explaining the sandbox_backend field and why it is critical for auditability and security.

2. Error Handling

   • The run_scenario function uses assert statements to check for the presence of a receipt. While this is acceptable for a demo, it might be better to raise a more descriptive exception or log an error message.
     • Action: Replace assert with explicit error handling and logging to improve clarity and maintainability.

3. Policy Validation

   • The sandbox-policy.yaml file is critical to the example. However, there is no validation step to ensure the policy adheres to the expected schema.
     • Action: Add a validation step for the policy file using the policy engine's schema before running the scenarios.

4. Test Coverage

   • While the example includes a test plan, it does not integrate with the repository's pytest framework.
     • Action: Create a pytest-compatible test suite for this example to ensure it is automatically tested in the CI/CD pipeline.

5. Receipt Format Documentation

   • The draft-farley-acta-signed-receipts-02 format is referenced but not explained in detail. This could be a barrier for new users.
     • Action: Include a brief summary of the receipt format in the README or link to a detailed explanation.

6. Sandbox Backend Abstraction

   • The SandboxBackend enum is used to specify the sandbox layer. If new backends are added in the future, this example will need to be updated.
     • Action: Consider abstracting the sandbox backend configuration to make it easier to extend the example with new backends.

7. Logging

   • The example uses print statements for output. While this is acceptable for a demo, using a logging library would provide more flexibility and consistency.
     • Action: Replace print statements with Python's logging module to allow configurable log levels and better integration with other tools.

8. Dependency Management

   • The example relies on the sb-runtime-skill package being installed in editable mode. This could lead to issues if the package is not installed correctly.
     • Action: Add a check at the start of the script to verify that the sb-runtime-skill package is installed and importable.

---

Final Assessment

The PR is well-constructed and demonstrates the architectural claims effectively. However, there are critical areas that need to be addressed to ensure the example is secure, robust, and extensible. Once the identified issues are resolved, this PR will be a valuable addition to the repository.

[github-actions]

🤖 AI Agent: code-reviewer

Review Summary

This PR introduces a worked example demonstrating the portability of signed receipts across multiple sandbox backends (sb_runtime_builtin, nono, and openshell). The example validates the architectural claim that the same Cedar policy produces semantically equivalent receipts regardless of the sandbox backend, with tamper-evidence and chain linkage guarantees. The implementation is well-documented and adheres to the project's conventions.

Below are detailed observations and actionable feedback:

---

🔴 CRITICAL: Security Issues

1. Tamper-evidence scope validation:

   • The sandbox_backend field is claimed to be inside the signature scope, ensuring tamper-evidence. However, the implementation does not explicitly validate whether other critical fields (e.g., ring, policy_digest) are also protected within the signature scope. If these fields are tampered with, it could lead to security bypasses.
   • Action: Add explicit tests to verify that all critical fields (ring, policy_digest, previousReceiptHash) are covered by the signature scope. This ensures that tampering with any field invalidates the receipt.

2. Receipt chain linkage integrity:

   • The previousReceiptHash field is used for chain linkage, but there is no explicit validation in the code to ensure that the hash matches the previous receipt's canonicalized payload. This could allow an attacker to inject a fake receipt into the chain.
   • Action: Add a validation step in verify_receipt to ensure previousReceiptHash matches the hash of the prior receipt's canonicalized payload.

---

🟡 WARNING: Potential Breaking Changes

1. Dependency on sb_runtime_agentmesh skill:

   • The example assumes the sb_runtime_agentmesh skill is installed and available. While this is acceptable for the example, it introduces a tight coupling that could break if the skill's API changes in the future.
   • Action: Document the exact version of sb_runtime_agentmesh required for compatibility and consider pinning the dependency in the example's README.

2. Policy backward compatibility:

   • The example uses a specific Cedar policy format (sandbox-policy.yaml). If the policy schema evolves, this example may break.
   • Action: Add a compatibility note in the README specifying the Cedar policy schema version used.

---

💡 Suggestions for Improvement

1. Thread safety in concurrent execution:

   • The GovernanceSkill class is instantiated separately for each scenario, which avoids shared state issues. However, if the skill is reused across threads in other contexts, thread safety could become a concern.
   • Action: Add a note in the GovernanceSkill documentation clarifying its thread-safety guarantees (e.g., whether it is safe to use across threads).

2. Enhanced tamper-evidence demonstration:

   • The tamper-evidence test flips the sandbox_backend field but does not demonstrate tampering with other fields like ring or policy_digest.
   • Action: Extend the tamper-evidence test to modify additional fields and confirm verification fails.

3. Policy engine correctness:

   • The policy rules in sandbox-policy.yaml rely on regex-based matching (matches, contains). Regex-based policies can be error-prone and may lead to false negatives or positives.
   • Action: Add unit tests for the policy engine to ensure correct behavior for edge cases (e.g., partial matches, malformed actions).

4. Documentation clarity:

   • The README is comprehensive but could benefit from a visual diagram illustrating the receipt generation and verification process across sandbox backends.
   • Action: Add a simple flowchart or diagram to the README to improve clarity for new users.

5. External verifier integration:

   • The example mentions @veritasacta/verify for offline verification but does not include automated tests to confirm compatibility with this tool.
   • Action: Add a CI step to run @veritasacta/verify against the generated receipts to ensure compatibility.

---

Code Quality Observations

1. Hardcoded agent DID:

   • The agent DID (did:mesh:sb-runtime-demo-agent) is hardcoded in the example. While acceptable for a demo, this could lead to confusion if reused in production.
   • Action: Parameterize the agent DID in getting_started.py to allow customization.

2. Receipt writing logic:

   • The receipt writing logic in write_receipts_to_disk uses json.dumps with indent=2 and sort_keys=True. While this ensures readability, it may not match the canonicalization requirements for verification.
   • Action: Confirm that the written receipts match the canonicalization requirements of @veritasacta/verify.

3. Error handling:

   • The ImportError handling for sb_runtime_agentmesh.skill is clear but could benefit from a more specific error message (e.g., checking if the skill directory exists).
   • Action: Enhance the error message to include troubleshooting steps for missing dependencies.

---

Conclusion

This PR is well-structured and demonstrates the architectural claims effectively. However, critical security validations (tamper-evidence scope, chain linkage integrity) need to be strengthened to ensure robustness. Addressing these issues will enhance the security and reliability of the example. Additionally, minor improvements to documentation, testing, and error handling will improve usability and maintainability.

Recommended Actions:

• Address 🔴 CRITICAL issues immediately.
• Consider 🟡 WARNING items for long-term compatibility.
• Implement 💡 SUGGESTIONS to improve usability and robustness.

Let me know if you need further clarification or assistance!

[github-actions]

🤖 AI Agent: code-reviewer

Code Review for examples: sb-runtime-governed — multi-backend receipt portability

This PR introduces a worked example demonstrating the portability of signed receipts across different sandbox backends (sb_runtime_builtin, nono, and openshell). The example validates the architectural claim that the same Cedar policy produces semantically equivalent receipts regardless of the sandbox backend, with the sandbox_backend field included in the signature scope for tamper-evidence.

---

🔴 CRITICAL Issues

1. Insufficient Validation of sandbox_backend Field:

   • The sandbox_backend field is critical for ensuring the integrity of the receipt. While the tamper-evidence test demonstrates that modifying this field invalidates the signature, the code does not explicitly validate that the sandbox_backend value is one of the expected values (sb_runtime_builtin, nono, openshell). This could allow an attacker to inject unexpected values into the field, potentially bypassing downstream logic that relies on this field.
   • Recommendation: Add explicit validation for the sandbox_backend field in the GovernanceSkill class to ensure it matches one of the expected values.

2. Potential Replay Attack on Receipts:

   • The previousReceiptHash field is used for chain linkage, but there is no mechanism to ensure that the chain is unique or that the same receipt cannot be replayed in a different context. This could allow an attacker to replay a valid receipt in a different scenario.
   • Recommendation: Introduce a nonce or timestamp in the receipt payload to ensure uniqueness and prevent replay attacks. This should also be included in the signature scope.

3. Lack of Cryptographic Key Rotation:

   • The example uses a single operator key (Signer.generate()) for signing receipts, but there is no demonstration or mention of how key rotation would be handled. This is a critical consideration for long-term security.
   • Recommendation: Add documentation or an example demonstrating how to handle key rotation and how to verify receipts signed with both old and new keys.

---

🟡 WARNING Issues

1. Backward Compatibility:

   • While this PR does not introduce changes outside the examples/sb-runtime-governed/ directory, it relies on the sb-runtime-skill package introduced in #1203. If any changes are made to the sb-runtime-skill API in the future, this example may break.
   • Recommendation: Add a compatibility test to ensure that future changes to sb-runtime-skill do not break this example.

2. Policy Changes:

   • The sandbox-policy.yaml file is shared across all three scenarios. If this policy is updated in the future, it may affect the behavior of this example.
   • Recommendation: Consider versioning the policy file or documenting its expected behavior to ensure future changes do not unintentionally break the example.

---

💡 Suggestions for Improvement

1. Thread Safety:

   • The GovernanceSkill class is instantiated separately for each scenario, which avoids thread-safety issues in this example. However, if this class is used in a multi-threaded context in the future, ensure that the Signer and GovernanceSkill classes are thread-safe.
   • Recommendation: Add a note in the GovernanceSkill and Signer class documentation about their thread-safety guarantees.

2. Type Safety:

   • The ScenarioResult dataclass could benefit from type annotations for the receipts field to specify the expected structure of the receipt dictionaries.
   • Recommendation: Define a Pydantic model for the receipt structure and use it in the ScenarioResult dataclass to enforce type safety.

3. Error Handling:

   • The verify_receipt function is used to validate receipts, but there is no detailed error handling for cases where verification fails. This could make debugging more difficult.
   • Recommendation: Enhance the error handling in verify_receipt to provide more detailed error messages, such as whether the failure was due to an invalid signature, a missing field, or a mismatch in the sandbox_backend.

4. Documentation:

   • The README is comprehensive, but it could benefit from a section on potential security considerations, such as the importance of key rotation and the risks of replay attacks.
   • Recommendation: Add a "Security Considerations" section to the README to discuss these issues and provide guidance on mitigating them.

5. Test Coverage:

   • The example includes a # pragma: no cover comment in several places, which could lead to gaps in test coverage.
   • Recommendation: Write additional tests to cover these cases, or remove the # pragma: no cover comments if they are not necessary.

6. Policy Engine Correctness:

   • The sandbox-policy.yaml file uses regex-based conditions, which can be error-prone and may lead to false negatives or positives if the regex patterns are not carefully crafted.
   • Recommendation: Add unit tests for the policy engine to ensure that the regex patterns match the intended actions and do not introduce false negatives or positives.

7. External Dependencies:

   • The example relies on the @veritasacta/verify CLI tool for external verification. While this is a useful demonstration, it introduces an external dependency that may not always be available.
   • Recommendation: Provide an alternative Python-based verification script using the cryptography library to make the example fully self-contained.

8. Code Readability:

   • The print_verify_instructions function could be enhanced to include a brief explanation of the verification process for users who may not be familiar with the @veritasacta/verify tool.
   • Recommendation: Add comments or inline explanations in the function to make it more beginner-friendly.

---

Summary

This PR is a well-structured and comprehensive example that effectively demonstrates the architectural claims of the sb-runtime integration. However, there are critical security issues related to the validation of the sandbox_backend field, replay attack prevention, and key rotation that must be addressed. Additionally, there are some potential backward compatibility concerns and opportunities for improving type safety, error handling, and documentation.

Action Items:

1. 🔴 Add validation for the sandbox_backend field in the GovernanceSkill class.
2. 🔴 Introduce a mechanism to prevent replay attacks (e.g., nonce or timestamp in the receipt payload).
3. 🔴 Document or demonstrate key rotation for the operator key.
4. 🟡 Add compatibility tests for the sb-runtime-skill package.
5. 🟡 Consider versioning the sandbox-policy.yaml file or documenting its expected behavior.
6. 💡 Add notes about thread safety in the GovernanceSkill and Signer classes.
7. 💡 Use Pydantic models for receipt structure validation.
8. 💡 Enhance error handling in the verify_receipt function.
9. 💡 Add a "Security Considerations" section to the README.
10. 💡 Write unit tests for the policy engine's regex patterns.
11. 💡 Provide a Python-based alternative to the @veritasacta/verify tool for receipt verification.
12. 💡 Improve the readability of the print_verify_instructions function.

Once the critical issues are addressed, this PR will be ready for merging.

[imran-siddique]

Completing the 3-PR sequence from #748. sb-runtime governed example under examples/ — extension path, good tests.