Skip to content

fix(benchmarks): define fail-closed production trust boundary - #1226

Merged
aimasteracc merged 2 commits into
developfrom
fix/no1-002d-trust-contract
Aug 5, 2026
Merged

fix(benchmarks): define fail-closed production trust boundary#1226
aimasteracc merged 2 commits into
developfrom
fix/no1-002d-trust-contract

Conversation

@aimasteracc

Copy link
Copy Markdown
Owner

Summary

  • add the NO1-002D operator-controlled production run specification and trust configuration boundary
  • reject bundle-controlled trust material, expired specs, pre-created artifact roots, and incomplete external trust roles
  • keep production status NOT_EVALUATED and model callbacks disabled even when local configuration is complete, until signed attestations and an independent Judge verdict exist

Verification

  • uv run --frozen pytest tests/unit/test_production_trust.py -q (5 passed)
  • uv run --frozen pytest tests/unit/test_production_trust.py tests/unit/test_benchmark_harness.py -q (354 passed)
  • uv run --frozen pytest -q (1318 passed, 1 skipped)
  • patch coverage gate: no added executable misses
  • TSA health: A / 94.3; change impact: REVIEW, low risk

Refs #1223
Refs #1221
Refs #1195

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

🔍 Dogfood PR Check

Layer Status
Change Impact ✅ passed
Claim Invariant Suite ✅ passed

Change impact: change_impact changed=3 risk=low pytest_required=True
Verification: uv run pytest -q
Claim summary: ====================== 17 passed, 12 deselected in 1.64s =======================

Claim invariant test output (last 30 lines)
============================= test session starts ==============================
collected 29 items / 12 deselected / 17 selected

tests/benchmarks/claims/test_bm25_ranking_invariant.py ....              [ 23%]
tests/benchmarks/claims/test_reactive_push_e2e.py .......                [ 64%]
tests/benchmarks/claims/test_safety_layers_smoke.py ......               [100%]

- generated xml file: /home/runner/work/tree-sitter-analyzer/tree-sitter-analyzer/dogfood-claims.xml -
====================== 17 passed, 12 deselected in 1.64s =======================

What these tests guard:

  • 390× cross-language mis-wire claim
  • BM25 ranked search with relevance_score
  • 5 safety layers (safe/guard/constraints/impact/verdicts)
  • Reactive push subscribe/unsubscribe (RFC-0001)
  • Index build speed regression

Full results: workflow run


Dogfood PR Check — docs/superpowers/specs/2026-07-08-svcd-dogfood-system-design.md

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 552945a4db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchmarks/codegraph_compare/production_trust.py Outdated
Comment thread benchmarks/codegraph_compare/production_trust.py Outdated
Comment thread benchmarks/codegraph_compare/production_trust.py Outdated
Comment thread benchmarks/codegraph_compare/production_trust.py Outdated
Comment thread benchmarks/codegraph_compare/production_trust.py Outdated
@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@aimasteracc
aimasteracc merged commit 73436b1 into develop Aug 5, 2026
23 checks passed
@aimasteracc
aimasteracc deleted the fix/no1-002d-trust-contract branch August 5, 2026 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant