external validation: reproduce Causal Equilibrium benchmark

[safal207]

Context

CML now includes an experimental Causal Equilibrium Point (#152). Issues #153 and #154 define a portable fixture set and deterministic benchmark runner.

The strongest next trust signal is independent reproduction on a clean checkout.

Dependencies:

• benchmark: add Causal Equilibrium conformance fixtures #153 — conformance fixtures
• benchmark: add machine-readable Causal Equilibrium runner and reports #154 — machine-readable runner and reports

Goal

Have an external contributor, reviewer, or researcher run the documented equilibrium benchmark without changing semantics and submit a short validation note with the generated report digest.

Suggested validation note path:

docs/evidence/external_validation/YYYY-MM-DD-equilibrium-reproduction.md

Reproduction procedure

From a clean checkout:

1. record operating system, Python version, and tested commit;
2. install the repository using the documented dependency path;
3. run the equilibrium fixture tests;
4. run the benchmark command from benchmark: add machine-readable Causal Equilibrium runner and reports #154;
5. compute SHA-256 for the generated JSON report;
6. compare total/pass/fail counts and per-fixture outputs with the repository baseline;
7. record any ambiguity, failure, or undocumented setup step.

Validation note template

Validator:
Date:
Environment:
Repository commit:
Fixture schema version:
Command:
Total / passed / failed:
JSON report SHA-256:
Result: PASS | PARTIAL | FAIL
Observed differences:

Acceptance criteria

• Validation is performed by someone other than the primary author account.
• The tested commit and environment are explicit.
• The generated JSON report digest is included.
• Any mismatch is reported rather than normalized away.
• A PASS means the published fixture expectations were reproduced exactly.
• The note clearly preserves the experimental and non-normative status.

Requested semantic review

In addition to mechanical reproduction, feedback is especially useful on:

• whether recalled_memory_refs adequately represents declared cross-session influence;
• whether provenance loss during consolidation should remain FAIL;
• whether missing required counterevidence should remain INDETERMINATE rather than UNSTABLE;
• whether finding ordering and report fields are sufficient for another implementation.

Non-goals

• Do not claim official CrewAI adoption or endorsement.
• Do not claim safety, fairness, compliance, or decision correctness.
• Do not change fixtures merely to obtain a PASS.
• Do not require hosted services, private credentials, or production deployment.

Why this matters

A successful result would establish:

same versioned fixtures
→ same deterministic states and findings
→ same machine-readable report
→ independently reproducible equilibrium semantics

[safal207]

The dependency chain is now complete on main:

• v1 fixtures: benchmark: add Causal Equilibrium v1 conformance fixtures #160
• deterministic runner + reports: benchmark: add deterministic Causal Equilibrium runner #161
• current merge commit: df7b5af3ad67a9b266c05fb3ad0421d601b0116d

Reproduction command:

python benchmarks/equilibrium/run.py \
  --fixtures benchmarks/equilibrium/v1/fixtures.json \
  --json-out /tmp/equilibrium-report.json \
  --markdown-out /tmp/equilibrium-report.md \
  --implementation-commit df7b5af3ad67a9b266c05fb3ad0421d601b0116d

Expected aggregate result:

total: 10
passed: 10
failed: 0
state_counts:
  BALANCED: 2
  UNSTABLE: 3
  INDETERMINATE: 5
finding_code_counts:
  CML-EQ-01-MISSING_COUNTEREVIDENCE: 1
  CML-EQ-02-UNRESOLVED_MEMORY_INFLUENCE: 2
  CML-EQ-03-CONSOLIDATION_IMBALANCE: 2
  CML-EQ-04-INDETERMINATE_STATE: 6

The PR CI independently executed the runner and uploaded the JSON/Markdown report artifact. The artifact archive digest was:

sha256:58263fbebd3b8aae01c497c04bd9c2426c8abf94664eb89637f4db52f295d076

That digest is for the uploaded ZIP archive, not the inner JSON report. The external validation note should record the runner-printed JSON SHA-256 separately.

Issue #155 is now ready for an independent clean-checkout reproduction.

[safal207]

@marcotannoia — you previously gave us a clean macOS ARM64 / Python validation on #30, so you are the strongest current candidate for this independent reproduction. Would you be willing to run the published command from a clean checkout and post the generated JSON SHA-256 plus the short validation note here?

@giskard09 — you already reviewed the portability contract and deterministic ordering. A clean second implementation or independent runner reproduction from your environment would also be highly valuable.

Requested scope is intentionally narrow:

1. checkout commit df7b5af3ad67a9b266c05fb3ad0421d601b0116d;
2. run the documented fixture tests and benchmark command;
3. report environment, 10 / 10 result or any mismatch, and the runner-printed JSON SHA-256;
4. do not change fixtures or evaluator semantics to obtain a PASS.

No policy, safety, fairness, or adoption claim is being requested — only reproducibility of the experimental audit contract.

[giskard09]

Runner looks clean — the portable contract makes reproduction straightforward.

For cross-spec work on our end, the verification path is against the action_ref outputs in the fixture chain: if the conformance runner produces action_ref fields, those can be independently verified against the canonical vectors in examples/conformance/cml/. The causal equilibrium semantics themselves are yours to define.

One addition since we last spoke: we published a formal integration protocol for SDKs that want to declare Mycelium as their verifiability backend — docs/mycelium-provider-protocol.md in argentum-core. CML already satisfies the conformance requirement. If you want to add the declaration to the CML README, the text is in the doc.