fix: grade an absent rehearsal as STALE, not RED (#144)

Heart reported RED `release validation FAILED` against a Release Integrate
run that was entirely green: 661 passed, 0 failed, `failures: []`.

The tick's auto-ingest (`release_run.py`) folds the latest release-integrate
run's stage report on its own, with no merge base. That artifact is
integrate-only — `to_stage_report()` emits a single top-level `stage` — so it
can never carry a `rehearse` stage, and `_Accumulator.release_ready()` requires
one. `release_ready` was therefore false by construction on this path however
green the run was, and `readiness.py` mapped every false to the RED axis.

One boolean cannot answer both "did anything fail?" and "is the evidence
complete?". Add `validation_outcome: pass|fail|incomplete` and read that for
severity; `release_ready` is unchanged beside it for compatibility.

- validate.py: emit `validation_outcome`. `fail` covers any adverse evidence —
  a failed stage, positive `totals.failed`/`totals.timeout`, or a non-empty
  `failures` list — not just a stage literally marked "fail", which
  `release_ready()` alone tested. Also apply `_norm_status` in `add_report`,
  which previously copied stage statuses verbatim, so a merged report's
  "failure" never normalised and read as not-a-failure.
- readiness.py: `fail` -> RED (unchanged), `incomplete` -> STALE. A report
  predating the field falls back to `release_ready` and stays RED — fail closed.
  The STALE text keeps the literal "release validation" so the Health Agent
  classifier routes it to the `validate` capability instead of `unknown`.
- dashboard.py: render `incomplete` as WARN, not a FAIL row beside a stale
  header; add `stale_reasons` to `to_dict()`, the surface the Health Agent and
  mobile read, so a reason moved onto the stale axis is re-reported rather than
  disappearing.
- release_run.py: stop calling every non-true ingest "FAILED", and re-fold once
  when the stored report predates the discriminator — otherwise the RED would
  persist behind the run-id cache. That migration is skipped when the report
  already holds a `rehearse` stage, which this check's artifact cannot
  reproduce, so a release drive's manual multi-stage ingest is never regressed.

Verified end-to-end against the real state and the real artifact, with no CI
re-dispatched: RED 45 / `release validation FAILED` -> YELLOW 70 /
`release validation incomplete: no rehearsal for current source`, red_reasons
empty. The remaining yellows are the pre-existing workspace-validation and
manifest-drift ones.

Regression tests pin the fail-closed cases a naive "no stage says fail" test
would have softened: positive failed counts, positive timeout counts, a
non-empty failures list, a "failure" status synonym arriving via `add_report`,
and a legacy report with no discriminator. The pre-existing
`test_validation_failed_is_red` is unmodified and serves as the control.

Diagnosis and plan were cross-reviewed by Codex, which rejected an earlier
readiness-only fix; its load-bearing findings were re-verified against the code
before being adopted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ENfMeYpRfAifDQ63uheMbV

Author: 2 people

docs/release_validation.md

@@ -138,7 +138,10 @@ a failing local artifact remains RED.
 `heart/validate.py` records `profile` and per-repo `commit_shas` in
 `validation_report.json`; `heart/readiness.py` then requires, for GREEN:
 
-- `release_ready == true` (no stage failed — else RED),
+- `validation_outcome == pass` (else RED for `fail`; **STALE** for `incomplete`,
+  which means nothing failed and the rehearsal evidence is simply absent — a
+  report predating the field falls back to `release_ready == false` → RED, so
+  the gate fails closed on evidence it cannot classify),
 - `profile == release` (else YELLOW — a smoke-fidelity run is not a release gate),
 - `commit_shas` matching the current `main` HEADs (else YELLOW — stale source),
 - freshness (a rehearsal older than `VALIDATION_STALE_DAYS` is YELLOW).

health_agent/capabilities.yaml

@@ -128,14 +128,21 @@ release_validation:
       `release-stage-report` artifact emitted by the release-integrate.yml
       channel — collected and handed over by the Brain Release Agent.
     produces: >-
-      validation_report.json (schema_version 1): release_ready, testpypi_version,
-      profile, per-repo commit_shas, per-stage status, per-project counts,
-      failures, run URLs, timestamp — persisted in Heart state + a
-      validation_history/ archive.
+      validation_report.json (schema_version 1): validation_outcome
+      (pass|fail|incomplete), release_ready (legacy boolean, kept for
+      compatibility), testpypi_version, profile, per-repo commit_shas, per-stage
+      status, per-project counts, failures, run URLs, timestamp — persisted in
+      Heart state + a validation_history/ archive.
     gate_role: >-
-      HARD readiness gate. release_ready==false → RED; absent / stale-by-age /
-      commit_shas not matching current main HEADs / profile != release → YELLOW;
-      fresh pass matching current source under the release profile → GREEN.
+      HARD readiness gate. Read validation_outcome, NOT release_ready — the
+      boolean collapses "a stage failed" and "no rehearsal evidence" into one
+      false, and the tick's integrate-only auto-ingest always lands on that
+      false however green the run was. validation_outcome==fail → RED;
+      ==incomplete → STALE (an evidence gap, nothing known-bad); absent /
+      stale-by-age / commit_shas not matching current main HEADs / profile !=
+      release → YELLOW; fresh pass matching current source under the release
+      profile → GREEN. A report predating validation_outcome falls back to
+      release_ready==false → RED (fail closed).
     orchestrated_by: "PyAutoBrain Release Agent (agents/release/) — dispatch/poll/download"
     boundary: "ingest-and-judge only; never dispatches release.yml or workspace-validation.yml"
 

heart/checks/release_run.py

@@ -16,11 +16,17 @@
 
 - a fresher local ingest is never regressed (report ts vs run creation time);
 - a run already ingested (sidecar-cached id) is never re-downloaded;
-- a FAILED rehearsal ingests too: ``release_ready: false`` is evidence, not an
-  evidence gap — readiness then shows the accurate ``release validation
+- a FAILED rehearsal ingests too: ``validation_outcome: "fail"`` is evidence,
+  not an evidence gap — readiness then shows the accurate ``release validation
   FAILED (stage integrate)`` instead of week-old STALE, and it self-clears on
   the next green night.
 
+The artifact this check downloads is an **integrate-only** stage report, so the
+folded report can never carry a ``rehearse`` stage and its ``release_ready`` is
+``false`` by construction — however green the run was. That is why severity is
+read from ``validation_outcome`` (``incomplete``, an evidence gap → STALE) and
+not from the boolean, which would report a failure that never happened.
+
 ``decide()`` is pure and no-network: the gh-backed callables are injected only
 by the ``main()`` tick/CLI entrypoint (the #83/#120 discipline).
 """
@@ -119,6 +125,12 @@ def decide(
 
     Actions: no-runs · in-progress · cached (already ingested) ·
     local-fresher (never regress a newer local ingest) · ingest.
+
+    A report that predates ``validation_outcome`` is re-ingested once even when
+    the run id is cached: without that, a report already folded by the old code
+    would keep its missing discriminator forever, and the readiness gate — which
+    fails closed on reports it cannot classify — would stay RED until some
+    unrelated future run happened to come along.
     """
     if not run_record:
         return {"action": "no-runs"}
@@ -131,15 +143,46 @@ def decide(
     }
     if run_record.get("status") != "completed":
         return {**out, "action": "in-progress"}
-    if isinstance(sidecar, dict) and sidecar.get("last_ingested_run_id") == run_id:
-        return {**out, "action": "cached"}
-    report_ts = _parse_ts((current_report or {}).get("ts"))
-    created = _parse_ts(out["created"])
-    if report_ts is not None and created is not None and report_ts >= created:
-        return {**out, "action": "local-fresher"}
+    # A one-time re-fold for reports written before `validation_outcome` existed.
+    #
+    # Skipped when the current report already carries a `rehearse` stage: that is
+    # evidence this check's integrate-only artifact cannot reproduce (it comes
+    # from a manual multi-stage ingest during a release drive), so re-folding
+    # would throw it away and turn a `pass` into an `incomplete`. Note the test
+    # is the rehearsal, NOT `local-fresher` — every ingest necessarily happens
+    # after the run it ingests, so a report being "fresher than the run" says
+    # nothing about where it came from.
+    stages = (current_report or {}).get("stages")
+    has_rehearsal = isinstance(stages, dict) and "rehearse" in stages
+    stale_schema = (
+        isinstance(current_report, dict)
+        and bool(current_report)
+        and not has_rehearsal
+        and current_report.get("validation_outcome") not in ("pass", "fail", "incomplete")
+    )
+    if not stale_schema:
+        if isinstance(sidecar, dict) and sidecar.get("last_ingested_run_id") == run_id:
+            return {**out, "action": "cached"}
+        report_ts = _parse_ts((current_report or {}).get("ts"))
+        created = _parse_ts(out["created"])
+        if report_ts is not None and created is not None and report_ts >= created:
+            return {**out, "action": "local-fresher"}
     return {**out, "action": "ingest"}
 
 
+def resolve_outcome(ingested: dict[str, Any] | None) -> str:
+    """``pass`` | ``fail`` | ``incomplete`` for an ingested report.
+
+    Pure, like ``decide()``, so the tick's wording is testable without the
+    network. Reports predating ``validation_outcome`` fall back to the legacy
+    boolean and fail closed.
+    """
+    outcome = (ingested or {}).get("validation_outcome")
+    if outcome in ("pass", "fail", "incomplete"):
+        return str(outcome)
+    return "pass" if (ingested or {}).get("release_ready") is True else "fail"
+
+
 def main(argv: list[str] | None = None) -> int:
     sys.path.insert(0, str(HEART_HOME))
     from heart import state, validate
@@ -160,16 +203,24 @@ def main(argv: list[str] | None = None) -> int:
                     "last_ingested_run_id": decision.get("run_id"),
                     "ingested_ts": ingested.get("ts"),
                     "release_ready": ingested.get("release_ready"),
+                    "validation_outcome": ingested.get("validation_outcome"),
                     "run_url": decision.get("url"),
                 })
 
     from heart.heart_color import c_fail, c_info, c_meta, c_ok, c_warn, glyph_fail, glyph_ok, glyph_warn
 
     label_id = decision.get("run_id", "?")
     if action == "ingest":
-        ready = (ingested or {}).get("release_ready")
-        if ready is True:
+        outcome = resolve_outcome(ingested)
+        if outcome == "pass":
             glyph, label = glyph_ok(), c_ok(f"rehearsal ingested (run {label_id}: pass)")
+        elif outcome == "incomplete":
+            # This is the ordinary state for this path: the artifact is an
+            # integrate-only stage report, so it carries no rehearsal evidence.
+            # Nothing failed — do not say FAILED.
+            glyph, label = glyph_warn(), c_warn(
+                f"integrate ingested (run {label_id}: no rehearsal evidence)"
+            )
         else:
             glyph, label = glyph_fail(), c_fail(f"rehearsal ingested (run {label_id}: FAILED)")
     elif action in ("cached", "local-fresher"):

heart/dashboard.py

@@ -571,9 +571,18 @@ def build_board(
         profile = vr.get("profile") or "?"
         stages = vr.get("stages") or {}
         meta = f"v{ver}  profile={profile}  ({vr.get('ts', '?')})"
-        if ready is False:
+        # Mirror the readiness gate: read `validation_outcome`, falling back to
+        # the legacy boolean. `incomplete` is an evidence gap (WARN), not a
+        # failure (FAIL) — a green integrate-only ingest lands there, and a FAIL
+        # row beside a stale header verdict reads as a broken release.
+        outcome = vr.get("validation_outcome")
+        if outcome not in ("pass", "fail", "incomplete"):
+            outcome = "fail" if ready is False else ("pass" if ready is True else None)
+        if outcome == "fail":
             st, summary = FAIL, f"NOT release_ready — {meta}"
-        elif ready is True:
+        elif outcome == "incomplete":
+            st, summary = WARN, f"incomplete — no rehearsal evidence — {meta}"
+        elif outcome == "pass":
             st, summary = OK, f"release_ready — {meta}"
         else:
             st, summary = WARN, f"release_ready unknown — {meta}"
@@ -798,6 +807,10 @@ def to_dict(board: Board) -> dict[str, Any]:
         "stale": board.stale,
         "red_reasons": board.red_reasons,
         "yellow_reasons": board.yellow_reasons,
+        # Evidence gaps belong on this surface too: the Health Agent and mobile
+        # read it, and a reason that moves from the red axis to the stale one
+        # would otherwise vanish from both rather than being re-classified.
+        "stale_reasons": board.stale_reasons,
         "pages_url": PAGES_URL,
         "sections": [
             {

heart/readiness.py

@@ -16,8 +16,10 @@
   exceeds the newest released version of its library (UNSATISFIABLE — no
   installable release can satisfy it), or an unparseable (BAD) floor/tag; or the
   deep install verification last reported ``ready == false``; or the
-  release-validation report last ingested reports ``release_ready == false`` (a
-  stage failed).
+  release-validation report last ingested reports ``validation_outcome ==
+  "fail"`` (a stage failed, or failing/timed-out counts were recorded). A report
+  predating that field falls back to ``release_ready == false`` → RED, so the
+  gate fails closed on evidence it cannot classify.
 - **YELLOW** (caution) for soft signals: workspace-validation not passing (the
   workspace scripts/notebooks carry standing debt, so this is advisory — never a
   hard block), script-timing regressions, stale open PRs, stale parked scripts, a
@@ -30,7 +32,9 @@
 - **STALE** (an evidence gap, the freshness tier) when nothing is known-bad but
   some evidence is *missing or expired*: a check that was never run, a
   passing-but-aged report, a rehearsal whose ``commit_shas`` no longer match
-  ``main``, an unknown repo/version status. The remedy for a stale reason is to
+  ``main``, a validation report whose ``validation_outcome`` is ``incomplete``
+  (nothing failed; the rehearsal evidence is absent), an unknown repo/version
+  status. The remedy for a stale reason is to
   **re-run the check**, never to fix code — which is exactly what separates it
   from yellow. The tier is not a skip lever: evidence whose *last known result
   was adverse* stays yellow/red until a fresh run says otherwise; only
@@ -453,13 +457,29 @@ def scope_local(msg: str, key: str) -> None:
     # This is the M2 gate: the report proves the exact source about to ship was
     # built, published to TestPyPI, installed from the wheel, and exercised at
     # release fidelity. Absent/stale/source-not-matching → YELLOW ("no release
-    # rehearsal for current source"); failing → RED. Pass/fail (release_ready)
-    # is the RED axis; fidelity+freshness (profile / commit_shas / age) is the
-    # YELLOW axis — a passing-but-stale report is a caution, not a blocker.
+    # rehearsal for current source"); failing → RED. `validation_outcome` is the
+    # RED axis; fidelity+freshness (profile / commit_shas / age) is the YELLOW
+    # axis — a passing-but-stale report is a caution, not a blocker.
+    #
+    # Read `validation_outcome`, NOT `release_ready`: the boolean collapses
+    # "something failed" and "the rehearsal evidence is missing" into one
+    # `false`, and the tick's integrate-only auto-ingest always lands on that
+    # `false` (see heart/validate.py). Grading it RED reported a failure that
+    # had not happened. A report predating the field carries no discriminator,
+    # so `false` there stays RED — fail closed.
     vr = snapshot.get("validation_report")
     if isinstance(vr, dict) and vr:
         ready = vr.get("release_ready")
-        if ready is False:
+        outcome = vr.get("validation_outcome")
+        if outcome not in ("pass", "fail", "incomplete"):
+            outcome = "fail" if ready is False else ("pass" if ready is True else None)
+        if outcome == "incomplete":
+            # Nothing is wrong; the rehearsal evidence is simply absent. The
+            # wording must contain "release validation" — the Health Agent
+            # classifier matches on that string to route the remedy.
+            stale.append("release validation incomplete: no rehearsal for current source")
+            hit("validation_absent")
+        elif outcome == "fail":
             failed_stages = [
                 n for n, s in (vr.get("stages") or {}).items()
                 if isinstance(s, dict) and s.get("status") == "fail"
@@ -469,7 +489,7 @@ def scope_local(msg: str, key: str) -> None:
                 + (f" (stage {', '.join(failed_stages)})" if failed_stages else "")
             )
             hit("validation_failed")
-        elif ready is True:
+        elif outcome == "pass":
             commit_shas = vr.get("commit_shas") or {}
             mismatched: list[str] = []
             unconfirmed: list[str] = []