diff --git a/skills/operational-rigor/SKILL.md b/skills/operational-rigor/SKILL.md index 82300cc..03019a9 100644 --- a/skills/operational-rigor/SKILL.md +++ b/skills/operational-rigor/SKILL.md @@ -255,6 +255,37 @@ When rigor conflicts with finishing sooner, rigor wins. - Between failed fixes, return to a clean state; stacked half-fixes hide causes. - Reproduce reported bugs before fixing. Fix the observed failure, not the implied one. Refutation is valid: report confirmed non-bugs and ship nothing. +- **A check's name is not its coverage** (`unprobed` — private incident as + shape; see Provenance). A named gate earns evidentiary weight from what + it asserts AND what it actually drives: one session cited a check whose + name implied it gated a model integration's behavior, then read its + source and found it exercised only a regex pre-filter in which the + model's name was a routing label — and had to correct a safety claim + already given to the user. Before citing a check, test, or CI job as + evidence a change is safe, trace it through to its pass/fail oracle — + the assertions (or, for a linter or build job, its rule set and + inputs) inspected at the revision the cited run actually used, the + invocation path and setup that feed them, whether that path executed + in the cited run, and whether its assertions PASSED there with their + failure controlling the check's final status (a run is not a pass — + the runs/passes/correct line later in this section) — and assert only the properties + that trace established: whatever the check's NAME implies but the + trace did not show stays unverified — say so; two checks with + identical assertions differ when one drives the real integration and + the other a pre-filter. A trace you cannot inspect leaves that + coverage unverified — say so. "There is a check called X" is a claim + about naming, not behavior. + ✅ "traced check X at run 1234's revision: it asserts A and B against + the real adapter; the run's log shows that path executed and A, B + passed with failures propagating to the job status; nothing in its + path drives C — C is unverified." + ❌ "the change is safe, check X covers it" (named, never read). + ❌ "read the source — it asserts A — so the cited run covers A" (the + run had that test conditionally skipped; static coverage is not the + cited run's coverage). + ❌ "read it — it's a regex pre-filter, but the name says integration, + so the integration is covered" — a trace read and then overridden by + the name. - **A failing check has two suspects: the code and the check itself.** Before editing either, open the statement of intended behavior (spec, README, docstring, type) and confirm which side it backs; a disagreement is the @@ -538,6 +569,15 @@ the deployed path (contributor-reported shape; the private repo is verifiable by the contributor, not linkable here). It ships `unprobed` — the pack's private fixtures have no interactive arm to drive it (cf. the grill-pass note above); the marker records that debt, not an exemption. +The §4 check-name rule (2026-07-22) comes from a private incident: a +session presented a named CI check as gating a model integration's +behavior, then read the check's source and found it exercised only a +regex pre-filter in which the model's name was merely a routing label, +and had to correct the safety framing it had already given the user. +Private evidence, cited as shape per the README covenant's second branch; +the executable probe — sample a repo's named checks and diff name-implied +vs actual assertion coverage — has not been run; the in-body `unprobed` +marker records that debt. Stable behavioral rules; the environment-specific facts to re-verify now travel with the rules that cite them — the external-systems set in `references/external-systems.md`, plus §2's mount-check commands