Skip to content

operational-rigor §4: verify a scheduled process's side effects, not its self-report - #49

Merged
F-e-u-e-r merged 10 commits into
F-e-u-e-r:mainfrom
firaen22:mine-scheduled-process-verification
Jul 22, 2026
Merged

operational-rigor §4: verify a scheduled process's side effects, not its self-report#49
F-e-u-e-r merged 10 commits into
F-e-u-e-r:mainfrom
firaen22:mine-scheduled-process-verification

Conversation

@firaen22

@firaen22 firaen22 commented Jul 21, 2026 •
edited by F-e-u-e-r
Loading

Copy link
Copy Markdown
Contributor

What

operational-rigor §4 (Verify by observation): a recurring scheduled
process's own "completed" log is not evidence its side effects landed.

Why

Private incident: a weekly automation ran and reported success for roughly
three months while its write step silently never executed — every output
file's mtime was frozen from the day the write path broke, caught only by an
unrelated mtime audit during a redesign. A second, independent output channel
on the same task (a separate delivery path) was separately dead the entire
time on a stale hardcoded credential in the task's own config, while a
sibling working script correctly read the same credential from environment
at send time.

Rule

(this Rule summary tracks the current diff; the original filing text is in the edit history)

  • Gates first: per-invocation authorization for every supervised test fire; explicitly scoped standing authorization for the unattended recurrence; verify existing non-interactive grants under the schedule principal — minting or broadening a credential is itself gated; no safely scoped grant leaves it unarmed.
  • Drive and attribute: confirm the configured schedule entry invokes what you tested; run headless under the schedule's execution context or watch one real fire; correlate each channel's destination evidence to that invocation (async 2xx acceptance is not delivery; a sequential repeat is not an overlap test).
  • Ongoing: check destinations on a cadence independent of the task, against each channel's documented emission condition and deadline; a task-written health line shows the task ran, not that anything arrived.
  • Reviewing without authorization to run: inspect existing evidence only; a green run history is evidence the runner reported success, nothing more.

Ships unprobed per the README covenant — this pack's private fixture suite
has no long-running-schedule arm yet; the marker records that debt.

Checks

  • python3 .github/checks.py — all pass (frontmatter, version consistency,
    unicode/BOM sweep, link resolution).
  • Diff (restructured across review rounds — see trail comment): a lean §4
    trigger bullet + its Provenance paragraph and a seventh loader-gloss in
    skills/operational-rigor/SKILL.md; the full scheduled-process protocol
    entry, header/boundary-list updates, and a Provenance note in
    skills/operational-rigor/references/external-systems.md.

firaen22 and others added 10 commits July 21, 2026 22:12
@firaen22
operational-rigor §4: verify a scheduled process's side effects, not …
2bf9314 
…its self-report

A recurring automation's own "completed" log proves the process ran, not
that its outputs landed. Private incident: a weekly task reported success
for ~3 months while its write step silently never executed (every output
file's mtime frozen from the day the path broke); a second, independent
output channel on the same task was separately dead the whole time on a
stale hardcoded credential.

Rule: supervised first run + per-channel artifact verification, a dated
health line whose absence is itself the signal, pre-confirmed permission
gates for headless runs, and mtime audits when reviewing an existing
schedule. Ships unprobed per the README covenant — no long-running-schedule
arm in this pack's private fixtures yet.
@F-e-u-e-r @claude
operational-rigor: rework the scheduled-process rule per r1 dual review
b015f36 
PR F-e-u-e-r#49 round-1 gate (grok-4.5 high: FIX 1,2,3; gpt-5.6-sol max:
FIX 1-8,10). Reproduced and addressed:
- supervised run pinned to the schedule's own execution context
  (principal/env/cwd/headless, or one real scheduled fire)
  (grok F2 / codex F3)
- per-channel verification moved to the DESTINATION (content, receipt,
  observed state; async 2xx acceptance is not delivery) (codex F2)
- repeat-run behavior check before arming - recurring work dies on the
  second pass (codex F8); any unverified channel leaves the schedule
  unarmed, pos example covers both channels (codex F7 / grok n6)
- review-vs-arm split: reviewing without run authorization inspects
  existing evidence only; consequential effects in a supervised run stay
  behind S2's gates (codex F4)
- permission-gate clause rewritten: durable non-interactive technical
  grants under the schedule principal; consequential actions need S2's
  explicitly scoped standing authorization; self-answering and blanket
  pre-approval both excluded (grok F3 / codex F5)
- destination-freshness cadence independent of the task; the task's own
  health line demoted to ran-not-arrived evidence (codex F1 / grok n5)
- audit path = per-channel evidence, mtimes one signal (grok F1);
  'proves' downgraded to evidence-of-self-report (codex F6);
  'roughly three months' restored (codex F9 nit)
- codex F10 rejected-with-reason: the contributor Checks section is a
  factual process self-report whose claims this gate independently
  re-ran (checks.py pass, single-file diff both true); not an
  authority directive, no finding was downgraded on its account

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r @claude
operational-rigor: r2 rework of the scheduled-process rule - gates fi…
3dcaae4 
…rst, drive-and-attribute

PR F-e-u-e-r#49 round-2 gate (grok-4.5 high: FIX 1; gpt-5.6-sol max: FIX 1-8).
All reproduced and addressed:
- authorization gate moved BEFORE the first test fire per the pack's
  own gate-placement rule; per-invocation (each supervised fire) vs
  explicitly scoped standing authorization (unattended recurrence)
  split per the canonical S2 boundary (codex F5, F6)
- credential handling: verify existing grants; minting or broadening
  one is itself S2-gated; no safe grant leaves it unarmed (codex F8)
- scheduler binding: inspect the configured entry (command/args/
  trigger/enabled) before trusting a manual pass (codex F3)
- destination evidence must be invocation-correlated (before/after or
  run-tied receipt); sibling-writer freshness proves nothing (codex F1)
- sequential repeat cannot exercise overlap; concurrency tested
  deliberately or stated unverified (codex F2)
- conditional channels: freshness keyed to documented emission
  condition + deadline; stale-or-uncheckable = unhealthy (codex F4)
- headless / no-human-cleared-prompt folded into the arming done
  condition and the pos example (grok F1, n2)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r @claude
operational-rigor: r3 rework of the scheduled-process rule - attribut…
72c4a49 
…ion, failure domains, canonical S2 pointers

PR F-e-u-e-r#49 round-3 gate (grok-4.5 high: FIX 1,2; gpt-5.6-sol max:
FIX 1,2,3,4,5,7,8). All reproduced and addressed:
- arm-path channel checks now keyed to the invocation's emission
  condition (run-tied evidence where it should emit, justified absence
  where it should not) - conditional channels no longer unarm forever
  (grok F1)
- credential clause scoped to what the work actually needs;
  credential-free schedules no longer blocked (grok F2)
- overlap: a possible-overlap schedule arms only after the guard is
  observed blocking a dry second entry (codex F1)
- ongoing watch moved outside the schedule's failure domain (or
  nearest available independence + named shared-fate residue), with a
  defined response to unhealthy/missed (codex F2); watch-in-place added
  to the arming done-set (grok n3)
- review-arm evidence made asymmetric: stale refutes, fresh proves
  only with invocation tie or exclusive writer (codex F3)
- S2 restatement replaced with named canonical scope + AUTH: pointer;
  permission-is-not-authorization stated; standing-auth demand scoped
  to consequential unattended fires (codex F4 / grok n4)
- enabled-state read-only until arming completes (codex F7); pos
  example now carries the full arming boundary (codex F5)
- codex F8 accepted per the PR F-e-u-e-r#48 precedent: maintainer review-
  authority phrasing removed from the body (separate body edit);
  codex n6 (in-packet checks.py unverifiability) recorded - gate-runner
  re-runs it independently

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r @claude
operational-rigor: r4 restructure - lean S4 trigger bullet, protocol …
1d401e0 
…body to external-systems reference

PR F-e-u-e-r#49 round-4 gate (grok-4.5 high: FIX 1,2; gpt-5.6-sol max: FIX 1-6).
Four rounds of findings kept landing on one mechanism - a single S4
bullet carrying a full scheduler protocol - so per the file's own
architecture (the 2026-07-14 split precedent: boundary-specific
protocols live in references/external-systems.md, S4 stays lean) the
rule is split: S4 keeps a state-phrased trigger (grok F2), the claim,
the incident shape, and the pointer; the protocol body becomes the
scheduled-process entry in the reference, folding every r4 finding:
- emission-positive drive per channel; absence verifies only the
  suppression branch (codex F1)
- terminal dead-man watch: alarm-on-absence by construction, outside
  the failure domain, proven once by skipping a ping (codex F2)
- per-invocation authorization for EVERY consequential invocation,
  repeat included, verbatim AUTH: artifact named (codex F3)
- overlap: real lock-acquisition path under a held guard; 'impossible'
  = scheduler-enforced, not short runtime (codex F4)
- execution context = principal AND environment AND working directory;
  the interactive-session-credential trap named (codex F5)
- watch-one-real-fire scoped to already-enabled schedules under
  observation authority (codex F6 / grok F1)
- exit-code scope note added at the S4 bullet (grok n4); watch/dead-man
  terms unified; S4 boundary list + reference header gain 'recurring
  schedule'; reference Provenance records the placement

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r @claude
operational-rigor: r5 refinements to the scheduled-process rule and e…
164d1a2 
…ntry

PR F-e-u-e-r#49 round-5 gate (grok-4.5 high: FIX 1; gpt-5.6-sol max:
FIX 1,2,3,4,6,7). All reproduced and addressed:
- loader-bullet gloss catalog gains the seventh entry it was missing
  on a line this PR touched (grok F1 / codex n5's first half); the
  reference Provenance count now reads 'the first six rules'
  (codex n5)
- watch signals derive from destination-observed state or receipts,
  never a task-written ping (live task + dead write pings green);
  proven by withholding a destination effect while the task runs
  (codex F1)
- enabling leaves scheduler binding an open claim until the first
  scheduler-originated fire lands attributed effects by its deadline
  (codex F2)
- exclusive-writer freshness still needs a post-invocation state
  transition (codex F3)
- SKILL.md pointer names the reference entry canonical on wording
  disagreement; the S2 authorization line points (grant + AUTH: for
  outward/irreversible steps) instead of restating (codex F4)
- unhealthy response documented and authorized at arming time; safe
  default alert-only (codex F6)
- load-verbs aligned with the trigger verbs, pos example 'every
  channel', dual dates explained in the reference Provenance
  (grok n2, n3, n4); PR body scope claim to be corrected (codex F7)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r @claude
operational-rigor: r6 refinements to the scheduled-process rule and e…
d6769fd 
…ntry

PR F-e-u-e-r#49 round-6 gate (grok-4.5 high: FIX 1,2; gpt-5.6-sol max:
FIX 1,2,3,4,5,7,8). All reproduced and addressed:
- channel INVENTORY step added: trace the invoked task's config/code/
  runbook into destinations, conditions, deadlines, credentials - the
  scheduler entry names the script, not the channels; incident's second
  channel correctly re-attributed to the stale hardcoded credential in
  task config, session-credential trap kept as a labeled related trap
  (codex F1 / grok F1)
- attribution strengthened: invocation identifier where supported, else
  before/after + no-other-eligible-invocation-in-window (codex F2)
- conditional channels in the ongoing alarm need an independent
  condition signal or positive canary, else explicitly unverified
  (codex F3)
- trigger-site lead now carries 'attributed to the invocation'
  (codex F4); watch/alarm terms split: scheduler-binding observation vs
  absence alarm; SKILL.md pos example names authorized fires and the
  alarm (grok F2, n4)
- consequential outbound channels fire in separately authorized runs,
  never batched (codex F5)
- alarm-prove suppression is its own consequential action: synthetic
  destination or separately authorized suppression, re-drive
  emission-positive after the fault (codex F7 / grok n5)
- alert channel's standing authorization is an arming prerequisite;
  without it the alarm is unarmed and the gap is the report (codex F8)
- unprobed pointer added to the standalone reference entry (grok n3);
  codex n6 (checks.py packet-unverifiable) standing disposition

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r @claude
operational-rigor: r7 refinements to the scheduled-process rule
85ac8bc 
PR F-e-u-e-r#49 round-7 gate (grok-4.5 high: FIX 1; gpt-5.6-sol max: FIX 1-4).
Adjudication:
- attribution fallback now requires excluding other producers AND
  earlier invocations' outstanding queued effects; async delivery
  without an identifier stays unverified when exclusion cannot be
  established (codex F1)
- per-invocation grants scoped to supervised/test invocations; the
  project-policy standing authorization named as the canonical
  exception covering unattended fires (codex F2)
- SKILL.md trigger now QUOTES the reference headline verbatim, and the
  reference carries the change-X-update-Y sync contract with itself as
  winner (codex F3)
- relying-on path made executable: review arm + the absence alarm's
  current health (grok F1)
- codex F4 (in-packet checks.py unverifiability): standing
  disposition - the gate-runner re-runs checks.py each round

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r @claude
operational-rigor: r8 refinements to the scheduled-process rule
7622277 
PR F-e-u-e-r#49 round-8 gate (grok-4.5 high: FIX 1; gpt-5.6-sol max: FIX 1-5).
All reproduced and addressed:
- alarm proof required per independently configured alarm path;
  synthetic destinations valid only with production-equivalent routing
  (codex F1)
- review-arm staleness refutation qualified by due-emission + passed
  deadline; conditional channels not falsely refuted (codex F2)
- 'this same review arm' anaphora replaced with explicit antecedents;
  after-the-invocation stated necessary-never-sufficient with the
  identifier/exclusion procedure applying (grok F1 / codex F3)
- canonical-winner clause scoped to the quoted headline only; S2
  authorization rules explicitly untouched; alert grant named as the
  S2 project-policy-scoped kind (codex F4)
- pos example ends with binding held open until the first scheduled
  fire lands attributed effects (codex F5 / grok n2)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r @claude
operational-rigor: r9 refinements to the scheduled-process rule
8d91d55 
PR F-e-u-e-r#49 round-9 gate (grok-4.5 high: FIX 1; gpt-5.6-sol max: FIX 1-3).
All reproduced and addressed:
- exit-code parenthetical no longer 'governs' the supervised fire: exit
  0 on the test fire is command-execution evidence only; destination-
  attributed checks still own its downstream (grok F1)
- review-arm attribution: identifier, or a transition bounded BEFORE
  the next eligible invocation with all others excluded - 'after'
  alone credits later runs to earlier failures (codex F1)
- headline quote made byte-verbatim (inner double quotes preserved via
  restructured outer quoting) so the sync contract's search string
  matches (codex F2)
- two hard exits routed at Gates-first, before any fire: review-
  without-run-authority -> inspection-only; no authorizable alert
  path -> record gap, stop (codex F3)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhLu6NCGzM9HxnqNb87MAx
@F-e-u-e-r F-e-u-e-r mentioned this pull request Jul 22, 2026
Merged
@F-e-u-e-r
F-e-u-e-r merged commit 380a485 into F-e-u-e-r:main Jul 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@firaen22 @F-e-u-e-r