Add opt-in cadence flags to runpane watch and fix STUCK false positives

[parsakhaz]

Summary

Makes runpane watch --follow cheap for an orchestrator to sit on, without changing what the CLI does by default.

• runpane watch keeps its current defaults in every mode: no settle, all kinds, no batching, IDLE every --idle-after, HEARTBEAT every 60 s under --follow. Panes and shell users still see events immediately.

• Four new opt-in flags, implemented in the daemon so every client benefits and both wrappers (npm and pip) stay thin:

  • --settle <ms>: a READY is emitted only if the panel is still idle after the window. A BUSY, BLOCKED, UNKNOWN, EXIT, or GONE for that panel inside the window cancels it silently. The emitted READY carries settledMs.
  • --blocked-settle <ms>: the same for BLOCKED, so a prompt answered in-pane within seconds does not wake anyone.
  • --min-interval <ms>: non-urgent lines (READY, IDLE, NEW, GONE, EXIT, UNKNOWN) are held and flushed together at most once per interval. BLOCKED bypasses the interval and carries the held lines with it.
  • --idle-backoff: IDLE fires at --idle-after, then 30 m, 1 h, 3 h, then daily, per panel, reset on any activity.
  • The flags require --follow. --settle, --blocked-settle, and --min-interval need a named consumer (--as or PANE_PANEL_ID); a follower without one is auto-named follow-<pid>, and combining them with --since is rejected. --self-test sends no consumer name, so the daemon applies no cadence.

• STUCK fix in the daemon: the journal now derives held input with the placeholder-aware helper the IDLE path already used, so the Claude Code prompt suggestion (> Try "fix the bug") no longer counts as held input on READY. Real undelivered composer text still does.

• The Pane Chat orchestrator skill (Liveness Contract, emitted for both Codex and Claude) is the expensive consumer, so the conservative values live there. It now arms:

  runpane watch --self-test
  runpane watch --follow --kinds agent.ready,agent.blocked,agent.idle,panel.exited,pane.gone --settle 180000 --blocked-settle 30000 --min-interval 600000 --idle-backoff
  

  and states the budget (about 6 wake-ups per active pane per hour worst case, usually 1 to 3, overnight runs must not burn the usage cap), explains that a /do pane's status flips while waiting on subagents are the false wake-ups being suppressed, says BUSY is not requested and carries no action, and keeps HEARTBEAT filtered out of the orchestrator monitor. Because HEARTBEAT is filtered, the dead-watch rule now judges death by a non-zero exit or a WATCH ERROR line, not by silence. The merged unattended-resilience text from Add opt-in unattended resilience to the Pane Chat orchestrator skill #609 is reconciled with this: its BUSY clause is gone and its re-arm bullets use the same rule.

• runpane doctor prints the effective --follow watch defaults. runpane agent-context --command watch --json documents the new flags.

Why (evidence from 2026-09-13)

Pane Chat runs a harness monitor on runpane watch --follow. Every emitted line is a wake-up that re-sends the orchestrator's whole context (150k to 250k tokens, mostly cache reads). Last night the orchestrator made 192 API calls and read 20M cached tokens in 90 minutes, and most wake-ups were useless:

• A Claude pane whose main loop waits on a background subagent flips idle/working every few seconds. The journal emitted READY, BUSY, READY, BUSY, READY, BUSY for one pane inside two minutes. Each READY also produced a STUCK line because the grey prompt suggestion counted as held input.
• IDLE repeated every 10 minutes forever for panes idle for two days (seo-foundations-2: 2770m, 2830m, 2840m...).
• Real turn ends, measured from usage_events gaps over 2 min across all worktree panes since Sep 11 (n=113): 63 in 2 to 5 min, 23 in 5 to 10 min, 5 in 10 to 15 min, 22 over 15 min. Most 2 to 5 min gaps are mid-turn waits on subagents or Codex dispatches, not turn ends the orchestrator should act on.

Design notes

• Cadence state is per named consumer in the daemon. The read filter is widened to the state-change kinds so a BUSY the consumer did not ask for still cancels a settling READY; the consumer's --kinds is applied on output.
• All journal pages are drained before a flush, so a BUSY on a later page still cancels a READY on an earlier one.
• The durable named cursor never passes an entry still pending or held in memory. A live cadence instance resumes from its own in-memory read cursor, so nothing is re-read or repeated while it lives. If the instance is discarded (filter change, eviction, reset, or a call without cadence flags), the re-read re-delivers the held entries under the new filter. This is the existing at-least-once contract for named cursors.
• Cadence identity includes the full filter, so a consumer that changes panes, repo, or held-input projection never receives entries held under the old scope.
• The cadence map is pruned with stale-cursor eviction and when a named consumer calls without cadence flags.

Docs

Every normal discovery path now describes the flags, the unchanged responsive defaults, the recommended orchestrator invocation with the conservative values, the wake-up budget, the STUCK fix, the reconciled dead-watch rule, and that Pane Chat arms the flags automatically so a user only needs them for their own scripts.

• runpane watch --help (npm and pip): contracts/runpane/contract.json help.npm.watch and help.pip.watch, with a footer on defaults, the orchestrator invocation, Pane Chat auto-arming, STUCK, and dead-watch.
• runpane agent-context --command watch --json: agentContext.commands.watch now lists all 26 flags with a default and when to use each, four examples, and eight notes; agentContext.brief.tools gains a watch entry.
• Generated contract doc: docs/RUNPANE_CLI_CONTRACT.md now has watch in the Commands usages and a full description paragraph (regenerated from the contract, together with packages/runpane/src/generated/contract.ts, shared/types/generatedRunpaneContract.ts, and packages/runpane-py/src/runpane/generated_contract.py).
• packages/runpane/README.md and packages/runpane-py/README.md: new "Watching the Workspace" section under Advanced.
• CHANGELOG.md: Unreleased entry (Added, Changed, Fixed) written for a first-time reader with the copy-pasteable orchestrator invocation. GitHub release notes are auto-generated from PR titles by scripts/publish-github-release.sh, so this PR's title is the release-notes line.
• Root README.md: does not mention watch; unchanged.
• No other docs/ page covers watch or the Liveness Contract; the contract lives in the emitted skill text in main/src/services/skillCacheManager.ts.

Testing

• pnpm lint && pnpm typecheck: pass.
• pnpm --filter main exec vitest run: 102 files passed, 1 skipped; 978 tests passed, 2 skipped.
• pnpm run test:runpane-contract: pass for npm and pip, including the new cadence request shape, follow-<pid> naming, self-test sending no consumer name (so the daemon applies no cadence), and rejection of --settle nope, --settle 5 without --follow, and --settle with --since.
• New unit tests: workspaceWatchCadence.test.ts (settle cancels READY on BUSY, READY matures with settledMs, BLOCKED settle and bypass carrying held lines, BLOCKED answered in-pane emits nothing, batch grouping, BUSY never emitted when excluded, pane.gone cancellation, lowestUnflushedGen), workspaceIdleTracker.test.ts (10m, 30m, 1h, 3h, 27h, 51h schedule; custom first step; deadline follows schedule), workspaceJournal.test.ts (READY with a > Try "..." screen has no held input, real text does), runpane.test.ts (pending READY survives a request timeout, page-limit cancellation, cursor cap and raw re-delivery, filter and idle-schedule changes drop held entries, a reused instance resumes from its read cursor, reset discards the cadence), skillCacheManager.test.ts (arm command flags, budget, HEARTBEAT and BUSY wording, no bare --follow).
• Reviews: Claude implementation reviewer and Codex code reviewer (two passes); all Must Fix findings addressed in the second and third commits. A four-angle simplify pass (reuse, simplification, efficiency, altitude) produced the fourth commit. The fifth commit rebases onto the merged Add opt-in unattended resilience to the Pane Chat orchestrator skill #609 and reconciles the Liveness Contract wording.
• runpane watch --self-test unchanged and covered by the contract test.

https://claude.ai/code/session_01Tb796QFKPJCTgRXi9MWy9R

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-14T04:13:48.176188Z	c4de31c	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c4de31cea1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Add watchDefaults to the doctor result schema

When a consumer validates runpane doctor --json against the published doctorResult contract, this newly emitted property makes the response invalid: contracts/runpane/contract.json does not list watchDefaults under doctorResult.properties and sets additionalProperties to false. The Python wrapper emits the same undeclared property, so strict clients of either wrapper will reject every doctor response until the schema and generated contracts include its shape.

Useful? React with 👍 / 👎.