Description
Deferred from specs/075-orchestration-node-control-parity/ (PR #6242, source issue #6021). v1 ships idle_timeout_secs as a defined, config-surfaced, but documented no-op — no progress-signal plumbing exists anywhere in the codebase today (confirmed: zero heartbeat/liveness/progress signals across zeph-subagent and zeph-orchestration). This issue tracks wiring the real idle-detection mechanism.
Why this is deferred, not implemented directly
The obvious approach — route a per-task progress signal through the existing completion-event channel (mpsc::channel(64) in crates/zeph-orchestration/src/scheduler/mod.rs) — is unsafe: wait_event() already drops buffered events under saturation (logged as "completion event dropped — task may remain Running until timeout"). Multiplexing high-frequency progress signals (one per agent-loop iteration or tool-call) onto that same channel risks evicting a real terminal completion, hanging a healthy task until run_timeout. This eviction hazard was judged to exceed a P3's risk budget for v1.
Target design (already spec'd, not yet implemented)
- Preferred mechanism: a coalescing per-task progress timestamp, NOT a queued message. Each running task holds a shared
Arc<AtomicU64> (or a watch::channel<Instant>) that the executor writes on each progress boundary; the scheduler reads it in check_timeouts() as last_progress_at. A timestamp write coalesces (latest wins) and cannot evict anything — O(1) memory, no queue, lock-free with no torn reads on tier-1 platform targets.
- If a channel is used instead, it MUST be separate from the completion channel, and progress MUST be consumed inline at recv (update
last_progress_at, never pushed to any buffered-events structure).
- Bounded emission granularity must be defined (e.g. one signal per agent-loop iteration or per tool-call completion — coarse enough not to spin).
RunInline tasks cannot participate in this model as-is (the tick loop is blocked during inline execution) — idle_timeout should remain a documented no-op for RunInline specifically, or (as an extension) the inline loop could self-check idle between its own iterations.
- Document precisely what emits "progress" and warn in config docs /
--init wizard help text that idle_timeout must be set above the longest expected single-operation duration (a legitimate long tool call — build, large download — crosses no progress boundary and would otherwise trip idle detection on a healthy task).
Environment
- Source:
specs/075-orchestration-node-control-parity/srs.md §9 (FR-D-02), spec.md
- Design-review handoff:
.local/handoff/2026-07-13T21-05-40-critic.md (S3 — channel-eviction risk finding)
Priority
P3 — no demonstrated urgent need; run_timeout alone covers the majority of the LangGraph TimeoutPolicy parity value.
Description
Deferred from
specs/075-orchestration-node-control-parity/(PR #6242, source issue #6021). v1 shipsidle_timeout_secsas a defined, config-surfaced, but documented no-op — no progress-signal plumbing exists anywhere in the codebase today (confirmed: zero heartbeat/liveness/progress signals acrosszeph-subagentandzeph-orchestration). This issue tracks wiring the real idle-detection mechanism.Why this is deferred, not implemented directly
The obvious approach — route a per-task progress signal through the existing completion-event channel (
mpsc::channel(64)incrates/zeph-orchestration/src/scheduler/mod.rs) — is unsafe:wait_event()already drops buffered events under saturation (logged as "completion event dropped — task may remain Running until timeout"). Multiplexing high-frequency progress signals (one per agent-loop iteration or tool-call) onto that same channel risks evicting a real terminal completion, hanging a healthy task untilrun_timeout. This eviction hazard was judged to exceed a P3's risk budget for v1.Target design (already spec'd, not yet implemented)
Arc<AtomicU64>(or awatch::channel<Instant>) that the executor writes on each progress boundary; the scheduler reads it incheck_timeouts()aslast_progress_at. A timestamp write coalesces (latest wins) and cannot evict anything — O(1) memory, no queue, lock-free with no torn reads on tier-1 platform targets.last_progress_at, never pushed to any buffered-events structure).RunInlinetasks cannot participate in this model as-is (the tick loop is blocked during inline execution) —idle_timeoutshould remain a documented no-op forRunInlinespecifically, or (as an extension) the inline loop could self-check idle between its own iterations.--initwizard help text thatidle_timeoutmust be set above the longest expected single-operation duration (a legitimate long tool call — build, large download — crosses no progress boundary and would otherwise trip idle detection on a healthy task).Environment
specs/075-orchestration-node-control-parity/srs.md§9 (FR-D-02),spec.md.local/handoff/2026-07-13T21-05-40-critic.md(S3 — channel-eviction risk finding)Priority
P3 — no demonstrated urgent need;
run_timeoutalone covers the majority of the LangGraphTimeoutPolicyparity value.