Skip to content

fix(orchestrator): cumulative per-conversation budget on automatic run-completion previews - #1689

Merged
artokun merged 1 commit into
mainfrom
fix/1516-session-preview-budget
Aug 18, 2026
Merged

fix(orchestrator): cumulative per-conversation budget on automatic run-completion previews#1689
artokun merged 1 commit into
mainfrom
fix/1516-session-preview-budget

Conversation

@artokun

@artokun artokun commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #1516.

  • automatic run-completion previews now stop at a cumulative per-conversation budget (count and bytes) instead of compounding a provider conversation without bound
  • past the budget the completion still arrives — every output named, zero pixels attached, and an honest note with get_image coordinates; nothing is swallowed and no notice claims pixels the turn does not carry
  • a user's explicit image attachments never spend the automatic-preview budget (they keep their own reviewed policy, as the issue asks)
  • the ledger persists beside the session id in the session store, so an orchestrator restart does not reset a budget the still-living provider conversation keeps accumulating against

Root cause

#1521 (v0.51.53) removed the immediate replay amplifier and made the preview bound per turn (8 images). But a long session drains 8 previews per turn into the SAME conversation, and Codex persists every delivered localImage as an inline input_image data URL in the thread rollout, which its compaction path retains and recopies without a media-aware bound (upstream: openai/codex#33493). So N turns still accumulated 8N inline images — the reporter measured 265 persisted input_image records, an 18.2 GiB rollout, ~487 MB recopied per later compaction, and a resumed app-server stalled at ~9 GB. The per-turn cap alone cannot see that shape; the conversation itself needed a ceiling.

The fix

New src/orchestrator/preview-budget.ts: MAX_SESSION_PREVIEW_ATTACHMENTS (default 48) and MAX_SESSION_PREVIEW_BYTES (default 128 MiB), env-overridable via COMFYUI_MCP_SESSION_PREVIEW_ATTACHMENTS / COMFYUI_MCP_SESSION_PREVIEW_BYTES. Enforced at the same estimate/fact split the per-turn budget already uses:

  • injectEvent (estimate) — a matched completion gets at most min(what the turn has left, what the conversation has left); when the session budget binds, the notice names it (...cumulative automatic-preview budget (48 images / ~128 MB) is spent...) and lists fetchable coordinates.
  • channel drain (fact) — the merged batch is capped at the tighter of the two budgets, delivered previews are charged to the conversation, and the existing claim-correction note now names the cumulative ceiling too.
  • codex-backend image loop (byte fact) — PanelAgent cannot see the bytes of a batch already in flight, so the backend enforces the same byte budget where the fetch actually happens: an automatic preview past the budget is not attached, and a [panel note: ...] appended to the turn corrects the claim with coordinates. Bytes are charged per actual delivery — a replayed turn really does write the pixels into the rollout again, which is the accumulation being bounded.
  • durability — the ledger ({images, bytes}) rides the session-store entry (Entry.pl), survives an orchestrator restart, is adopted only when the live session id is exactly the one it was recorded against, is dropped when a fresh conversation starts, and setPreviewLedger refuses a session mismatch rather than misfile a safety bound.
  • diagnosticsPanelAgent.previewBudgetStatus() exposes delivered images/bytes, withheld count, and the budget.

Explicit user attachments and the #468 journal ack/replay semantics are unchanged.

Verification (worktree at origin/main 212132a)

  • npm ci --ignore-scripts + npm rebuild better-sqlite3 — clean
  • npm run build (tsc) — clean
  • npm run lint (tsc --noEmit) — clean
  • New: npx vitest run src/__tests__/orchestrator/session-preview-budget.test.ts src/__tests__/orchestrator/codex-preview-byte-budget.test.ts9/9 (cumulative 8+2-then-none across turns with honest notices; byte budget binding with count room; user-attachment exemption; ledger persistence + exact-session adoption + mismatch refusal; codex byte backstop incl. claim correction and per-thread reset)
  • Neighbors: run-completion-continuation.test.ts + session-store.test.ts124/124
  • Full src/__tests__/orchestrator/162 files, 2812/2812

What this deliberately does NOT do (follow-ups)

  • Pre-resume rollout size guard (the issue's item 4): refusing or confirming the resume of an already-oversized Codex rollout — e.g. one written before these bounds existed, or grown by user attachments — is a separate mechanism with a real product choice attached (confirm vs. auto-fresh-thread) and deserves its own PR.
  • Byte reporting for non-Codex backends: the byte half of the budget is currently enforced where bytes are reported (Codex). Other vision backends are bounded by the image count; teaching Claude/Gemini automaticPreviewBytes() is a small follow-up.
  • Surfacing previewBudgetStatus() in a panel-visible diagnostic is unwired; the data is available.

…ad of compounding a Codex rollout without bound

The per-turn ceiling (v0.51.53) bounds ONE turn, but Codex persists every
delivered preview as an inline input_image data URL and its compaction
recopies them without a media-aware bound, so N turns still accumulated
8N images into one rollout (measured: 265 images, 18.2 GiB, ~487 MB
recopied per compaction). Add the cumulative half of the budget:

- a per-conversation count AND byte budget (defaults 48 images / 128 MiB,
  COMFYUI_MCP_SESSION_PREVIEW_ATTACHMENTS / _BYTES), enforced as an
  estimate at injectEvent and as fact at the drain and at the codex
  image loop, where the real fetched bytes are known
- past the budget the completion still arrives with every output named,
  zero pixels attached, and an honest note with get_image coordinates
- a user's explicit attachments never spend the automatic budget
- the ledger persists beside the session id in the session store, so an
  orchestrator restart does not reset a budget the living conversation
  keeps accumulating against; a stored ledger is adopted only by the
  exact session it was recorded against, and setPreviewLedger refuses a
  mismatch rather than misfile a safety bound
- delivered/withheld counts and bytes are exposed via
  PanelAgent.previewBudgetStatus()
Copilot AI balanced review requested due to automatic review settings August 18, 2026 08:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@artokun
artokun merged commit 2f46ffb into main Aug 18, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex run-completion previews can replay and compound into stalled sessions

2 participants