notifications: fenced session runtime silently blackholes Telegram->session inbound while daemon ACKs

[Yeachan-Heo]

Symptom

User sends a message in a session's Telegram topic. The daemon consumes the update (seen-updates advances), sets the QUEUED reaction (user sees an ACK), forwards user_message over the session ws — and the message never reaches the session. No log line anywhere.

Reproduced live (2026-08-14, ~10:05 KST)

• Daemon healthy, freshly attached to the session's current endpoint (ws://127.0.0.1:52715, pid 59092), same endpoint the broker index advertises (live: true, mtime strict-equal).
• Direct probe: connected to that endpoint with SdkClient.connect(url, token) and sent a well-formed user_message frame — accepted at the socket, silently discarded by the host. No sendUserMessage failed warn, no drop diagnostic.
• The host runtime also stopped publishing outbound (turn receipts frozen), while its endpoint registration was never rotated or marked stale, and daemon-restart-triggered reconciliation never replaced it.

Root cause surfaces (src/sdk/bus/index.ts server.onInbound)

Two unconditionally silent drop gates:

1. if (initializedRuntime.inboundFenced) return; — inboundFenced is set during identity rotation / predecessor-terminal paths and is never reset; a fenced predecessor keeps its native server listening and its broker endpoint advertised, so the daemon keeps delivering into a blackhole indefinitely.
2. if (runtime?.policySuspended && notificationOrigin) { …defer control_command…; return; } — swallows user_message/reply/config_command while policy is provisional; session-control.ts has exits that leave a runtime provisional forever (e.g. reconcile attempt exhaustion at the tail of #reconcile).

Compounding: the daemon flips the Telegram QUEUED reaction immediately after transport.send, before any session-side acceptance, so the user gets a positive ACK for a dropped message.

Fixed so far

PR #4526 (11e98cf6ce) adds bounded warn diagnostics at both drop gates. Not fixed (needs design):

• successor lifecycle: a fenced runtime must either rotate its endpoint registration out of the broker index or be replaced so reconciliation can converge;
• daemon ACK should reflect session acceptance, not socket write;
• #reconcile must not exit leaving policySuspended = true on a live runtime.

Related: #4527 (silent reconcile non-convergence), PR #4526 (registry brick).

[Yeachan-Heo]

Bounded overlap hold — PR #4533 is the sole current mutation owner.

The dedicated #4528 investigation has stopped without commit, push, or PR after classifying overlap with exact PR #4533 head 4046c85ce91c7d8f8a627bb019943631b3318976.

Covered by #4533:

• session-frame token restoration in SessionRouter;
• bounded diagnostics at the fenced/policy-suspended inbound gates;
• legacy stray-daemon reap.

Residual #4528 contract not covered by #4533:

• explicit session-side dropped-frame acceptance semantics for both gates;
• daemon retraction/correction of the positive queued reaction after a dropped frame;
• deterministic production-path proof that policy suspension and fenced predecessor authority cannot silently blackhole an acknowledged user message.

The worktree preserves uncommitted reproduction evidence. Debug-only tests and generated declaration drift are marked removable; none was pushed. This lane will refresh from exact dev and resume only the residual contract after #4533 reaches terminal disposition. Until then it is an evidence-backed non-mutating hold, preventing duplicate ownership of the same bus hunk.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Residual implementation is now in PR #4558.

Exact-head evidence:

• explicit accepted/rejected/dropped session acknowledgement contract;
• fenced and policy-suspended production gates emit correlated dropped before returning;
• Telegram queues only after acceptance and retracts on rejection/drop;
• production-path no-silent-blackhole assertions plus focused runtime/protocol verification.

PR #4533's token containment, diagnostics, and lifecycle work remain untouched.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Evidence refresh — exact PR #4558 head 153c73da00a92103eb5bbb8df22a7a0583bb7eaf (advanced past the earlier 5e25bde5b4ea58b7e52f7e3781039fae66fc219).

Contract coverage on the refreshed head (verified from the e6f8c926dd6f...→153c73da00a9... diff, 4 commits, 12 files):

• inbound_ack states accepted|rejected|dropped with bounded reasons (inbound_fenced, policy_suspended, invalid_input, injection_failed) in crates/gjc-sdk/src/protocol.rs, incl. drop-reason roundtrip test;
• both production drop gates emit correlated dropped acks before returning: inbound_fenced gate and notification-origin policy_suspended user-message gate in packages/coding-agent/src/sdk/bus/index.ts;
• policy-suspended valid control_command frames are deferred to activate() (outcome defer) — no contradictory terminal dropped ack for frames that are later executed;
• accepted commits at AgentSession preflight acceptance (before turn_start, idempotent latch), late admission failure acks rejected;
• daemon reaction contract: queued only after acceptance, retract (reaction: []) on rejected/dropped, per-update serialized monotonic terminal states (InboundReactionSequencer, 512-update tombstone window);
• regression suites: notifications-inbound-acceptance/-production-path/-reaction-ordering/-turn-ordering + notifications-topic-registry + generation guard (manifest 167→168), cargo test -p gjc-sdk inbound_ack --lib.

Exact-head review state: 3 reviews, all COMMENTED (latest on 153c73da00a9... at 13:30:48Z, pre-dating the head advance) — no authenticated APPROVED review exists on any head. PR body verdict line is gajae.pr-review-verdict.v1 needs-human sha256:510edf2e3fac75a9e18f3a4fce810404ff4f55ed7be6a324ffd75c14177be3d5 reviewer:human reviewer-id:pending-independent-review — digest independently recomputed from git diff --binary --full-index e6f8c926dd6f388be8d87868eaeee1afff651b40...153c73da00a92103eb5bbb8df22a7a0583bb7eaf and matches. "PR contract bootstrap" on run 31806706855 fails closed on it (Verdict needs-human intentionally blocks merge) — by design, not a defect.

CI classification (pinned head 5e25bde5b4, run 31805011771): Affected path validation / native-build uploaded all 61,718,830 bytes (artifact SHA256 6a191304a28d83eaaa0f8dcefc3c49aa0aab2d37c79496882c001349ce0955d9) and then failed only at Failed to FinalizeArtifact: Unable to make request: ETIMEDOUT (13:37:25Z). The evidence producer/Affected path validation failures are the downstream fail-closed cascade of CI_DEV_NATIVE_RESULT: failure. Windows Telegram daemon safety, Telegram daemon generation guard, and all four gjc-state-gates lanes passed. Infrastructure/downstream aggregate — no new code defect. Run 31805062227 ("PR contract") failed the same needs-human intentional block, same classification.

Current head 153c73da00 run 31806706855 is in_progress (native-build and Windows Telegram daemon safety running); prior runs at this head on the same workflow are cancelled supersessions, not failures.

Disposition: hold #4528 open tied to exact PR head 153c73da00a92103eb5bbb8df22a7a0583bb7eaf while nonterminal (needs-human verdict stands, no exact-head APPROVED review, fresh run in flight). This lane owns issue evidence and closure only; PR #4558 mutation stays with its dedicated owner. On merge into dev plus merged-history/product verification, #4528 closes immediately with signed evidence.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Terminal exact-head CI receipt — supersedes the in-progress wording in my prior update. PR #4558 head 153c73da00a92103eb5bbb8df22a7a0583bb7eaf, base e6f8c926dd6f388be8d87868eaeee1afff651b40.

Run 31806706855 (Dev CI, head 153c73da00a9) is now completed: 35/39 jobs green, 4 platform-conditional skips, and exactly one red job — PR contract bootstrap, failing closed on Verdict needs-human intentionally blocks merge under the exact-head digest sha256:510edf2e3fac75a9e18f3a4fce810404ff4f55ed7be6a324ffd75c14177be3d5 (independently recomputed from git diff --binary --full-index e6f8c926dd6f...153c73da00a9 and matched). Green includes: Affected path validation / native-build (the earlier 5e25bde5b4 FinalizeArtifact ETIMEDOUT is thereby confirmed transient infrastructure, not a code defect), Windows Telegram daemon safety, Telegram daemon generation guard, Virtual integration validation, all notifications-inbound/-reaction-ordering/-turn-ordering/-topic-registry suites, rust-test/rust-check/cli-smoke, and all gjc-state-gates lanes.

Review state (revalidated live): 4 reviews, all COMMENTED by chatgpt-codex-connector[bot] (latest 2026-08-14T13:56:01Z on the exact head) — authenticated APPROVED reviews on exact head 153c73da00a9: zero. PR open and unmerged; origin/dev unchanged at e6f8c926dd6f.

Disposition (unchanged, now with terminal CI evidence): hold #4528 open tied to exact head 153c73da00a9 while nonterminal — the only remaining gates are the human-only authenticated independent APPROVED review and the merge by the dedicated PR mutation owner. #4528 closes only after #4558 merges into dev and merged-history/product verification proves resolution; this lane mutates issue evidence and closure only.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Closed by #4558 — signed merged-history and product evidence

Merged: PR #4558 → dev, merge commit 9d2a2d2f2d8f074333bba92fe1fc4d6902f5b2df (GitHub-reported merge_commit_sha).
Approved head: 153c73da00a92103eb5bbb8df22a7a0583bb7eaf — verified in merge-commit ancestry (git merge-base --is-ancestor passes).
Authenticated approval: probepark APPROVED at 2026-08-14T16:14:20Z on that exact head, collaborator permission write, non-author.
Verdict: gajae.pr-review-verdict.v1 merge-approved sha256:510edf2e3fac75a9e18f3a4fce810404ff4f55ed7be6a324ffd75c14177be3d5 reviewer:human reviewer-id:probepark — digest recomputed with the repository verifier's canonical rule and independently validated by PR contract run 31818385074 (success).
Exact-head Dev CI 31818382607: success — all required jobs green.

Contract delivered for #4528:

• explicit session-side inbound_ack outcomes (accepted at preflight acceptance before the turn / rejected on any admission failure / dropped only for genuinely discarded frames);
• policy-suspended valid control_command deferred to activation instead of terminal-dropped;
• per-update reaction transitions serialized with monotonic terminal states (a late 👀 can no longer overwrite ✅ or a retraction);
• retraction sends the Bot API's empty reaction list, not an invalid empty emoji;
• bounded sequencer state (settled chains deleted, distance-windowed tombstones);
• daemon generation 167→168 with refreshed semantic manifest; CHANGELOG entry.

Post-merge verification on current dev: bun run build success (natives + coding-agent dist/gjc); focused notification suites 101 pass / 0 fail across acceptance, production-path, reaction-ordering, turn-ordering (incl. the real-WS preflight lifecycle regression), topic-registry; cargo test -p gjc-sdk inbound_ack --lib 2 pass; check:types pass.

Post-merge dev CI note: dev CI 31820749528's Telegram daemon generation guard failure is a pre-existing discord/slack session-router digest staleness introduced by #4542 (independently reproduced at pre-merge dev 21d0a44209), not by #4558 — all telegram digests matched. Follow-up repair PR #4563 (digest-only, canonical --write-manifest) is green on every product job including the generation guard, pending only its own authenticated approval.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Reopened — merged-state evidence update

#4558 product contract: MERGED and verified. Merge commit 9d2a2d2f2d8f074333bba92fe1fc4d6902f5b2df on dev; approved head 153c73da00a92103eb5bbb8df22a7a0583bb7eaf in ancestry; probepark authenticated APPROVED; verifier-validated merge-approved digest 510edf2e…; exact-head Dev CI 31818382607 success. Notification-focused verification on merged dev: focused suites 101 pass / 0 fail, bun run build success, cargo -p gjc-sdk inbound_ack 2 pass, check:types pass.

Dev-wide closure is blocked on two independent postmerge repairs (not #4558 defects):

1. PR chore(guard): refresh discord/slack session-router digests after #4542 #4563 — Telegram daemon generation guard red at the merged head is the pre-existing fix(sdk): run initial attachment replay off the reconcile tail (#4527) #4542 stale discord/slack session-router digest defect (independently reproduced at pre-merge dev 21d0a44209). One manifest-only commit at head 06cbd1a1501e7fbcb6d1ca9c7fc17477a9c4a51f, 20 product jobs green including the generation guard, sole red is the intentional needs-human gate; probepark review requested.
2. PR fix(session): restore synchronous append for uncontended canonical admission #4564 — shards 6/4 reds are the deterministic fix(session): Cursor split persists tool result after its continuation in canonical order #4536 postmerge regressions (agent-session-issue-2261-esc-subgraph-cancel ×2, agent-session-deep-interview-continuation, agent-session-midrun-maintenance), owned there at head 9a98c35d70, base 9d2a2d2f2d; fresh CI in progress; probepark review requested.

The evidence producer/aggregate reds are downstream of those two.

Close condition: both #4563 and #4564 merged and a fresh dev CI run at the resulting dev head fully green — then this issue closes immediately with the signed receipt. This lane does not duplicate their fixes.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Bounded hold — #4528 final closure awaiting approvals on #4563 and #4564

#4558 (the #4528 product fix) is merged at 9d2a2d2f2d8f074333bba92fe1fc4d6902f5b2df; notification-focused verification passed on merged dev (focused suites, build, cargo, typecheck). Remaining dev-wide reds belong to the two independent postmerge repairs:

• chore(guard): refresh discord/slack session-router digests after #4542 #4563 — exact head cae16160a4fa350c2f84ecdc1ea6cef67a0b1d88: generation-fenced guard repair; all product jobs green in runs 31831417583/31833702901, sole red intentional needs-human; verdict digest 5369ed1ffecc1ce37cb3eb1b105457b0cbf5e19f3f4a70d10a7443192b841cd5 (verifier-matched). Reviewers requested: probepark, HaD0Yun.
• fix(session): restore synchronous append for uncontended canonical admission #4564 — exact head e8a956ae0b718b3893475d5e75c2fff4b7dc0616: deterministic fix(session): Cursor split persists tool result after its continuation in canonical order #4536 session regressions fixed; all product jobs green in run 31839757513, sole red intentional needs-human; verdict digest 7473766943882f941a61af13f42f2d96165b2b5a9667caa4031ec125f836a38f (recomputed and matching). Reviewers requested: probepark, HaD0Yun.

No autonomous work remains on either PR: code is terminal-green and honestly verdicted. The only outstanding dependency is one authenticated APPROVED review per PR on its exact head (contract forbids self-approval by the PR author). On each approval: flip that PR's verdict to merge-approved bound to the actual reviewer, merge, and after both merge verify a fresh fully-green dev CI run, then close #4528 with the signed receipt. No approval will be fabricated or impersonated.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Final disposition — #4528 resolved by merged #4558

The sole #4528 implementation owner, PR #4558, is MERGED into dev at 9d2a2d2f2d8f074333bba92fe1fc4d6902f5b2df (remote dev and this disposition worktree both resolve to that commit). Its approved head 153c73da00a92103eb5bbb8df22a7a0583bb7eaf is in the merge ancestry; the authenticated independent approval and verifier-matched merge-approved digest 510edf2e3fac75a9e18f3a4fce810404ff4f55ed7be6a324ffd75c14177be3d5 are recorded on #4558. Exact-head Dev CI 31818382607 succeeded.

Fresh merged-dev verification in this recovery lane:

• bun test over the five notifications: fenced session runtime silently blackholes Telegram->session inbound while daemon ACKs #4528 notification acceptance/production-path/reaction-ordering/turn-ordering/topic-registry suites: 101 pass, 0 fail;
• cargo test -p gjc-sdk inbound_ack --lib: 2 passed;
• bun --cwd=packages/coding-agent run check:types: passed.

PR #4563 is a separate Discord/Slack generation-manifest repair and PR #4564 is a separate #4536 regression repair. Neither is an open implementation owner for #4528 or changes the delivered #4528 acceptance contract. Their pending reviews therefore are not a terminal dependency for this issue.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]