Repository navigation
[P1][CF-6] Extend the existing event journal and paired-client replay to preserve typed conversation state #279
Description
Goal and dependencies
Parent: #273. Priority: P1 — durable recovery. Blocked by: #274 (versioned identity/projection) and #276 (human-request lifecycle). Do not invent a parallel event schema or re-execute tool events.
After reload, sleep, reconnect or restart, recover the same typed transcript and truthful request/outcome state without duplicate actions or lost drafts.
Existing behavior to preserve
The journal guide already documents SQLite/WAL, MODEX_EVENT_JOURNAL=1, journal identity/cursors, batched absolute updates, paired desktop SSE resume, phone delta snapshots and retention. State files remain authoritative. Journal disable/failure falls back to existing reload behavior. This is an extension/migration issue, not a missing-journal implementation.
Preserve defaults unless separately approved: opt-in flag; 250 ms batching; 7-day / 250,000-row retention; 10,000-event replay limit; existing 4 MB replay and 8 MB slow-stream fallbacks. These are the documented baseline values, not new service-level guarantees.
Integration points
Locate the current EventJournal implementation with rg -n 'class EventJournal|replayItems' apps/desktop/src; use that implementation. Existing consumers include apps/desktop/src/main/engine/{store,runner,desktop-host,desktop-client,companion}.ts, apps/desktop/src/renderer/transcript.ts, apps/store-desktop/, and apps/ios/ModexCompanion/.
Extend apps/desktop/test/{event-journal,journal-resume}.test.ts and apps/desktop/e2e/event-journal.spec.ts; add apps/desktop/test/conversation-migration.test.ts. Update docs/event-journal.md and docs/conversation-events.md from #274.
Implementation steps
- Inventory actual snapshot/journal/client schema versions and create saved legacy/current fixtures. Add richer identity, phase, part, outcome and request-state metadata without promoting the journal to authoritative execution state.
- Define a consistent snapshot + high-water cursor boundary. Keep coalescing safe for typed fields: do not merge across item/part/attempt/runtime identities, overwrite a terminal state with an old running update, or discard durable request withdrawal. Replay uses idempotent projections, not callbacks that run tools.
- Extend serializers/validators and the existing desktop/phone merge paths. Negotiate schema capability; unknown future actionable items become non-actionable fallbacks. Preserve historical text and old-client compatible records. Invalid, foreign, expired, ahead-of-log or oversized cursors explicitly reload rather than silently skipping events.
- Reconcile restart with authoritative native state when available. Pending requests from a dead runtime are expired/unavailable, not active approvals. Unknown execution stays unknown. Never automatically resend a user prompt, command, question answer or approval after an uncertain outcome; explicit user retry creates a new attempt with visible consequences.
- Keep disk/journal failure behavior safe: existing state/drafts survive, paired streams reset to snapshot mode, and failure of a projection journal does not claim an action was durably audited. Redact sensitive request/credential content before persistence, rather than relying on renderer masking.
- Exercise migration, batching, wake and paired races with real journal/TLS fixtures, then verify native UI behavior. Record rollback/read compatibility and docs/status.md updates in a signed PR. Default-on journaling is out of scope.
Acceptance criteria
- Direct projection, snapshot + replay and supported client catch-up converge on identical item identity, text, phase and outcome.
- Duplicate identified frames are harmless; legitimately repeated deltas remain represented; old-attempt updates do not mutate new attempts.
- Request withdrawal and resolution survive batching/replay; reconnect never revives a dead card or emits a second native decision.
- Cursor reset, pruning, journal replacement, out-of-range cursor, replay/stream size limits and journal-off fallback all request the correct reload path.
- Crash between persistence/broadcast/native send yields truthful unknown/expired state with zero implicit re-execution; old snapshots migrate without text/draft loss.
- Desktop/Store/iPhone capability mismatches retain readable safe fallback. A legacy binary client cannot approve an unsupported structured question.
- Existing privacy/retention limits are retained and documented; no raw credentials or sensitive question bodies enter the journal.
Verification: RED → GREEN fixtures, pnpm build, pnpm typecheck, pnpm test, pnpm test:e2e, plus the documented paired journal-resume and supported native iPhone tests. Record exact SHAs, environments and skips rather than claiming all-platform acceptance from one OS.
Long-running goal prompt — implement
Implement CF-6 in TypeSafeAI/modex under #273 using the landed #274/#276 contracts. Read current AGENTS.md, docs/event-journal.md, actual journal/snapshot/client implementations and existing paired tests. Work in an owned worktree with pinned pnpm and Superpowers TDD. Extend the SQLite journal and authoritative state files already present; do not build a new event store, change its default flag or replace the native runtime.
Preserve typed item/part/attempt/generation identity, phases, outcomes and human-request withdrawal across batching, snapshots and desktop/iPhone catch-up. Make snapshot/cursor boundaries coherent and updates idempotent. Treat replay as presentation only. A restart or uncertain native send must not resend prompts, commands or answers; dead requests become unavailable and unknown outcomes remain unknown. Maintain legacy readable fallback, existing retention/size limits, draft ownership and journal-failure recovery. Add hostile migration/cursor/batching/reconnect tests before fixes and count native execution calls to prove no duplicate side effects. Run applicable full and paired suites, update compatibility/rollback docs and status, then prepare a signed linked PR. Leave exact SHAs, commands/results, skips, supported environments and next steps in the checkpoint. Do not enable journaling globally, merge or release.
Follow-up long-running goal prompt — verify and finish
Resume CF-6 from the actual PR/head and checkpoints after verifying worktree ownership. Compare journal-off, direct live projection, snapshot-plus-replay and paired-client catch-up on the same mixed transcript. Inject duplicate frames, an old-attempt update, a cursor from a deleted journal, pruning, future/expired cursors, oversized replay, journal write failure and a crash immediately after an approval write.
Check that request withdrawal survives coalescing, terminal outcomes cannot regress, all sensitive data is filtered before storage, and no executor or native response is called by replay. Load old snapshots and unsupported-client fixtures while preserving drafts. Reproduce defects with failing tests, narrowly repair the existing implementation, and rerun build/typecheck/unit/E2E and paired/native checks. Do not expand into an authoritative event store or switch the feature default. Update docs and the issue receipt with exact schemas, SHAs, results, skips and remaining native gates. Leave incomplete acceptance open; no merge or release without separate authorization.
Activity
BunsDev commentedon Oct 18, 2026
Input for CF-6 from the #273 tracker session, after #282 (CF-1) merged as
7780093.Older paired desktops can read unresolved tools as success. CF-1 leaves
okunset on a tool whose outcome isunknownorinterrupted(okForinshared/conversation-events.ts, asdocs/conversation-events.mddocuments). Before7780093, the runner setok: falseon a tool still open when the turn ended.A renderer built before
7780093reads onlyok. It showsok: undefinedas Done, and its fold rule (item.ok !== false) folds the tool away with completed work. A Store client or paired desktop on an older build, connected to a newer host, therefore shows an interrupted or unknown tool as quietly successful. The tracker's done list asks that unsupported clients keep a safe readable fallback. Two options:ok: falsewith those outcomes as the legacy fallback. Current renderers readoutcome, so their labels don't change.I raised this in review on #282 before it merged (review). The contract chose
undefineddeliberately, so I've left the decision to this issue rather than change the contract myself.The iPhone ignores
okandoutcomeentirely.CompanionViews.swiftshows a checkmark for every finished tool (item.status == "running" ? "gearshape.2" : "checkmark.circle"), andcompanion.ts'smobileItemforwards onlystatusandok. So failed, denied, interrupted and unknown tools all look done on the phone. This predates CF-1. It's the companion's half of "supported paired-client recovery converge with safe non-actionable fallbacks".BunsDev commentedon Oct 18, 2026
CF-6 claim: the half that depends only on #274, 2026-10-18, from the session that did CF-1 (#282) and the CF-4 typed-turn slice (#284)
#276 (CF-3) has not landed;
cf3-human-requestsis still at89c98f2. Val asked this session to take only the CF-1-dependent part of this issue now:test/conversation-migration.test.ts. Unsupported clients keep a readable fallback.docs/event-journal.mdanddocs/conversation-events.mdget updated.Left for after #276 lands: request withdrawal and resolution through batching and replay, dead-card revival, the second native decision, and any structured-question capability negotiation.
.worktrees/cf6-typed-replay, branchcf6-typed-replay, frommainat7780093.event-journal.ts,companion.ts(the phone's item projection),desktop-host.ts/desktop-client.tsonly if replay needs it, the journal tests, and the docs.Share the captured-stream test helpers between test files
Replay journal items as upserts and give the phone typed items
Settle work a dead process left running in the snapshot and the journal
Document typed conversation state in the event journal
Carry typed conversation state through the journal, snapshots and the…
BunsDev commentedon Oct 18, 2026
CF-6 checkpoint, first half (the part resting on #274): merged in #285 as
fb76aacon 2026-10-18. Base8e26534after a signed rebase over CF-4; PR head13a8b33. All 11 checks green, includingunit + e2e (macOS)and iPhone companion.Done
replayItemsnow upserts by id, where before it appended a stale copy. Repeated deltas stay represented, and a retry reusing a native id never merges into the first attempt.unknownin the saved snapshot, the journal and every client, not only in one window, and nothing runs again. Before, clients resuming from the journal kept a spinner forever.test/conversation-migration.test.ts).Checks at
01a395b, before the rebasepnpm test: the first stage ran 1,520 tests, with 1,504 passed, 0 failed and 16 skipped. The second stage passed 21 of 21.event-journal,companion,store-desktop,store-demo,desktop-host-launch,app,sourcesandagents.Still open, after #276 (CF-3) lands
Settle a thread on first load without resending items that did not ch…
Capture fidelity states in every theme, measure a long transcript, an…
Record who answered each request and whether the answer reached the CLI
Say on a card when the phone answered or delivery is uncertain
Document the request lifecycle across replay and reconnect
Let the phone act on threads started on the Mac
23 remaining items
Add the e2e test for the journal identity across a relaunch and a kill
BunsDev commentedon Oct 19, 2026
Correction to the #309 receipt (2026-10-19)
The receipt gave #309's head as
e973246and listed ane2e/event-journal.spec.tstest: the app keeps its journal id across a clean relaunch and takes a new one after a SIGKILL. That test was written, and it passed in the 15/15 targeted e2e run on the working tree, but it was never committed. Soe973246, and the545c09ahead that followed it, did not contain it, and CI on those heads never ran it.It is now committed as
0e2964e, on top of a merge ofmainatb804e9c, which brings in #312 and #313. The current state:0e2964e, and its CI is running.0e2964e:event-journal,human-requestsandcompanionpassed 7/7. That covers the kill test and Carry each request's answer, who gave it and its delivery through the journal, replay and the phone (CF-6, request half) #308's journal-on restart test.event-journal,journal-resume,conversation-migration,request-journal,companion,human-requests,human-requests-runner,conversation-eventsandtask-contextall pass, with no failures.The rest of the receipt stands. Val has authorized merging #309 through the guard once its CI passes.
BunsDev commentedon Oct 19, 2026
CF-6 review findings re-checked against
mainat396b17c(2026-10-19)From the docs steward session. Each state below is backed by a test or probe from this session.
test/request-crash.test.ts. On396b17cthe phone's first poll after a crash returnsdelta: truewith 0 items, so the card stays open. With #309's journal, which takes a new id after an unclean shutdown, the old cursor reloads and the card reads closed. A clean quit passes either way (control case). Committed oncf6-residueas2201eca, to push once #309 is onmain.delivery: "sent"recorded before the writesentonly after the backend's write callback.protocol: 1and a fixed app name (desktop-client.ts:128, 146, 226), never its version, so the host can't tell a pre-CF-3 Store build from a current one.mobileItemsends withdrawn, sending and uncertain approvals as notices, which today's iPhone build already renders.cf6-residue(a498ca7), PR to followbyundefined),answerNote, a new runner test and the Store e2e (no note under the paired desktop's approval). Green: 25/25 unit tests, and 15/15 e2e across store-desktop, human-requests, approvals and approval-policy.396b17csaveItemsthrowing once:{"first":"threw ENOSPC: no space left on device","retry":{"ok":false,"reason":"already-resolved"},"cliHeard":[],"status":"waiting","card":{"answer":"yes","delivery":"sending","withdrawn":null}}.cf6-phone-refresh, PR to followModexCompanionTestspasses 47/47 on an iPhone 16 Pro simulator.Finding 10 and #316. #316 rewrites exactly this part of
answer(). It now keeps the pending entry until delivery, but a failed save still leaves the requestresolvinguntil Stop, which then turns ituncertain. One narrow fix: if the save fails, put the broker back topendingand refuse the answer, so "saved before the CLI hears anything" still holds and the person can retry. To #316's owner: say here whether you'd rather fold that into #316. Otherwise I'll take it once #316 lands, so the two branches don't editanswer()in parallel.Question for Val (finding 6). Store builds cut before CF-3, including TestFlight Store 0.0.8, show nothing for a Codex question. That's safe, because their answers are refused, but the thread just appears to wait. Options:
DESKTOP_PROTOCOLto 2. Older Store builds show the existing "The host desktop protocol is incompatible. Update the host and Store app" screen, which keeps saved access and drafts, until they update. Small change. They stop working until updated.questionitems, and any later kind, as a readable notice to clients without it, as it already does for the phone. Old builds keep working. This needs per-client mapping in both journal replay and the live broadcast, so it's more code.I'd pick A, since the Store app is still a preview. I won't change the protocol until you choose.
Nothing was merged, tagged or released. #309 and #315 merge through the guard after #299, as posted on #273.
Coven-Harness: Claude Code / claude-opus-5-5
Harden the event journal against failed writes, crashes and oversized…
BunsDev commentedon Oct 19, 2026
CF-6 residue: where each review finding stands (2026-10-19)
#309 landed at 17:55Z as
c345779, with all 11 checks green onb18a90a. Its squash tree matches the branch. That covers review findings 1, 2, 5 and 8.test/request-crash.test.tsfails on396b17c(delta: true, items: 0) and passes onc345779sentrecorded before the writex-modex-item-kinds, and a client that names none gets aquestionor aplanas a notice. Red first inc44985f.Checks on each head (pnpm 10.34.5, Node 24.18.1):
30ac266onc345779):pnpm testpassed: desktop 1,642 passed, 16 skipped, 0 failed, and every other suite green;store-desktop,human-requests,approvals,approval-policy,event-journal,companion).3da1d75onc345779):pnpm testpassed: desktop 1,639 passed, 16 skipped, 0 failed;store-desktop,workspace-store,desktop-host-launch,human-requests,event-journal).f996cc5on396b17c):ModexCompanionTests47/47;All three are code PRs, so they wait for Val's go-ahead to merge.
To #316's owner: #316 now conflicts with
mainincompanion.ts. #309 added a fallback for unknown item kinds tomobileItem, right after thequestioncase, where #316 rewrites the approval case. Itsstatus.mdandhuman-requests.mdchanges still merge cleanly with #329, #330 and #331.Still open for #279: finding 10, answering questions from the phone, and the native gates: live providers, a physical iPhone, Windows and Linux, the 8 MB slow-stream drop, and power loss. None of #318, #329 or #330 has run against a live provider, the installed TestFlight Store 0.0.8 or a physical iPhone.
Coven-Harness: Claude Code / claude-opus-5-5
BunsDev commentedon Oct 19, 2026
#329 CI update. On
30ac266,unit + e2e (macOS)was cancelled at the 45-minute limit. The unit-test step went silent afterattachment-recovery-fileat 18:14Z and stayed silent until the cancel.attachment-transfer-journal, isn't touched by Record who answered from the phone's YOLO switch and paired desktops, and test a card open at a crash (CF-6) #329. It passed alone three times locally, in 20–27 s.mainin (Compare removal-only edits and notebook cells with the working tree (CF-5) #328, Name what is still open for each fidelity child in status.md (steward cycle 15) #331) asfaa69d7(signed). Every check passed there, with unit tests in about 3 minutes, and the branch is current withmain.So #318, #329 and #330 are all green on their heads. #318 and #330 are behind
mainand get it merged in on Val's go-ahead.Coven-Harness: Claude Code / claude-opus-5-5
Record who answered from the phone's YOLO switch and paired desktops,…
BunsDev commentedon Oct 19, 2026
Val gave the go-ahead to land the three CF-6 repair PRs. I'm landing them one at a time through the merge guard:
9a2e15f.mainmerged in (187243d); its CI is running and the guard is waiting on it.mainrequires branches to be up to date, so a landing in between sends the next PR back through CI. If you can hold other merges until #318 lands, please do.BunsDev commentedon Oct 19, 2026
All three CF-6 repair PRs have landed through the merge guard, on Val's go-ahead. Each squash commit has the same tree as its branch head.
main9a2e15f9a2e15f9c0b063de856e0On #330, CodeQL flagged one high-severity alert. The cause was the test's raw legacy client, which turned certificate validation off.
71bae66makes it pin the host certificate the same wayDesktopClientdoes. #317 landed during that CI run, so #330 needed one more.Still open on this issue:
#316 conflicts with
main, and its Codex thread last wrote at 2026-10-19 03:55Z. I've asked Val on #290 whether it's still live.Not run: a live provider, an installed TestFlight Store 0.0.8, or a physical iPhone. #318 reaches phones only in the next Companion build. The acceptance boxes above stay unticked until those gates run.
BunsDev commentedon Oct 20, 2026
At Val's request, finding 10 (a failed save at accept time strands the request) is now in its own PR: #336, on a fresh branch from
main.answer()andanswerQuestion()now save the card before deleting the pending entry. If the save throws, the broker state and the card go back to what they were and the error goes to the caller, so the person can answer again. This is the same ruleaddItemalready follows. Two runner tests failed first onde856e0, one for an approval and one for a question; both pass now. A third test pins that a failed write that still reached the disk reads asuncertainafter a crash.Local results:
pnpm testpass: desktop 1,664 passed, 16 skipped, 0 failed;human-requests,approvals,approval-policy,companionandstore-desktop).To #316's owner: #316 rewrites the same part of
answer(). #336 changes only the order (save, then delete the pending entry) and adds a rollback if the save fails, so it should carry over to whichever branch lands second.#336 is a code PR, so it waits for Val's go-ahead to merge.
Say the phone no longer shows a withdrawn approval as declined
BunsDev commentedon Oct 20, 2026
Every CF-6 review finding has now landed on
main.mainc345779,9a2e15f0f5c7859c0b0630f5c7859a2e15f1b6f7a43de856e0On finding 10:
de4c6d4b, whose tree matches the squash. Its two runner tests also failed againstmainafter Record native answer delivery only after its transport handoff (CF-3, lands #316) #335, and pass with the fix.30ee279, before Leave a request open when saving its answer fails (CF-6, finding 10) #336, so finding 10 is not in v0.3.0. Record v0.3.0 publication and advance the website fallback #339 lists it alone under "On main, unreleased".On finding 7:
companion.ts,mobileItem).human-requests.md, which still said the phone shows it as declined, and adds the missing test for the withdrawn case.Still open on this issue:
None of the repairs has run against a live provider, the installed TestFlight Store 0.0.8 or a physical iPhone. The acceptance boxes above stay unticked until those gates run.
Reconcile historical CF8 evidence and keyboard review kit with main (#…
Reconcile CF8 history with current main verification records (#364)
Exercise CR2 reads through production Electron IPC (#361)