Skip to content

fix: relaunch workers after endpoint loss - #2417

Open
twilwa wants to merge 10 commits into
kunchenguid:mainfrom
twilwa:fm/fm-relaunch-missing-endpoint
Open

fix: relaunch workers after endpoint loss#2417
twilwa wants to merge 10 commits into
kunchenguid:mainfrom
twilwa:fm/fm-relaunch-missing-endpoint

Conversation

@twilwa

@twilwa twilwa commented Aug 15, 2026

Copy link
Copy Markdown

Summary

  • recreate a worker on a fresh backend endpoint only when the recorded endpoint is proven absent, then publish the replacement endpoint in task metadata
  • distinguish alive, strongly missing, and ambiguous endpoint inspections; ambiguous tmux/socket results preserve the record and request an explicit recovery decision with the consulted socket and response
  • preserve the recorded worktree and all uncommitted work, including the validated secondmate worktree carve-out, while exact endpoint matching and rechecks keep repeated relaunches duplicate-safe

Validation

  • no-mistakes run 01M01V90RNYA80ZT7ZQZH3C7SM: intent, rebase, review, test, documentation, and lint steps completed; its delivery push alone failed because it targeted upstream instead of the configured fork
  • real tmux coverage exercises alive, wiped/missing, ambiguous, confirmed-ambiguous, repeated relaunch, exact-session matching, and secondmate recreation behavior
  • Herdr lifecycle behavior verified through the required named, non-default isolated lab helper

twilwa added 10 commits August 14, 2026 17:10
* commit 'refs/no-mistakes/recover/01M01V90RNYA80ZT7ZQZH3C7SM':
  no-mistakes(lint): quote hyphenated lint literals and mark output globals
  no-mistakes(document): document tmux missing-grade nuance and relaunch recovery verification
  no-mistakes(review): grade tmux missing endpoints from one verified inspection
  no-mistakes(review): match the tmux container session exactly on ensure
  no-mistakes(review): match tmux sessions exactly and cover secondmate recreation
  no-mistakes(review): grade missing endpoint verdicts before recreating
  fix: relaunch workers after endpoint loss
  fix(ci): fail hung Herdr behavior runs in 20 minutes (kunchenguid#2413)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 77b17de08a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/backends/tmux.sh
Comment on lines +179 to +180
if tmux has-session -t "$session" 2>/dev/null; then
fm_backend_tmux_create_task "$session" "$window" "$worktree"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove the overriding non-exact recreation function

Captain, when a recorded session is gone but a prefix look-alike exists (for example, smok is missing while smoke is live), this bare has-session -t "$session" resolves the look-alike and takes the existing-session branch; the subsequent exact lookup in fm_backend_tmux_create_task then fails instead of recreating the recorded session. This second definition also overrides the exact-match implementation immediately above, breaking the scenario explicitly exercised in tests/fm-backend-tmux-smoke.test.sh:246-288; retain the first definition's -t "=$session" behavior and remove the duplicate.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant