fix(proxy): recover goal restarts from unavailable owners - #1679

Summary

A Codex conversation restart can resend a self-contained thread under the same process-session identifier after its legacy owner exhausts quota. Raw legacy codex_session rows are intentionally hard, so ordinary requests must fail closed; before this change, that same row also trapped an explicit, self-contained goal restart on an unavailable account.

This PR adds one proof-gated direct-routing exception. A request may retire an unavailable raw legacy owner only when it carries Codex's recognized goal-continuation marker and its canonical upstream Responses payload is account-neutral and self-contained. Retirement is compare-and-set, policy-scoped, and request-owned. Ordinary, incremental, file-pinned, conversation-bound, and unresolved-tool requests remain fail-closed.

HTTP bridge reuse/replacement is deliberately split into the dependent #1680. No public wire format, setting, or default timeout changes. One nullable sticky-session column records source-qualified abandonment without backfilling historical rows.

Linked issue: none exists for this incident-derived defect; routed regression coverage exercises the public Codex Responses and direct WebSocket paths.

Behavior and safety

• Recognize the existing goal-continuation context only after canonicalizing the request to the upstream Responses body.
• Permit guarded retirement only for a raw legacy process-session row whose owner is durably PAUSED, RATE_LIMITED, or QUOTA_EXCEEDED.
• Require the persisted owner to be inside authenticated account-assignment and security-policy scope, computed before model and service-tier eligibility.
• Scope abandonment to process-session interpretation while retaining the raw account as hard ownership for an equal explicit turn-state value.
• Compare mapping owner and unavailable status atomically; a concurrent owner change or recovery wins.
• Exclude a successfully retired owner from the remainder of the same selection, even when selection loaded a stale ACTIVE account snapshot.
• Preserve request-scoped authority through direct WebSocket selection and strip only proxy-generated turn state when the restart changes accounts.
• Keep every unproved or account-dependent continuation hard owner-bound.

OpenSpec

• Behavior is specified by the archived, verified change at openspec/changes/archive/2026-08-10-recover-restarted-conversation-affinity/.
• Stable requirements and rationale are synchronized in openspec/specs/sticky-session-operations/.
• Parent requirements cover canonical classification, scoped CAS retirement, stale-selection exclusion, and direct WebSocket state provenance.
• HTTP bridge requirements are isolated in the stacked fix(http-bridge): preserve goal-restart recovery across reconnects #1680 delta.

Origin and concurrent work

Landed lineage:

• fix(proxy): purge stale hard codex_session mappings pinned to a durably unavailable owner #1417 (b1d27bc6) introduced bounded stale hard-owner cleanup while keeping hot-path requests hard. This PR preserves that default and adds only a proof-gated explicit-restart path.
• fix(proxy): preserve developer-interleaved fresh resends #1537 (201281b5) established the canonical account-neutral fresh-resend classifier. This PR reuses that boundary rather than defining another movable payload heuristic.
• fix(proxy): never inject a cross-account previous_response_id anchor (takeover of #1274) #1638 (68397054) enforced ownership of previous_response_id; requests with that anchor remain immovable here.
• fix(proxy): fail over stuck HTTP bridge owner requests to a fresh account #1401 (f2f8f916) and fix(proxy): recover dead durable bridge anchors fast #1625 (85f1ee4b) expanded bridge recovery and dead-owner handling. Their bridge lifecycle concerns are handled separately by fix(http-bridge): preserve goal-restart recovery across reconnects #1680, not hidden in this object.
• fix(proxy): durably recover hard HTTP bridge operations #1657 (7a0b6719) and fix(http-bridge): keep idle retirements out of retry circuit #1677 (7c467198) landed before this rebase. Their durable-operation and retry-circuit behavior is retained by the parent tree and exercised further in fix(http-bridge): preserve goal-restart recovery across reconnects #1680.

Concurrent work reviewed for overlap:

• fix(http-bridge): preserve goal-restart recovery across reconnects #1680 is the dependent HTTP-bridge object. It consumes this request-scoped capability and must merge after this PR.
• fix(proxy): make durable bridge claim_session a real compare-and-set #1643/fix(proxy): recover ownerless durable bridge leases (bridge_instance_mismatch regression) #1648 change durable bridge claim semantics but do not authorize legacy sticky-row retirement.
• feat(proxy): add OpenAI-compatible fallback for exhausted subscriptions #1664/fix(compact): recover previous-response-pinned compaction when the owner account is unavailable #1490 add separate subscription-overflow and compact recovery paths; neither may reinterpret an ordinary hard session request as a restart.

Review findings addressed

Concrete review findings incorporated in this object:

1. Retirement is limited to the authenticated account-policy scope.
2. Successful retirement overrides stale account-selection snapshots.
3. Accepted compatibility and transport-envelope fields classify through the canonical upstream request body.
4. Proxy-generated direct-WebSocket turn state does not cross the account change, while client-supplied state remains hard.
5. Raw-key abandonment retains colliding explicit turn-state ownership.
6. Model and service-tier eligibility do not narrow authenticated mutation authority.

The cloud Codex review was clean at 65103b40. Current-head review was requested after the rebase, but the fork connector rejected it with an account-linking error; there are no unresolved non-outdated review threads.

Current-main rebase

Rebased on current main at 7c467198 (#1677), which already includes #1657 and its Alembic merge head. The additive migration is renamed to 20260812_120000_add_sticky_abandonment_scope and now follows 20260812_000000_merge_recovery_dispatch_and_hourly_cancelled_heads, preserving a single-head graph.

Validation

affected suites: 1,195 passed
load-balancer refresh signature regressions: 87 passed, 3 skipped
Ruff check and format: passed
type and architecture/import checks: passed
additive migration upgrade/downgrade regression: passed
OpenSpec validation: 49 specs passed, 0 failed
git diff --check: passed

The first current-head cloud run passed unit, core integration, PostgreSQL, migration, Ruff, type, packaging, Docker, and browser jobs. Its bridge shard had one isolated bridge_instance_mismatch in test_v1_responses_http_bridge_classifies_responses_lite_developer_interleaved_full_resend[unowned-developer-message]; that exact test passes locally on 736dd4cd, and the same bridge suite is green on the stacked #1680 head. The fork author cannot use GitHub's failed-job rerun action without upstream repository administration.

Screenshots / output

No dashboard-visible change.

Before:

marked self-contained restart + unavailable raw owner -> hard-affinity failure

After:

verified marked restart -> guarded CAS retirement -> eligible replacement owner
ordinary/account-dependent request -> unchanged fail-closed ownership

Simplicity

• Zero configuration; no setting or environment variable.
• One additive nullable migration with no historical backfill; no setup step, README section, or dashboard surface.
• Reuses existing replay classification, sticky tombstones, and selection.
• No new retry budget or timeout.

Checklist

• Conventional Commit PR title.
• OpenSpec behavior is present, synchronized, verified, and archived.
• Direct route, direct WebSocket, scoped-policy, stale-snapshot, source-provenance, model-authority, and migration regressions are included.
• Simplicity gates P1-P5 reviewed.
• CHANGELOG.md was not edited.
• Current-head cloud CI green.
• Current-head Codex review evidence clean or all findings addressed.