fix(proxy): quarantine silent HTTP bridge sessions

[WangErgouaaaa]

Summary

Bound accepted-but-silent HTTP bridge submissions so one stuck upstream
websocket cannot hold a session gate and stall later Codex requests. The
ambiguous current call fails without an internal replay; the next independent
retry temporarily bypasses that bridge over direct HTTP while preserving
previous_response_id account ownership.

Fixes #1404

Type of change

• fix: — bug fix (no behavior change beyond the bug)
• feat: — new user-facing feature or capability
• refactor: — internal refactor (no behavior change, no API change)
• docs: — documentation only
• chore: / ci: / build: — tooling, CI, packaging
• test: — test-only change
• Breaking change

Linked issue: Fixes #1404

OpenSpec

• This PR includes / updates an OpenSpec change
• Not applicable — bug fix that matches the existing spec
• Not applicable — docs / CI / chore only
• This PR touches a codex-faithful Responses path and preserves
  upstream-equivalent request/SSE behavior

Change directory:
openspec/changes/quarantine-silent-http-bridge-sessions/

Changes

• Record when bridge response.create is sent and bound the wait for
  response.created (default 5 seconds).
• Fail the ambiguous current request without replay, retire/close the silent
  bridge, and quarantine its affinity key for a bounded interval (default 60
  seconds).
• Send only the next independent retry through direct HTTP; existing owner
  resolution keeps previous_response_id chains on the account that created
  them.
• Close the underlying bridge lifecycle iterator and upstream websocket when a
  downstream stream closes, including after only the initial SSE heartbeat.
• Add regression coverage for quarantine/retry, continuity ownership,
  unrelated-account concurrency, and disconnect cleanup.

Safety invariant

Once response.create has been sent, upstream acceptance is ambiguous. This PR
deliberately does not replay the same request inside that client call;
automatic replay could duplicate work or fork a response chain.

Simplicity

• Works with zero configuration and preserves existing bridge defaults
• No new required setup step
• No README section, .env.example entry, dashboard nav item, or UI change
• New internal settings and rationale:
  • http_responses_session_bridge_response_created_timeout_seconds=5:
    conservative working default; override remains available only for incident
    tuning on unusually slow upstreams.
  • http_responses_session_bridge_quarantine_seconds=60: conservative
    recovery window; override remains available only for incident tuning and
    can be set to zero to disable quarantine.

Test plan

pytest tests/integration/test_http_responses_bridge.py tests/integration/test_proxy_api_extended.py -q
146 passed in 184.42s

ruff check .
All checks passed!

ruff format --check .
802 files already formatted

ty check --python-platform linux
All checks passed!

openspec validate quarantine-silent-http-bridge-sessions --strict --no-interactive
Change 'quarantine-silent-http-bridge-sessions' is valid

openspec validate --specs --strict --no-interactive
47 passed, 0 failed

The full make ci parity target was not run on this Windows host because GNU
make, Docker/Helm, and local PostgreSQL gates are unavailable; GitHub CI
remains required. The standalone architecture script reaches a pre-existing
origin/main ratchet failure (load_balancer.py: 3260 lines vs 3021) in an
untouched file. Touched ratcheted files remain within limits (service.py:
2600, HTTP bridge mixin: 2393, streaming mixin: 1100).

Screenshots / output

No dashboard-visible change. Expected recovery sequence is observable as
low-cardinality bridge events:

response_created_timeout -> bridge retired/closed
next independent retry -> quarantine_http_fallback -> direct HTTP

Checklist

• Title is in Conventional Commits format.
• Linked the related issue.
• Added integration tests at the externally failing HTTP/Codex streaming
  path.
• Ran uv run pre-commit run local-ci --hook-stage manual --all-files
  (unavailable on this host; relevant local gates are listed above).
• Strict OpenSpec validation passes.
• Simplicity gates reviewed.
• CHANGELOG.md was not edited.

[Soju06]

To keep the silent-upstream family coherent we're consolidating on the #1410 watchdog track first (rationale in the #1394 discussion). The quarantine idea can still complement it — a per-key HTTP-fallback window is orthogonal to watchdog retirement — so keeping this open as a follow-up candidate rather than a competing root fix.

Two functional concerns from the current diff (c08983b) worth addressing in any revival:

1. The timer only arms while request_state.response_event_count == 0 (streaming.py). The production wedge fix(http-bridge): unstarve the stuck-gate watchdog on silent upstreams (follow-up to #1394) #1410 instrumented was a reattached stream that delivered 54 events whose response.created was never assigned — that shape never trips this quarantine, so the worst observed incident stays unhandled.
2. http_responses_session_bridge_response_created_timeout_seconds defaults to 5s. Under normal queueing response.created regularly arrives later than that; each false positive retires the whole session (failing every pending request with stream_incomplete) and then forces 60s of raw-HTTP fallback for the affinity key, sacrificing prompt-cache/continuity affinity on healthy accounts.

Also note CI is red on this branch's own account, not just the known architecture ratchet: tests/unit/test_settings_reference.py::test_settings_surface_ratchet fails (116 > 115 — the two new settings need the simplicity-budget discussion per PRINCIPLES.md P2 / #1340) and test_generated_settings_reference_matches_code fails (generated reference not regenerated). The branch is also conflicting with main now.

[github-actions]

This pull request has been automatically marked as stale because it has had no activity for 7 days.

It will be closed in 23 more days unless there is new activity.

If this is still relevant, please:

• Rebase or push an update if the branch drifted
• Address pending review feedback if there is any
• Leave a short comment confirming it is still being worked on

Thanks for the contribution 🙏

[Soju06]

Maintainer decision: quarantine (this PR's direction) is the chosen approach for silent-bridge sessions over the account-health penalty alternative (#1574, now closed). To proceed it needs: (1) rebase across #1562/#1394 (the reconnect/recovery surface moved substantially), (2) the 07-24 findings addressed (quarantine never trips on the reattached shape from #1534; the 5s response.created timeout is too aggressive post-#1394 — align with the 60s eventless deadline). If you don't have bandwidth, say so and we can pick it up.

[Soju06]

@WangErgouaaaa checking in — quarantine remains the chosen approach for silent HTTP-bridge sessions (maintainer decision above, #1574 closed in its favor), so this PR is wanted; it needs a revival pass to proceed. Concrete checklist:

1. Rebase onto current main — the reconnect/recovery surface moved substantially with feat(proxy): add durable capability lineage for reconnect #1562/fix(http-bridge): stabilize silent and clean-close recovery #1394 (and fix(proxy): release idle bridge sessions' account stream leases #1476/fix(proxy): preserve terminal delivery across detach #1527 since), and the branch is conflicting.
2. Arm the quarantine on the shape that actually wedged production (HTTP-bridge reattach stalls with no response.created and needs a client restart to recover #1534): a reattached stream that delivers response events but never gets response.created. The current response_event_count == 0 gate in streaming.py never trips on it.
3. Align the 5s http_responses_session_bridge_response_created_timeout_seconds default with the merged 60s eventless deadline from fix(http-bridge): stabilize silent and clean-close recovery #1394, and integrate the quarantine window with its durable retry circuit instead of a bespoke raw-HTTP fallback timer — false positives at 5s retire healthy sessions and sacrifice prompt-cache affinity.
4. Fix the branch-owned CI failures: test_settings_surface_ratchet (116 > 115 — the new settings need the simplicity-budget discussion per PRINCIPLES.md P2 / Simplicity backlog: settings-surface reduction (164 → ~110 fields) & deferred follow-ups #1340) and regenerate the settings reference for test_generated_settings_reference_matches_code.

If you don't have bandwidth, just say so and we'll take it over with attribution. Absent a reply or a push within a week, we'll assume the latter so the chosen direction doesn't stale out.

[Soju06]

Superseded by #1630, which landed the quarantine direction first-party today (owner decision after no response to the 07-24 review, the 08-04 takeover offer, and the 08-06 revival checklist). Your quarantine-window concept, per-key deadline registry with pruning and size-cap eviction, and event wording were carried forward with attribution — the merge commit credits you via Co-authored-by. Thanks for pioneering the approach; the account-health-neutral session-scoped shape you argued for is exactly what shipped.