Skip to content

HTTP bridge: aborted completed-event bookkeeping leaks API-key reservation heartbeat #1594

Description

@Soju06

Summary

When HTTP bridge completed-event bookkeeping aborts (raises or is cancelled) after the terminal request has been popped from session.pending_requests, the request's API-key reservation heartbeat is never cancelled and the reservation is never released.

Details

Ordering (observable on main; surfaced while reviewing #1527 but not introduced by it):

  1. response.completed processing pops the terminal request from session.pending_requests under pending_lock and proceeds to continuity/settlement bookkeeping.
  2. Bookkeeping raises or is cancelled before finalization.
  3. The downstream stream eventually times out and its finally calls _detach_http_bridge_request.
  4. In _detach_http_bridge_request, the request is no longer in session.pending_requests, so detached stays False and the function returns before reaching:
    • self._cancel_request_state_api_key_reservation_heartbeat(request_state)
    • await self._release_websocket_request_state_reservation(request_state)
  5. The reservation heartbeat loop (while not stop_event.is_set(): ...) keeps touching the reservation indefinitely, retaining quota and generating repeated database writes.

The error-path regression added in #1527 (test_http_bridge_stream_waits_only_while_completed_delivery_is_active[error]) proves the client-facing fallback (response.failed + idle timeout), but does not assert reservation settlement — the leak is invisible to that test.

Expected

A request whose completed bookkeeping aborts after the pending pop must still be settled exactly once: heartbeat cancelled, API-key reservation released, and state finalized or restored to an ownership that cleanup can reach.

Notes

  • Related: fix(proxy): preserve terminal delivery across detach #1527 (review comment item 2).
  • Candidate direction: on abort after the claim, either restore the request to a cleanup-reachable state or run the reservation settlement path directly in the abort handler; regression should assert heartbeat cancellation and reservation release on the abort path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions