Session close retains stdio children and per-session upstream caches

[Yatsuiii]

cmcp-gateway version

Reproduced using the original close/rebind path at 56062cb, with those methods loaded unchanged into the local integration harness. Current upstream main 69355037325927f28045686d548d3ea78d80ab6e still contains the same missing cleanup in _session_close() and rebind_session().

Python version

Python 3.13.15.

TEE provider

Local software-only test configuration with mocked attestation and policy. This exercises the real HTTP close endpoint and a real stateful stdio child; no hardware or production deployment was tested.

Reproduction steps

1. Configure a stdio tool whose process keeps an in-memory call counter and returns its PID and counter. Have it answer tools/list with the approved definition.
2. Call the tool through /mcp. Record the child PID and counter.
3. POST /sessions/{internal_session_id}/close and confirm a successful response and a new internal session ID.
4. Call the same tool again through /mcp.

Expected behavior

Closing the session terminates its stdio child. The next session creates a fresh child and performs its own provenance/drift checks.

This follows the explicit design in #485: one child per server per session. docs/spec/stdio-transport.md also says session close terminates the child. Reuse within one session is expected; retention across close contradicts that lifetime.

Actual behavior

The close endpoint rotates session and audit references through rebind_session(), which recreates call logs but retains _stdio_servers, _http_clients, _provenance, and _drift_checked. It never calls the proxy's cleanup method.

The child remains alive and the new session reuses its accumulated state. The provenance and drift caches also remain populated. This demonstrates retained process state and skipped cache reset, not a production cross-user exploit.

Relevant logs or TRACE Claim output

close_status:        200
session_changed:     true
child_alive:         true
provenance_retained: true
drift_cache_retained:true
before PID:counter:  15:1
after PID:counter:   15:2

Related fixes #502 and #587 separate cache entries for different upstream identities. They do not reset resources at the session boundary.

Proposed approach and intent to take this

I would like to take this fix. My proposed scope is:

• Make close a coordinated lifecycle transition: stop admitting calls to the old session, drain existing calls with a bounded policy, then finalize and rotate.
• Close session-owned stdio and HTTP resources and reset provenance/drift caches before admitting calls to the next session.
• Preserve resource ownership during failed or cancelled cleanup, and ensure a finalized session cannot resume accepting calls after a failed rotation.
• Coordinate concurrent first-use child creation so every spawned child remains tracked for cleanup.

The drain timeout and treatment of calls still running at that deadline need an explicit contract. Cancellation must preserve existing terminal audit handling rather than silently dropping a call from the closing claim.

Acceptance coverage would include the real endpoint/stateful-child regression, in-flight call outcomes in the old session's audit, calls arriving during close, concurrent closes, cleanup failure/cancellation, and concurrent child creation.

A local implementation is under review; it is not ready for a PR yet. I will keep this separate from the execution-correlation foundation. Please assign this to me if that scope fits.

Conformance test ID (if applicable)

No dedicated conformance ID identified. The existing stdio session-lifetime contract is the basis for the regression.

[imran-siddique]

Confirmed, and the reproduction holds at current main 14fef43. Assigning this to you.

rebind_session (proxy.py:291) resets _session, _audit and the two call logs, and nothing else. aclose() (proxy.py:399) does exactly what close needs and has zero call sites anywhere in the repo, tests included, which is why this survived: nothing ever asserted the child dies.

Three things before you start.

This is a stated guarantee, not an unspecified corner. The comment above the cache you found spells out the invariant it fails to hold: "never pooled across sessions: a server that holds anything in memory would carry it from one agent's session into the next, and the audit chain cannot see that happen" (proxy.py:276). docs/spec/stdio-transport.md line 12 says session close terminates the child, and the lifecycle note at line 112 says children are "closed with that session". The fix restores something the spec already promises.

There are two missing call sites, not one. Starlette(...) at server.py:372 is constructed with no lifespan and no shutdown handler, so aclose() is never reached on gateway exit either. Children leak when the process ends, from the same cause. Please bring that into scope while you are in here.

The drain contract, since you asked for a decision on it. Drain and cleanup go before the claim is signed. _session_close signs at server.py:834 and rotates after. A call cancelled at the drain deadline is still a call this gateway made, so its terminal outcome belongs in the claim that closes the session. Cleanup placed after signing yields a signed claim that omits it, and a claim missing a call it made costs more than a slow close does. So: stop admitting, drain to the deadline, record every terminal outcome including cancellations, sign, then release resources and rotate. The deadline value is yours to propose. Make it configurable, and pick a default you can argue for.

Your acceptance list covers the right cases. Add one to it: a close that leaves no child running, asserted on the PID rather than on the absence of an error.

Timing: I would like the PR open by 25 September. If it has not opened by then I will land the two aclose() calls to stop the retention, and leave the drain and finalization contract with you.

Keeping this separate from the execution-correlation work is the right call.