You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[FEATURE]: Chat completions run on the runtime #782
On nightly, POST /v1/chat/completions prepares an agent, begins a run, takes the run's event receiver with AgentRun::take_agent_events(), reads content from the StreamItem stream, projects the rest through run_event_sse, and on disconnect, timeout, or shutdown cancels through the run's token — all inside the request's task. It is the reference implementation of a run, and it is why a run cannot outlive a request.
Move it onto the runtime. The handler attaches to the session as a claiming observer with presence, starts a run, projects the subscription to the wire it serves today, and detaches when the client goes. Cancel-on-unclaimed (#781) is today's behavior exactly.
Goals
The handler attaches to the session first — claiming, present, cursor Live ([FEATURE]: Observers declare whether they collect, claim, or attend #781) — then describes the run — agent, the request's forwarded headers and client tools, the messages, the chat liveness default — and the runtime prepares and starts it ([FEATURE]: A runtime that owns sessions and their runs #780). Because the claim is on the session, the handler is the claimant from the run's first event and Started is the first thing it streams. It never holds a PreparedAgent or an AgentRun.
The handler is three things in a row: the Subscription, project_sse, and Body::from_stream. No chunk channel between the run and the response body, no result oneshot for the collected path, no channel end anywhere in the handler but the stream inside the subscription.
One projection over SessionEvent. SessionEventPayload::Agent goes through run_event_sse as it stands on nightly; content comes from the journal's TextDelta / Reasoning / Completed rather than from StreamItem, which makes this handler the first consumer of the copy tee_content makes; a terminal lifecycle event ends the stream the way the item stream ending does now. If [FEATURE]: SSE Producer from Agent Events #576 has landed, its producer is that projection.
aura.session_info and aura.mcp_status, which the handler emits before the agent produces anything, come from the run's Started lifecycle event and the Agent(McpStatus) the run's writer task appends right after it from the prepared agent's server status ([FEATURE]: A runtime that owns sessions and their runs #780), so a late observer sees them too.
Disconnect, timeout, and shutdown become a detach with a reason. The runtime's cancel-on-unclaimed does what the handler's disconnect arms do now, from one place.
A Gap on the handler's subscription is an error, never a skip. Today a slow client back-pressures the LLM stream; under the journal the handler is one subscriber, and lagging would hand the user a hole in the answer. The handler ends the stream with an error frame — the same outcome a2a-server gives a subscriber that "fell behind active execution" — and the broadcast capacity is sized so a live client does not lag short of pathology.
The non-streaming path starts a run the same way and drains it.
The orchestrator's StreamItem channel (orchestration::factory) is left without a consumer, since content now comes from the journal. Removing it, and ToolCallObserver with it, is [FEATURE]: Remove the request-scoped broker path #627's; this issue does not touch orchestration.
Data structures
Proposed. The projection is a function; the only new type is what it returns.
/// Projects one session event onto the OpenAI-plus-`aura.*` wire. `None` for/// events the wire has no frame for (an observer attaching, a liveness decision,/// anything belonging to a different run of the session).pubfnproject_sse(event:&SessionEvent,run:&RunId,// the run this response serves; other runs' events are Nonecorrelation:&CorrelationContext,// session id, trace id — applied here, as todaycustom_events:bool,// AURA_CUSTOM_EVENTS; approvals ignore it) -> Option<SseFrame>;pubenumSseFrame{Chunk(ChatCompletionChunk),// from Agent(TextDelta | Reasoning | Completed)Custom(AuraStreamEvent),// from Agent(ToolStart | ToolProgress | ...) via run_event_sseDone,// from Lifecycle(Finished | Cancelled | Parked)Error(StreamError),// from Lifecycle(Failed), and from JournalItem::Gap}
Additional Context
Session handling is unchanged: chat_session_id from the request or generated, returned in X-Chat-Session-Id. It is the session the handler attaches to and the key of the stream the run is written to (#779), which is what makes a later attach by session or by run (#785) possible.
Summary
On nightly,
POST /v1/chat/completionsprepares an agent, begins a run, takes the run's event receiver withAgentRun::take_agent_events(), reads content from theStreamItemstream, projects the rest throughrun_event_sse, and on disconnect, timeout, or shutdown cancels through the run's token — all inside the request's task. It is the reference implementation of a run, and it is why a run cannot outlive a request.Move it onto the runtime. The handler attaches to the session as a claiming observer with presence, starts a run, projects the subscription to the wire it serves today, and detaches when the client goes. Cancel-on-unclaimed (#781) is today's behavior exactly.
Goals
Live([FEATURE]: Observers declare whether they collect, claim, or attend #781) — then describes the run — agent, the request's forwarded headers and client tools, the messages, thechatliveness default — and the runtime prepares and starts it ([FEATURE]: A runtime that owns sessions and their runs #780). Because the claim is on the session, the handler is the claimant from the run's first event andStartedis the first thing it streams. It never holds aPreparedAgentor anAgentRun.Subscription,project_sse, andBody::from_stream. No chunk channel between the run and the response body, no result oneshot for the collected path, no channel end anywhere in the handler but the stream inside the subscription.SessionEvent.SessionEventPayload::Agentgoes throughrun_event_sseas it stands on nightly; content comes from the journal'sTextDelta/Reasoning/Completedrather than fromStreamItem, which makes this handler the first consumer of the copytee_contentmakes; a terminal lifecycle event ends the stream the way the item stream ending does now. If [FEATURE]: SSE Producer from Agent Events #576 has landed, its producer is that projection.aura.session_infoandaura.mcp_status, which the handler emits before the agent produces anything, come from the run'sStartedlifecycle event and theAgent(McpStatus)the run's writer task appends right after it from the prepared agent's server status ([FEATURE]: A runtime that owns sessions and their runs #780), so a late observer sees them too.Gapon the handler's subscription is an error, never a skip. Today a slow client back-pressures the LLM stream; under the journal the handler is one subscriber, and lagging would hand the user a hole in the answer. The handler ends the stream with an error frame — the same outcomea2a-servergives a subscriber that "fell behind active execution" — and the broadcast capacity is sized so a live client does not lag short of pathology.ActiveRequestTrackerretired; the runtime's drain ([FEATURE]: A runtime that owns sessions and their runs #780) is what shutdown waits on. Whatever still needs the tracker is a gap in the runtime, not a reason to keep it.StreamItemchannel (orchestration::factory) is left without a consumer, since content now comes from the journal. Removing it, andToolCallObserverwith it, is [FEATURE]: Remove the request-scoped broker path #627's; this issue does not touch orchestration.Data structures
Proposed. The projection is a function; the only new type is what it returns.
Additional Context
Session handling is unchanged:
chat_session_idfrom the request or generated, returned inX-Chat-Session-Id. It is the session the handler attaches to and the key of the stream the run is written to (#779), which is what makes a later attach by session or by run (#785) possible.Searched Issues
Code of Conduct