[agentserver] Add typed Voice event relay

[knit (knitvoger)]

Why

Enable existing text-in/text-out hosted agents to participate in managed Voice Live
calls without implementing audio, STT, TTS, VAD, or a private WebSocket protocol.
The platform-side Bridge owns media processing and turn-taking; the SDK exposes the
typed text/control surface used by hosted agents.

What

Add the preview azure.ai.agentserver.invocations.voice submodule to the existing
azure-ai-agentserver-invocations Python package. It implements Voice Live Bridge
Protocol 1.0 over /invocations_ws, including:

• typed callbacks for the selected session, user-turn, response, and termination events;
• explicit outbound text/control models and a send-only Session context;
• context-free frame/schema validation and a 1 MiB WebSocket admission limit;
• serialized concurrent WebSocket writes without response or task lifecycle ownership;
• documentation, APIView artifacts, safe model representations, and a runnable sample.

Voice ships inside Invocations; it is not a separate distribution or top-level namespace.

How

• Voice codec, typed models, callbacks, and send-only session context remain under
  azure.ai.agentserver.invocations.voice.
• Invocations owns the shared WebSocket transport behavior: reserved route handling,
  platform identity, W3C context extraction, and first-terminal close diagnostics.
• The codec performs context-free envelope, field, JSON-shape, and size checks;
  the relay retains no dedupe ledger or cross-message protocol state.
• Application code explicitly sends every outbound event and owns response identity,
  generation tasks, cancellation, history, and reconnect restoration.
• admission_timeout_ms remains Bridge-owned, avoiding a competing SDK timer.
• The PR has zero Core diff and depends on azure-ai-agentserver-core>=2.1.0b1.
  Opt-in rides existing untyped agent metadata (voiceLiveCompatible, invocations_ws,
  bridgeProtocolVersion), so no TypeSpec / service-model change is required.

Spec: voice_live_bridge/spec.md · Design: hosted_text_agent_and_voice_live_bridge.md

Architecture and responsibility boundaries

Voice is deliberately a thin typed transport adapter (a "plumber"), not a conversation, response, or application-task runtime.

SDK-owned responsibilities

• Context-free validation and encoding of one frame at a time, plus typed callback dispatch.
• Standard /invocations_ws route wiring while preserving supported Starlette Host and Mount authority.
• Serialized connection writes and an irreversible write gate once termination begins.
• Binding and releasing the thin Session, Foundry request context, and W3C trace context.
• Bounded best-effort transport-close admission/wait and structured close diagnostics.

Application-owned responsibilities

• Response and item identities, model/tool/generation tasks, cancellation, conversation history, and reconnect restoration.
• The work performed by callbacks and any tasks or resources those callbacks create.
• on_connection_terminating is a cooperative synchronous notification that must return promptly; on_disconnect observes an actual peer disconnect. The SDK does not supervise or force-stop application work.

Bridge and infrastructure responsibilities

• The Bridge owns media processing, STT/TTS, VAD, turn-taking, Bridge timers, proactive admission, and cross-message protocol semantics.
• Starlette/Hypercorn own the ASGI transport; the configured OpenTelemetry and logging runtimes own their exporters and handlers. The SDK isolates catchable failures and bounds only its own transport wait; it does not force-stop opaque infrastructure work.

Explicit non-goals

The Voice relay retains no response lifecycle, conversation history, reconnect/replay/retry state, application-task supervisor, or cross-message dedupe ledger.

[github-actions]

Thank you for your contribution knit (@knitvoger)! We will review the pull request and get back to you soon.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
7 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Adds a preview typed Voice Live Bridge relay to the Invocations package.

Changes:

• Adds typed Voice protocol models, validation, callbacks, and serialized sends.
• Adds a runnable full-duplex voice-agent sample and documentation.
• Adds comprehensive codec, host, session, and sample tests.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
.chronus/changes/wujin-voice_submodule-2026-7-12-16-36-34.md	Records the feature.
sdk/agentserver/azure-ai-agentserver-invocations/MANIFEST.in	Packages sample support files.
sdk/agentserver/azure-ai-agentserver-invocations/README.md	Documents the Voice API and sample.
sdk/agentserver/azure-ai-agentserver-invocations/api.md	Adds APIView declarations.
sdk/agentserver/azure-ai-agentserver-invocations/api.metadata.yml	Updates APIView metadata.
sdk/agentserver/azure-ai-agentserver-invocations/pyproject.toml	Adds the Azure Core dependency.
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/__init__.py	Exports the public Voice API.
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_codec.py	Implements frame validation and serialization.
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_models.py	Defines immutable protocol models.
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_session.py	Implements serialized outbound sends.
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py	Implements callback dispatch and transport handling.
sdk/agentserver/azure-ai-agentserver-invocations/samples/basic_voice_agent/README.md	Documents sample setup and deployment.
sdk/agentserver/azure-ai-agentserver-invocations/samples/basic_voice_agent/agent.manifest.yaml	Configures Bridge compatibility.
sdk/agentserver/azure-ai-agentserver-invocations/samples/basic_voice_agent/basic_voice_agent.py	Demonstrates application-owned lifecycle handling.
sdk/agentserver/azure-ai-agentserver-invocations/samples/basic_voice_agent/requirements.txt	Declares the sample dependency.
sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_basic_voice_agent.py	Tests sample lifecycle behavior.
sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_voice_codec.py	Tests codec and model behavior.
sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_voice_host.py	Tests WebSocket dispatch and admission.
sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_voice_session.py	Tests session construction and send serialization.

Suppressed comments (3)

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:235

• This non-text-frame close also returns normally, causing the shared wrapper to record a normal 1000 close instead of 1003. Surface the actual terminal code to the wrapper after sending the close frame.

            frame = raw_message.get("text")
            if frame is None:
                await websocket.close(code=1003, reason="Voice messages must be text frames")
                return

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:240

• After sending the codec-selected close code (including 1009 for oversized frames), returning makes _invoke_user_handler report 1000. Raise WebSocketDisconnect with exc.close_code so operational diagnostics match the close sent to the peer.

            try:
                event = decode_inbound_message(frame)
            except VoiceProtocolError as exc:
                await websocket.close(code=exc.close_code, reason="Invalid Voice message")
                return

sdk/agentserver/azure-ai-agentserver-invocations/api.md:596

• This APIView artifact says reconnect defaults to False, but SessionStart.reconnect is required in _models.py. Reconcile the source signature and regenerate the API artifacts so reviewers and consumers see the actual public constructor contract.

        def __init__(
                id: str, 
                ts: str, 
                protocol_version: str, 
                reconnect: bool = False, 
                response_timeouts: ResponseTimeouts, 
                greeting: str | None, 
                no_input_timeout_ms: int | None, 
                caller: Mapping

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.

Suppressed comments (3)

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:275

• A direct CancelledError while websocket.accept() is pending bypasses this assignment and escapes _ws_endpoint before either finalization path runs. That drops the required first-terminal close diagnostic for handshake cancellations (the wrapped-cancellation test only verifies identity propagation). Record the cancelled 1011 outcome here and re-raise the same cancellation.

        accept_error, voice_session, close_code, handler_exc, pending_error = await self._run_voice_connection_context(
            websocket,
            session_id,
        )

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:351

• These resets happen before _complete_voice_endpoint, so on_connection_terminating, on_disconnect, endpoint error logs, and the structured close event all run without the Foundry request context or extracted W3C context. In particular, an async disconnect callback that performs a Foundry call loses the required call ID even though normal Voice callbacks receive it. Keep the contexts attached through application callback and diagnostic finalization, while retaining the context-free transport-close task.

                platform_token.var.reset(platform_token)
        finally:
            if trace_token is not None:
                trace_token.var.reset(trace_token)

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:442

• A failure from on_connection_terminating is omitted from the structured outcome, so an otherwise normal peer disconnect is emitted with error_code=None even though application cleanup failed. The adjacent disconnect-callback path correctly reports internal_error; include termination_error in the same classification so close telemetry does not record this failure as success.

                error_code="internal_error" if disconnect_error is not None else error_code,

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:828

• SessionEnd is documented as Bridge-initiated session termination, but after dispatching it the receive loop remains active and the Session stays writable. If the peer does not immediately disconnect, later frames and application sends are still accepted and on_connection_terminating never runs. Close the send gate when this event is received and return after its callback so normal endpoint finalization closes the transport.

                callback = self._voice_callbacks.get(event.type)
                if callback is not None:
                    await _await_with_cancellation_guard(callback(session, cast(InboundVoiceMessage, event)))

[Copilot]

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated no new comments.

Suppressed comments (1)

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_models.py:242

• SessionStart is a public constructor, but a caller-supplied dict is stored by reference here. That contradicts the documented immutable caller context: mutating the original or a nested value changes an already-created frozen event. Deep-freeze this field in __post_init__, as the outbound voice mappings already do.

    caller: Mapping[str, Any] | None = None

[github-actions]

[Pilot] PR Pipeline Failure Analysis

A CI pipeline failed on this pull request. Here is an automated analysis of what went wrong and how to get the build green.

What failed

Two validation checks failed in the azure-ai-agentserver-invocations package (build results):

1. CSpell (spell check) — validation: Two unknown words were flagged in sdk/agentserver/azure-ai-agentserver-invocations:

   • _codec.py:77 — unknown word idtoken
   • tests/voice/test_voice_codec.py:121 — unknown word XAPITOKEN

2. Pylint — validation: azure-ai-agentserver-invocations failed with exit code 16 (convention violations) in _session.py:191 (_run_transport_operation):

   • C4739 — missing param operation_coroutine in docstring
   • C4741 — missing return description in docstring
   • C4742 — missing return type in docstring

All other packages (activity, core, optimization, responses) passed pylint cleanly.

Recommended next steps

• Fix CSpell errors: Either correct the spelling in _codec.py and test_voice_codec.py, or add the words (idtoken, XAPITOKEN) to the cspell.yaml allowlist at sdk/agentserver/cspell.yaml. See the spellcheck guide.
• Fix pylint docstring violations in sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_session.py line 191: add operation_coroutine to the Args: section and add a Returns: + :rtype: section to the _run_transport_operation docstring. See [docstring guidelines]((azure.github.io/redacted)
• See the CI troubleshooting guide: https://aka.ms/ci-fix
• Push new commits to address the failures; this comment updates automatically on the next failing run.

Raw pipeline analysis (azsdk ci analyze)

Build: 6701922 Project: public
PipelineUrl: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6701922

--- CSpell errors ---
##[error]sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_codec.py:77:10 - Unknown word (idtoken)
##[error]sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_voice_codec.py:121:30 - Unknown word (XAPITOKEN)
##[error]Spelling errors detected. To correct false positives or learn about spell checking see: https://aka.ms/azsdk/engsys/spellcheck

--- Pylint errors (azure-ai-agentserver-invocations, exit 16) ---
azure/ai/agentserver/invocations/voice/_session.py:191: [C4739(docstring-missing-param), _run_transport_operation] Params missing in docstring: "operation_coroutine"
azure/ai/agentserver/invocations/voice/_session.py:191: [C4741(docstring-missing-return), _run_transport_operation] A return doc is missing in the docstring.
azure/ai/agentserver/invocations/voice/_session.py:191: [C4742(docstring-missing-rtype), _run_transport_operation] A return type is missing in the docstring.

--- Summary ---
/mnt/vss/_work/1/s/sdk/agentserver/azure-ai-agentserver-invocations   pylint  FAIL(16)
Total checks: 5 | Failed: 1 | Worst exit code: 16

Failing checks:
  python - pullrequest [FAILURE] https://dev.azure.com/azure-sdk/29ec6040-b234-4e31-b139-33dc4287b756/_build/results?buildId=6701922
  python - pullrequest (Build Analyze) [FAILURE] https://dev.azure.com/azure-sdk/29ec6040-b234-4e31-b139-33dc4287b756/_build/results?buildId=6701922&view=logs&jobId=b70e5e73-bbb6-5567-0939-8415943fadb9

Copilot detected the failing pipeline and generated the analysis above. To have it attempt a fix automatically, reply with @copilot please fix the failing pipeline on this PR.

Generated by Pipeline Analysis - Next Steps · 39.4 AIC · ⌖ 8.88 AIC · ⊞ 6.6K · ◷

[Copilot]

Pull request overview

Copilot reviewed 20 out of 22 changed files in this pull request and generated no new comments.

Suppressed comments (1)

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:330

• Detach this token through the OpenTelemetry context API rather than reaching into Token.var. The token belongs to the configured runtime, so assuming a contextvars.Token can fail with another runtime and skip context cleanup. This package already uses _otel_context.detach(...) with ValueError handling in azure/ai/agentserver/invocations/_invocation.py:592-595 (and Core does the same in _tracing.py:509-513).

            if trace_token is not None:
                trace_token.var.reset(trace_token)

[Copilot]

Pull request overview

Copilot reviewed 20 out of 22 changed files in this pull request and generated no new comments.

Suppressed comments (1)

sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:330

• attach() returns an OpenTelemetry runtime token, which is opaque and is only guaranteed to work with opentelemetry.context.detach(). Accessing .var assumes the default contextvars runtime and can raise during cleanup (and leak the Voice trace/baggage context) when another supported runtime is configured. Detach through the OpenTelemetry API instead.

                trace_token.var.reset(trace_token)