feat: support for agent v1 #520
Conversation
## Walkthrough
This update introduces a comprehensive refactor and enhancement of the agent-related configuration and client code within the Deepgram SDK. Key changes include the renaming and restructuring of configuration classes (e.g., `SettingsConfigurationOptions` to `SettingsOptions`, `UpdateInstructionsOptions` to `UpdatePromptOptions`), the introduction of new provider abstractions (`ListenProvider`, `SpeakProvider`, `ThinkProvider`), and the addition of an `Endpoint` class for custom HTTP endpoints. Enum members and event names are updated to match the new terminology. Example scripts are revised to use the new configuration schema, and a new example demonstrates agent usage without a microphone. Documentation and ignore rules are also updated.
## Changes
| Files/Paths | Change Summary |
|-----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `.github/CONTRIBUTING.md` | Added a "Building Locally" section with step-by-step instructions for building and installing the package using `pipenv`, including running an example script. |
| `.gitignore` | Added ignore patterns for `chatlog.txt` and `output_*.wav` files generated by examples. |
| `examples/requirements-examples.txt` | Added `requests` package to the example requirements. |
| `deepgram/__init__.py`, `deepgram/client.py`, `deepgram/clients/__init__.py`,<br>`deepgram/clients/agent/__init__.py`, `deepgram/clients/agent/v1/__init__.py`,<br>`deepgram/clients/agent/v1/websocket/__init__.py` | Refactored imports to replace `SettingsConfigurationOptions`/`UpdateInstructionsOptions` with `SettingsOptions`/`UpdatePromptOptions`, removed `Provider`/`Context`, and added new provider classes and `Endpoint`. |
| `deepgram/clients/agent/enums.py` | Renamed enum members: `SettingsConfiguration` → `Settings`, `UpdateInstructions` → `UpdatePrompt` in `AgentWebSocketEvents`. Removed `FunctionCalling` member. |
| `deepgram/clients/agent/v1/websocket/options.py` | Major refactor: replaced/renamed configuration dataclasses (`SettingsConfigurationOptions` → `SettingsOptions`, etc.), introduced new provider classes (`ListenProvider`, `SpeakProvider`, `ThinkProvider`), added `Endpoint` and `CartesiaVoice`, restructured `Listen`, `Speak`, `Think`, `Agent`, and `Language` classes, and updated deserialization logic. |
| `deepgram/clients/agent/v1/websocket/async_client.py`,<br>`deepgram/clients/agent/v1/websocket/client.py` | Updated all type annotations, method signatures, and settings handling to use `SettingsOptions` instead of `SettingsConfigurationOptions`, changed endpoint to `"v1/agent/converse"`, updated nested attribute access for provider settings, removed `FunctionCalling` handling, and revised logging/error messages to new terminology. |
| `deepgram/clients/agent/client.py` | Updated import aliases to match new provider and options class names. |
| `deepgram/clients/agent/v1/websocket/response.py` | Removed `FunctionCalling` dataclass and related comments. Renamed `WelcomeResponse.session_id` to `request_id`. Added note that `AgentThinkingResponse` is only received if `experimental` flag is true. |
| `examples/agent/simple/main.py`, `examples/agent/async_simple/main.py` | Refactored example scripts to use `SettingsOptions` and updated configuration structure to match new schema (e.g., nested `provider`, renamed `instructions` to `prompt`, added `greeting`, `language`, etc.). Removed `FunctionCalling` event handling. |
| `examples/agent/no_mic/main.py` | Added a new example script demonstrating agent usage without microphone input, including detailed agent configuration, event handling, audio file streaming, and chat logging. |
## Sequence Diagram(s)
```mermaid
sequenceDiagram
participant UserScript
participant DeepgramClient
participant AgentWebSocketClient
participant AgentProvider
UserScript->>DeepgramClient: Initialize with API key and options
UserScript->>AgentWebSocketClient: Connect with SettingsOptions
AgentWebSocketClient->>AgentProvider: Send Settings (with nested provider config)
AgentProvider-->>AgentWebSocketClient: Acknowledge settings
UserScript->>AgentWebSocketClient: Register event handlers
UserScript->>AgentWebSocketClient: Stream audio data (if applicable)
AgentWebSocketClient->>AgentProvider: Transmit audio/data/events
AgentProvider-->>AgentWebSocketClient: Emit events (audio, text, state)
AgentWebSocketClient-->>UserScript: Invoke registered handlers with event data
UserScript->>AgentWebSocketClient: Close connectionPossibly related PRs
Suggested reviewers
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🔭 Outside diff range comments (1)
deepgram/clients/agent/v1/websocket/client.py (1)
181-189: 🛠️ Refactor suggestion
start()rejectsNoneforoptions, breaking previous behaviour
optionsis declared asOptional[SettingsOptions], but the subsequentisinstance/elifchain ends with an unconditionalelse: raise DeepgramError("Invalid options type").
PassingNone– which used to be valid for callers that wanted to send the Settings message later – will now raise an exception and make the call site fail at runtime.If
Noneis no longer acceptable, the type hint should be changed toSettingsOptions(non-optional) and the docstring updated.
IfNoneshould stay legal, add an early guard and skip the type-based conversion instead of raising.- options: Optional[SettingsOptions] = None, + # `None` means: “I will send the Settings message manually later”. + options: Optional[SettingsOptions] = None, ... - else: - raise DeepgramError("Invalid options type") + elif options is None: + self._logger.notice("No SettingsOptions supplied – caller will send later") + self._settings = None + else: + raise DeepgramError( + f"Invalid options type {type(options).__name__}; " + "expected SettingsOptions | dict | str | None" + )Also applies to: 199-204
🧹 Nitpick comments (6)
deepgram/clients/__init__.py (1)
367-382: Provider model refactored with granular provider typesThe agent configuration has been refactored from a general-purpose
Providerclass to specific provider types (ListenProvider,SpeakProvider,ThinkProvider) and a newEndpointabstraction.This change provides better separation of concerns and more explicit configuration options for each capability of the agent. The granular provider model allows for more targeted configuration and clearer API boundaries.
deepgram/clients/agent/v1/websocket/client.py (2)
229-231: Safety check can dereferenceNonewhenmodelis missing
self._settings.agent.listen.provideris guaranteed by defaults, butmodel
may still beNoneif the caller intentionally leaves it unset (e.g. to let
the server decide).
The current condition:if self._settings.agent.listen.provider.keyterms is not None and \ self._settings.agent.listen.provider.model is not None and \ not self._settings.agent.listen.provider.model.startswith("nova-3"):will raise
AttributeErrorifprovideritself isNone(possible if a
malformed payload sneaks throughfrom_dict). A defensive guard eliminates the
risk.-if self._settings.agent.listen.provider.keyterms is not None and \ - self._settings.agent.listen.provider.model is not None and \ - not self._settings.agent.listen.provider.model.startswith("nova-3"): +listen_provider = getattr(self._settings.agent.listen, "provider", None) +if ( + listen_provider + and listen_provider.keyterms + and listen_provider.model + and not listen_provider.model.startswith("nova-3") +):
191-197: Consider redacting sensitive data in INFO logs
self._logger.info("settings: %s", options)prints the entire Settings object,
which may contain organisation-specific prompts, model identifiers or future
secret fields. Moving this to DEBUG, or implementing a redaction helper (e.g.
masking long strings), protects users who run the SDK with verbose logging in
production.deepgram/clients/agent/v1/websocket/options.py (3)
215-223:SpeakProvider.__getitem__assumes lists forvoice/language, conflicting with schema
voiceis a singularCartesiaVoiceinstance, andlanguage/
language_codeare strings, but the getter transforms them into lists. This
breaks symmetry (SpeakProvider.from_dict(...).voicereturnslist, not
CartesiaVoice) and surprises downstream consumers.- if "voice" in _dict: - _dict["voice"] = [ - CartesiaVoice.from_dict(voice) for voice in _dict["voice"] - ] - if "language" in _dict: - _dict["language"] = [str(language) for language in _dict["language"]] + if "voice" in _dict and isinstance(_dict["voice"], dict): + _dict["voice"] = CartesiaVoice.from_dict(_dict["voice"])(The same pattern occurs in
Think,Listen, andSpeakclasses and could be
refactored via a helper.)
78-81:Endpoint.methodisOptional[str]but default is"POST"Either drop
Optional(preferred) or make the defaultNone. Staying
consistent avoids accidentalNoneassignment later.
364-370:Languagedataclass shadows the built-intypekeywordAlthough legal, using the attribute name
typeinvites confusion with
Python’s built-in. Consider renaming tocodeorlangfor clarity.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (14)
.github/CONTRIBUTING.md(1 hunks)deepgram/__init__.py(1 hunks)deepgram/client.py(1 hunks)deepgram/clients/__init__.py(1 hunks)deepgram/clients/agent/__init__.py(1 hunks)deepgram/clients/agent/client.py(2 hunks)deepgram/clients/agent/enums.py(1 hunks)deepgram/clients/agent/v1/__init__.py(1 hunks)deepgram/clients/agent/v1/websocket/__init__.py(1 hunks)deepgram/clients/agent/v1/websocket/async_client.py(7 hunks)deepgram/clients/agent/v1/websocket/client.py(7 hunks)deepgram/clients/agent/v1/websocket/options.py(7 hunks)examples/agent/async_simple/main.py(2 hunks)examples/agent/simple/main.py(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
deepgram/clients/agent/v1/websocket/__init__.py (2)
Learnt from: dvonthenen
PR: deepgram/deepgram-python-sdk#426
File: deepgram/clients/listen/v1/websocket/__init__.py:8-8
Timestamp: 2024-07-01T19:21:39.778Z
Learning: Unused imports in `deepgram/clients/listen/v1/websocket/__init__.py` are retained to maintain backward compatibility and should not be flagged for removal in reviews.
Learnt from: dvonthenen
PR: deepgram/deepgram-python-sdk#426
File: deepgram/clients/listen/v1/websocket/__init__.py:8-8
Timestamp: 2024-10-09T02:19:46.086Z
Learning: Unused imports in `deepgram/clients/listen/v1/websocket/__init__.py` are retained to maintain backward compatibility and should not be flagged for removal in reviews.
🧬 Code Graph Analysis (7)
deepgram/client.py (1)
deepgram/clients/agent/v1/websocket/options.py (13)
SettingsOptions(373-404)UpdatePromptOptions(411-417)Listen(257-270)ListenProvider(145-160)Speak(274-294)SpeakProvider(177-223)Header(21-27)Function(94-127)Think(227-253)ThinkProvider(164-173)Agent(298-318)Audio(346-360)Endpoint(73-90)
deepgram/clients/agent/v1/websocket/__init__.py (1)
deepgram/clients/agent/v1/websocket/options.py (13)
SettingsOptions(373-404)UpdatePromptOptions(411-417)Listen(257-270)ListenProvider(145-160)Speak(274-294)SpeakProvider(177-223)Header(21-27)Function(94-127)Think(227-253)ThinkProvider(164-173)Agent(298-318)Audio(346-360)Endpoint(73-90)
deepgram/__init__.py (1)
deepgram/clients/agent/v1/websocket/options.py (22)
SettingsOptions(373-404)UpdatePromptOptions(411-417)UpdateSpeakOptions(424-430)InjectAgentMessageOptions(437-443)FunctionCallResponse(450-457)AgentKeepAlive(464-469)Listen(257-270)ListenProvider(145-160)Speak(274-294)SpeakProvider(177-223)Header(21-27)Item(31-37)Properties(41-52)Parameters(56-69)Function(94-127)Think(227-253)ThinkProvider(164-173)Agent(298-318)Input(322-328)Output(332-342)Audio(346-360)Endpoint(73-90)
deepgram/clients/agent/v1/websocket/async_client.py (3)
deepgram/clients/agent/v1/websocket/options.py (3)
SettingsOptions(373-404)UpdatePromptOptions(411-417)check(391-404)deepgram/utils/verboselogs/__init__.py (1)
notice(150-153)deepgram/clients/common/v1/websocket_response.py (1)
ErrorResponse(41-51)
deepgram/clients/agent/v1/__init__.py (1)
deepgram/clients/agent/v1/websocket/options.py (13)
SettingsOptions(373-404)UpdatePromptOptions(411-417)Listen(257-270)ListenProvider(145-160)Speak(274-294)SpeakProvider(177-223)Header(21-27)Function(94-127)Think(227-253)ThinkProvider(164-173)Agent(298-318)Audio(346-360)Endpoint(73-90)
deepgram/clients/agent/v1/websocket/client.py (3)
deepgram/clients/agent/v1/websocket/options.py (3)
SettingsOptions(373-404)UpdatePromptOptions(411-417)check(391-404)deepgram/utils/verboselogs/__init__.py (1)
notice(150-153)deepgram/clients/common/v1/websocket_response.py (1)
ErrorResponse(41-51)
deepgram/clients/agent/v1/websocket/options.py (2)
deepgram/clients/common/v1/shared_response.py (1)
BaseResponse(16-44)deepgram/clients/agent/enums.py (1)
AgentWebSocketEvents(10-37)
🪛 GitHub Actions: Check - static
deepgram/clients/agent/v1/websocket/options.py
[error] 136-136: mypy: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
[error] 139-139: mypy: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
🔇 Additional comments (24)
.github/CONTRIBUTING.md (1)
51-65: Great addition of building documentation!The new "Building Locally" section provides clear step-by-step instructions for contributors to build and test the package. This is particularly valuable given the extensive refactoring of agent-related classes and options introduced in this PR.
deepgram/clients/agent/enums.py (1)
32-33: Renamed WebSocket events for consistency with option classesThese enum value renamings align with the broader restructuring of agent configuration classes in this PR:
SettingsConfiguration→Settings(matchesSettingsOptions)UpdateInstructions→UpdatePrompt(matchesUpdatePromptOptions)This is a breaking change as noted in the PR description, but improves API consistency.
deepgram/client.py (1)
350-352: Updated imports to match agent v1 provider architectureThese import changes reflect the significant restructuring of the agent configuration model:
Renamed options classes:
SettingsConfigurationOptions→SettingsOptionsUpdateInstructionsOptions→UpdatePromptOptionsReplaced generic
ProviderandContextwith specialized provider classes:
ListenProvider,SpeakProvider,ThinkProvider- Added
EndpointabstractionThe new structure enables more granular configuration of agent capabilities through specialized provider classes.
Also applies to: 358-360, 367-368, 372-373
deepgram/clients/agent/v1/websocket/__init__.py (1)
29-30: Updated imports to expose new agent v1 classesThese import changes are consistent with the broader refactoring in this PR and expose the restructured agent configuration classes to users of this namespace:
Renamed option classes:
SettingsConfigurationOptions→SettingsOptionsUpdateInstructionsOptions→UpdatePromptOptionsAdded specialized provider classes:
ListenProvider,SpeakProvider,ThinkProviderEndpointabstractionThese changes maintain consistency with the new agent v1 architecture throughout the codebase.
Also applies to: 37-39, 46-47, 51-52
deepgram/clients/agent/__init__.py (3)
34-36: Class renames align with restructured agent configuration modelThe renaming from
SettingsConfigurationOptionstoSettingsOptionsandUpdateInstructionsOptionstoUpdatePromptOptionsrepresents part of a broader restructuring of the agent configuration API.
42-45: Provider-specific classes replace generic ProviderThe introduction of specialized provider classes (
ListenProvider,SpeakProvider,ThinkProvider) replaces the previous genericProviderclass, allowing for more granular configuration options specific to each agent capability.Also applies to: 51-52
56-57: Added Endpoint abstraction supports custom endpoint configurationThe new
Endpointclass provides a formal way to configure custom endpoints for agent interactions.deepgram/clients/agent/client.py (2)
33-36: Updates to import aliases maintain API consistencyThe import aliases from
.v1have been updated to reflect the renamed classes and new provider types, ensuring consistency with the broader API restructuring.Also applies to: 41-44, 50-51, 55-56
82-84: Export aliases updated to match restructured importsThe export aliases at the bottom of the file have been consistently updated to match the restructured imports from the
.v1module.Also applies to: 90-92, 99-100, 104-104
examples/agent/async_simple/main.py (2)
14-15: Updated import to use new SettingsOptions classThe import statement has been updated to use the new
SettingsOptionsclass, which replacesSettingsConfigurationOptions.
113-121: Example updated to use new agent configuration structureThe example demonstrates the new provider-based configuration structure:
- Uses nested
providerproperties to access model configuration- Renames
instructionstopromptto match the new API terminology- Adds new configurations like
greeting,listen.provider.keyterms, and top-levellanguageThese changes showcase how to correctly use the new agent configuration model.
deepgram/__init__.py (3)
336-338: Updated top-level exports for agent configuration optionsThe exports have been updated to use the new class names
SettingsOptionsandUpdatePromptOptions, ensuring consistency with the new API structure.
344-347: Added provider-specific classes and endpoint abstraction to exportsThe specialized provider classes (
ListenProvider,SpeakProvider,ThinkProvider) and theEndpointabstraction have been added to the exports, making them available through the top-level package.Also applies to: 353-354, 358-359
334-359: Consider adding migration documentation for this breaking changeWhile the code changes are consistent and complete, the introduction of provider-based nesting and renamed fields constitutes a breaking change that will require client code updates.
Consider adding migration guides or documentation that explains how to update from the previous API to the new structure, especially highlighting the path changes (e.g.,
think.model→think.provider.model) and renamed fields (e.g.,instructions→prompt).examples/agent/simple/main.py (1)
11-11: Import updated to use new SettingsOptions classThe agent configuration class has been renamed from
SettingsConfigurationOptionstoSettingsOptions.deepgram/clients/agent/v1/__init__.py (2)
38-40: Core settings classes renamedThe agent settings classes have been renamed for clarity and consistency:
SettingsConfigurationOptions→SettingsOptionsUpdateInstructionsOptions→UpdatePromptOptions
46-61: Provider model refactored with explicit provider typesThe agent architecture has been refactored to use specific provider types instead of a general-purpose
Provider:
ListenProvider: Manages speech recognition configurationSpeakProvider: Manages speech synthesis configurationThinkProvider: Manages language model configurationEndpoint: New abstraction for custom API integrationsThis refactoring improves code organization and API clarity while allowing for more granular configuration of each agent capability.
deepgram/clients/agent/v1/websocket/async_client.py (7)
34-40: Import statements updated to reflect renamed classesImports have been updated to use the new class names:
SettingsOptionsreplacesSettingsConfigurationOptionsUpdatePromptOptionsreplacesUpdateInstructionsOptions
82-82: Class attribute type updatedThe
_settingsattribute type has been updated toSettingsOptions.
182-182: Method signature updatedThe
startmethod parameter type has been updated to useSettingsOptions.
198-201: Settings validation using updated class typeThe instance check and validation now use
SettingsOptions.
216-225: Settings object instantiation updatedThe code for instantiating settings objects from different input types has been updated to use
SettingsOptions.
228-229: Validation logic updated for nested provider structureThe validation logic for keyterms has been updated to check through the nested provider structure:
- Now validates using
self._settings.agent.listen.provider.keytermsandself._settings.agent.listen.provider.modelThis ensures that keyterms validation works with the new nested configuration structure.
280-288: Log message and error message references updatedReferences to "ConfigurationSettings" have been changed to "Settings" in log messages and error descriptions.
This maintains consistency with the class renaming from
SettingsConfigurationOptionstoSettingsOptions.
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
examples/agent/no_mic/output_0.wavis excluded by!**/*.wav
📒 Files selected for processing (2)
examples/agent/no_mic/chatlog.txt(1 hunks)examples/agent/no_mic/main.py(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- examples/agent/no_mic/chatlog.txt
🔇 Additional comments (4)
examples/agent/no_mic/main.py (4)
10-21: Replace ad-hoc prints & unused imports with structured logging
verboselogsis imported but never used, and a series ofprint()statements are scattered throughout the file for diagnostic purposes. Prefer the std-libloggingmodule (orverboselogsif you really need verbose levels) so that:
- log levels can be configured from the command line / env vars
- output can be redirected or silenced by downstream users
- unit tests aren’t polluted with console noise
-import requests -import wave -import io -import time -from datetime import datetime -from deepgram.utils import verboselogs - -# Add debug prints for imports -print("Checking imports...") +import logging +import requests +import wave +import io +import time +from datetime import datetime + +# Configure once, near the top-level of your application +logging.basicConfig(level=logging.INFO) +log = logging.getLogger(__name__) + +# Example replacement +log.debug("Imports successful")Later
print(...)calls can be replaced withlog.debug/info/warning/error(...)as appropriate.
[ suggest_optional_refactor ]
57-77:convert_audio_to_linear16neither re-samples nor is it ever invokedThe helper promises “convert … to linear16” but:
- It simply writes/reads the same frames – no resampling or format conversion is done.
- The function is never called, leaving dead code in the example.
If you genuinely need resampling, use a DSP library (e.g.
pydub,scipy,soundfile) or the Deepgram SDK’s built-in conversion. Otherwise, remove the function to reduce cognitive load.[ suggest_optional_refactor ]
226-235: Download loop lacks basic robustness
response.raise_for_status()is omitted – silent HTTP 4xx/5xx will yield empty audio.- The stream is never closed (
with requests.get(..., stream=True) as response:).- If the file is large, you may want back-pressure or to respect
response.iter_content(None)when the WebSocket is paused.-response = requests.get(url, stream=True) -for chunk in response.iter_content(chunk_size=8192): +with requests.get(url, stream=True, timeout=15) as response: + response.raise_for_status() + for chunk in response.iter_content(chunk_size=8192): if chunk: dg_connection.send(chunk)[ suggest_essential_refactor ]
234-241: Arbitrarytime.sleep(5)may end the script before the agent finishesA fixed delay is brittle – network latency and model inference time vary.
Instead, either:
- Wait for a specific terminating event (
AgentAudioDone+Close) before exiting, or- Call a (hypothetical)
dg_connection.wait_until_closed()if provided by the SDK.[ suggest_optional_refactor ]
There was a problem hiding this comment.
Actionable comments posted: 6
🔭 Outside diff range comments (4)
deepgram/clients/agent/v1/websocket/options.py (4)
312-317:⚠️ Potential issueFix inconsistent item handling in
Agent.__getitem__methodThe
listen,think, andspeakfields are defined as single objects, but the getter treats them as lists, which will cause a TypeError.- if "listen" in _dict: - _dict["listen"] = [Listen.from_dict(listen) for listen in _dict["listen"]] - if "think" in _dict: - _dict["think"] = [Think.from_dict(think) for think in _dict["think"]] - if "speak" in _dict: - _dict["speak"] = [Speak.from_dict(speak) for speak in _dict["speak"]] + if "listen" in _dict and isinstance(_dict["listen"], dict): + _dict["listen"] = Listen.from_dict(_dict["listen"]) + if "think" in _dict and isinstance(_dict["think"], dict): + _dict["think"] = Think.from_dict(_dict["think"]) + if "speak" in _dict and isinstance(_dict["speak"], dict): + _dict["speak"] = Speak.from_dict(_dict["speak"])
356-359:⚠️ Potential issueFix inconsistent item handling in
Audio.__getitem__methodThe
inputandoutputfields are defined as single objects, but the getter treats them as lists, which will cause a TypeError.- if "input" in _dict: - _dict["input"] = [Input.from_dict(input) for input in _dict["input"]] - if "output" in _dict: - _dict["output"] = [Output.from_dict(output) for output in _dict["output"]] + if "input" in _dict and isinstance(_dict["input"], dict): + _dict["input"] = Input.from_dict(_dict["input"]) + if "output" in _dict and isinstance(_dict["output"], dict): + _dict["output"] = Output.from_dict(_dict["output"])
116-118:⚠️ Potential issueFix inconsistent item handling in
Function.__getitem__forparametersThe
parametersfield is defined asOptional[Parameters](a single object), but the getter treats it as a list, which will cause a TypeError.- if "parameters" in _dict: - _dict["parameters"] = [ - Parameters.from_dict(parameters) for parameters in _dict["parameters"] - ] + if "parameters" in _dict and isinstance(_dict["parameters"], dict): + _dict["parameters"] = Parameters.from_dict(_dict["parameters"])
50-52: 💡 Verification agent🧩 Analysis chain
Fix inconsistent item handling in
Properties.__getitem__methodThe
itemfield appears to be a single object of typeItem, but the getter treats it as a list. Consider whether this is intentional or if it needs correction.
🏁 Script executed:
#!/bin/bash # Search for occurrences where Properties.item is used to understand the expected type rg -A 3 -B 3 "Properties.*item" # Check if the API serializes the 'item' field as an array or a single object rg -A 5 -B 5 "\"item\":" --type jsonLength of output: 70
🏁 Script executed:
#!/bin/bash # Confirm the target file exists ls -l deepgram/clients/agent/v1/websocket/options.py # Show the Properties class definition in context rg -n 'class Properties' deepgram/clients/agent/v1/websocket/options.py -A20 -B5 # Show the __getitem__ method implementation and surrounding lines rg -n 'def __getitem__' deepgram/clients/agent/v1/websocket/options.py -A10 -B5Length of output: 7668
Align
itemtype with its list handling inPropertiesThe docstring and
__getitem__implementation treatitemas a list, but the dataclass field is declared as a singleItem. To resolve this inconsistency:• In
deepgram/clients/agent/v1/websocket/options.py(line 46), changeitem: Itemto
from typing import List item: List[Item]• Ensure you import
Listfromtypingat the top of the file.
♻️ Duplicate comments (2)
deepgram/clients/agent/v1/websocket/options.py (2)
123-127:⚠️ Potential issueFix inconsistent item handling in
Function.__getitem__methodThe
endpointfield is defined asOptional[Endpoint]but the getter treats it as a list. This will fail withTypeError: 'Endpoint' object is not iterablewhen deserializing.- if "endpoint" in _dict: - _dict["endpoint"] = [ - Endpoint.from_dict(endpoint) for endpoint in _dict["endpoint"] - ] + if "endpoint" in _dict and isinstance(_dict["endpoint"], dict): + _dict["endpoint"] = Endpoint.from_dict(_dict["endpoint"])
136-141:⚠️ Potential issueFix type annotations in
CartesiaVoiceclass
modeandidare typed asstrbut default toNone, causing mypy errors as shown in the pipeline failures.- mode: str = field( + mode: Optional[str] = field( default=None, metadata=dataclass_config(exclude=lambda f: f is None) ) - id: str = field( + id: Optional[str] = field( default=None, metadata=dataclass_config(exclude=lambda f: f is None) )🧰 Tools
🪛 GitHub Actions: Check - static
[error] 136-136: mypy: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
[error] 139-139: mypy: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
deepgram/clients/agent/v1/websocket/options.py(7 hunks)
🧰 Additional context used
🪛 GitHub Actions: Check - static
deepgram/clients/agent/v1/websocket/options.py
[error] 136-136: mypy: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
[error] 139-139: mypy: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
🔇 Additional comments (1)
deepgram/clients/agent/v1/websocket/options.py (1)
169-170: 💡 Verification agent🧩 Analysis chain
Consider the appropriate default value for
ThinkProvider.typeBased on past review comments,
think.providerdoesn't have a default value in the API. Consider whether setting the default toNoneis intentional or if it should be omitted.
🌐 Web query:
What is the default provider for Deepgram Agent's think component?💡 Result:
The default provider for Deepgram Agent's think component is Anthropic. By default, Deepgram configures the think provider with the following settings:
- Provider type:
anthropic- Model:
claude-3-haiku-20240307This can be seen in example agent configuration snippets from Deepgram’s official SDK documentation:
agent.configure({ // ... agent: { // ... think: { provider: { type: "anthropic" }, model: "claude-3-haiku-20240307", instructions: "You are a helpful AI assistant. Keep responses brief and friendly.", }, }, });You can override these defaults and configure a different provider, model, or instructions as needed, but when not specified, Deepgram’s Agent will use Anthropic as the think provider[5].
Citations:
- 1: https://developers.deepgram.com/docs/voice-agent-agent-thinking
- 2: https://github.com/deepgram/dg_react_agent
- 3: https://developers.deepgram.com/docs/configure-voice-agent
- 4: https://blog.gogroup.co/building-an-ai-voice-assistant-with-deepgram-voice-agent-api-and-twilio-c8dcdc77dc23
- 5: https://github.com/deepgram/deepgram-js-sdk
- 6: https://deepgram.com/learn/how-do-voice-ai-agents-work
- 7: https://deepgram.com/learn/introducing-ai-voice-agent-api
- 8: https://www.youtube.com/watch?v=F4ITm_aEi6M
- 9: https://developers.deepgram.com/docs/voice-agent-settings-configuration
- 10: https://deepgram.com/learn/aura-text-to-speech-tts-api-voice-ai-agents-launch
Update default values for ThinkProvider to match API defaults
The Deepgram Agent’s think component defaults to Anthropic with
claude-3-haiku-20240307if no provider or model is specified. To surface these defaults in the client, update the dataclass fields accordingly.• deepgram/clients/agent/v1/websocket/options.py
- Replace the
Nonedefaults fortypeandmodelwith the API’s actual defaults.--- a/deepgram/clients/agent/v1/websocket/options.py +++ b/deepgram/clients/agent/v1/websocket/options.py @@ -169,7 +169,7 @@ """Configuration for the AI thinking provider.""" - type: Optional[str] = field(default=None) + type: str = field(default="anthropic") - model: Optional[str] = field(default=None) + model: str = field(default="claude-3-haiku-20240307")Likely an incorrect or invalid review comment.
There was a problem hiding this comment.
Actionable comments posted: 1
🔭 Outside diff range comments (1)
deepgram/clients/agent/v1/websocket/client.py (1)
182-188: 🛠️ Refactor suggestionWiden the
optionstype hint
start()acceptsdictandstrvariants, yet the signature narrows the type toOptional[SettingsOptions]. Static analysis (e.g. mypy) will flag every valid call that passes adict/str.- options: Optional[SettingsOptions] = None, + options: Union[SettingsOptions, Dict[str, Any], str, None] = None,This keeps runtime behaviour unchanged while eliminating false-positive type errors.
♻️ Duplicate comments (5)
deepgram/clients/agent/v1/websocket/options.py (5)
123-127:Function.__getitem__still treats singularendpointas a list
This was flagged in a previous review but remains unresolved.- if "endpoint" in _dict: - _dict["endpoint"] = [ - Endpoint.from_dict(endpoint) for endpoint in _dict["endpoint"] - ] + if "endpoint" in _dict and isinstance(_dict["endpoint"], dict): + _dict["endpoint"] = Endpoint.from_dict(_dict["endpoint"])
150-160:ListenProvider.keytermsdeserialisation is safe, butNoneguard is missingAttempting list-comprehension on
NoneraisesTypeError.- if "keyterms" in _dict: - _dict["keyterms"] = [str(keyterm) for keyterm in _dict["keyterms"]] + if isinstance(_dict.get("keyterms"), list): + _dict["keyterms"] = [str(keyterm) for keyterm in _dict["keyterms"]]
215-223:SpeakProvider.__getitem__– inconsistent handling of scalar fields
voiceandlanguageare optional scalars but are processed as lists, yielding runtimeTypeErrors.- if "voice" in _dict: - _dict["voice"] = [ - CartesiaVoice.from_dict(voice) for voice in _dict["voice"] - ] - if "language" in _dict: - _dict["language"] = [str(language) for language in _dict["language"]] + if "voice" in _dict and isinstance(_dict["voice"], dict): + _dict["voice"] = CartesiaVoice.from_dict(_dict["voice"]) + # 'language' is already a string; no transformation needed.
232-241:Thinknow embedsendpointandprompt– update__getitem__accordingly
provideris singular, yet the method still treats it as a list. Same issue for the newendpoint.- if "provider" in _dict: - _dict["provider"] = [ - ThinkProvider.from_dict(provider) for provider in _dict["provider"] - ] + if "provider" in _dict and isinstance(_dict["provider"], dict): + _dict["provider"] = ThinkProvider.from_dict(_dict["provider"]) ... - if "endpoint" in _dict: - _dict["endpoint"] = [ - Endpoint.from_dict(endpoint) for endpoint in _dict["endpoint"] - ] + if "endpoint" in _dict and isinstance(_dict["endpoint"], dict): + _dict["endpoint"] = Endpoint.from_dict(_dict["endpoint"])
136-141:⚠️ Potential issue
CartesiaVoice.mode/idviolate declared type – fixes pipeline failureBoth fields default to
Nonebut are typedstr, causing the mypy errors reported in CI.- mode: str = field( + mode: Optional[str] = field( ... - id: str = field( + id: Optional[str] = field([mypy now passes]
🧰 Tools
🪛 GitHub Actions: Check - static
[error] 136-136: mypy error: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
[error] 139-139: mypy error: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
🧹 Nitpick comments (2)
deepgram/clients/agent/v1/websocket/client.py (2)
99-104: Update the in-code comment or endpoint path for consistencyThe explanatory comment still points to the legacy path
"agent", while_endpointhas already been updated to"v1/agent/converse". This is misleading for future maintainers.-# needs to be "wss://agent.deepgram.com/agent" +# WebSocket path on `agent.deepgram.com` – keep in sync with server contract.
280-289: Send compact JSON to reduce websocket frame size
str(self._settings)pretty-prints with indentation, increasing payload by ~30-40 %.
Switch toself._settings.to_json()(no indent) for lower latency:- ret_send_cs = self.send(str(self._settings)) + ret_send_cs = self.send(self._settings.to_json())Minor but measurable for high-frequency reconnects.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (14)
deepgram/__init__.py(1 hunks)deepgram/client.py(1 hunks)deepgram/clients/__init__.py(1 hunks)deepgram/clients/agent/__init__.py(1 hunks)deepgram/clients/agent/client.py(2 hunks)deepgram/clients/agent/enums.py(1 hunks)deepgram/clients/agent/v1/__init__.py(1 hunks)deepgram/clients/agent/v1/websocket/__init__.py(1 hunks)deepgram/clients/agent/v1/websocket/async_client.py(7 hunks)deepgram/clients/agent/v1/websocket/client.py(7 hunks)deepgram/clients/agent/v1/websocket/options.py(8 hunks)deepgram/clients/agent/v1/websocket/response.py(2 hunks)examples/agent/async_simple/main.py(2 hunks)examples/agent/simple/main.py(2 hunks)
✅ Files skipped from review due to trivial changes (1)
- deepgram/clients/agent/client.py
🚧 Files skipped from review as they are similar to previous changes (11)
- deepgram/clients/agent/v1/websocket/response.py
- examples/agent/async_simple/main.py
- deepgram/client.py
- deepgram/clients/agent/init.py
- deepgram/clients/agent/v1/websocket/init.py
- deepgram/clients/agent/enums.py
- deepgram/init.py
- deepgram/clients/agent/v1/websocket/async_client.py
- deepgram/clients/init.py
- examples/agent/simple/main.py
- deepgram/clients/agent/v1/init.py
🧰 Additional context used
🧬 Code Graph Analysis (2)
deepgram/clients/agent/v1/websocket/client.py (3)
deepgram/clients/agent/v1/websocket/options.py (2)
SettingsOptions(373-404)check(391-404)deepgram/utils/verboselogs/__init__.py (1)
notice(150-153)deepgram/clients/common/v1/websocket_response.py (1)
ErrorResponse(41-51)
deepgram/clients/agent/v1/websocket/options.py (2)
deepgram/clients/common/v1/shared_response.py (1)
BaseResponse(16-44)deepgram/clients/agent/enums.py (1)
AgentWebSocketEvents(10-36)
🪛 GitHub Actions: Check - static
deepgram/clients/agent/v1/websocket/options.py
[error] 136-136: mypy error: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
[error] 139-139: mypy error: Incompatible types in assignment (expression has type "None", variable has type "str") [assignment]
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
deepgram/clients/agent/v1/websocket/options.py (1)
171-213: SpeakProvider getitem correctly handles the voice field but ignores language conversion.The method handles the conversion of the voice field from a dictionary to a CartesiaVoice object, but there was a past review comment about inconsistent handling of the language field.
Fix inconsistent item handling for
languageinSpeakProvider.__getitem__The
languagefield is defined asOptional[str]but the getter might incorrectly handle it. Ensure language is handled correctly.
🧹 Nitpick comments (3)
deepgram/clients/agent/v1/websocket/options.py (3)
72-91: NewEndpointclass looks well-structured but could use a safety check for empty headers.The implementation of the new Endpoint class looks good with appropriate field types and defaults. The getitem method correctly handles the conversion of headers.
Consider adding a check to prevent iterating over None headers:
def __getitem__(self, key): _dict = self.to_dict() - if "headers" in _dict: + if "headers" in _dict and isinstance(_dict["headers"], list): _dict["headers"] = [ Header.from_dict(headers) for headers in _dict["headers"] ] return _dict[key]
124-136: CartesiaVoice class uses empty strings as defaults for fields typed as str.While using empty strings as defaults helps avoid None values, the API might have specific default values or expect specific formats.
You might consider adding documentation comments explaining the allowed values for each field to help users understand valid values for
modeandid.
139-154: The ListenProvider class has a type inconsistency in the keyterms handling.The type check is correct, but there might be unnecessary string conversion in the list comprehension.
The string conversion in the list comprehension is likely redundant since keyterms is already typed as List[str]:
def __getitem__(self, key): _dict = self.to_dict() if "keyterms" in _dict and isinstance(_dict["keyterms"], list): - _dict["keyterms"] = [str(keyterm) for keyterm in _dict["keyterms"]] + _dict["keyterms"] = _dict["keyterms"] # No conversion needed return _dict[key]
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
deepgram/clients/agent/v1/websocket/options.py(6 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
deepgram/clients/agent/v1/websocket/options.py (2)
deepgram/clients/common/v1/shared_response.py (1)
BaseResponse(16-44)deepgram/clients/agent/enums.py (1)
AgentWebSocketEvents(10-36)
🔇 Additional comments (10)
deepgram/clients/agent/v1/websocket/options.py (10)
109-121: Good implementation for Function.getitem, follows previous code review suggestions.The implementation correctly checks the type of each field before converting it, addressing issues noted in previous code reviews where some fields were incorrectly treated as lists.
217-243: The Think class getitem method correctly handles provider, functions, and endpoint fields.The code now checks if the fields are of the expected type before converting them, addressing issues noted in previous reviews.
247-258: The Listen class getitem method correctly handles the provider field.The implementation includes the type check before conversion, addressing issues noted in previous reviews.
262-277: The Speak class getitem method correctly handles the provider and endpoint fields.The implementation includes type checks before conversion, addressing issues noted in previous reviews.
290-301: Agent class adds a new greeting field and properly handles all nested objects.The getitem method correctly handles the conversion of listen, think, and speak fields with appropriate type checks.
340-343: Audio.getitem now includes type checks for input and output fields.This properly addresses issues noted in previous reviews by checking field types before conversion.
357-372: SettingsOptions (renamed from SettingsConfigurationOptions) looks good but has type handling inconsistency.The class adds a new experimental field and changes the event type, which aligns with the changes in the AgentWebSocketEvents enum.
391-401: UpdatePromptOptions (renamed from UpdateInstructionsOptions) renames 'instructions' to 'prompt'.This name change appears intentional and aligns with the PR's goal of supporting agent v1, though it represents a breaking change for existing code.
414-414: UpdateSpeakOptions now uses the comprehensive Speak class instead of a simple model string.This enhancement provides more flexibility but represents a breaking change that will require updates to existing code.
348-353:✅ Verification successful
Language class (renamed from Context) simplifies to just use a type field.
The class has been significantly simplified, which might be a breaking change for code that was using the previous Context class.
Verify that code depending on the previous Context class has been updated to use the new Language class structure:
🏁 Script executed:
#!/bin/bash # Find code that might be referring to the old Context class echo "Searching for references to Context class..." rg "Context\(" rg "context\:"Length of output: 301
🏁 Script executed:
#!/bin/bash echo "Searching for Context references in deepgram code (excluding SSLContext and options.py)..." rg -n "\bContext\b" deepgram --color never | grep -Ev "SSLContext|options.py"Length of output: 270
No remaining references to the old
Contextclass – renaming toLanguageis safeA search across the
deepgram/directory (excludingSSLContextand the updated file itself) found zero occurrences ofContext. The library’s internal code no longer relies on the old class name, so consumers will be working with the newLanguagedefinition without breaking changes.
| class ThinkProvider(BaseResponse): | ||
| """ | ||
| This class defines the provider for the Think model. | ||
| """ | ||
|
|
||
| type: Optional[str] = field( | ||
| type: Optional[str] = field(default=None) | ||
| model: Optional[str] = field(default=None) | ||
| temperature: Optional[float] = field( | ||
| default=None, metadata=dataclass_config(exclude=lambda f: f is None) | ||
| ) |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
ThinkProvider type and model fields are nullable while previously they had default values.
The change from required to optional fields might be a breaking change if code relies on these fields having non-null values.
Verify that making these fields nullable aligns with the API's behavior and that existing code doesn't assume these fields will always have values:
🏁 Script executed:
#!/bin/bash
# Find code that might be affected by nullable ThinkProvider fields
echo "Searching for code that accesses think provider type or model fields..."
rg -A 2 "think.*provider.*\.(type|model)" Length of output: 1270
🏁 Script executed:
#!/bin/bash
# Display relevant parts of the options.py file including ThinkProvider definition and serialization
file="deepgram/clients/agent/v1/websocket/options.py"
echo "Showing ThinkProvider class and serialization logic:"
sed -n '1,200p' "$file" | sed -n '150,260p'
echo
echo "Searching for any to_dict or serialization methods in this file:"
rg -n "def .*serialize|def .*to_dict|asdict" "$file"Length of output: 1945
🏁 Script executed:
#!/bin/bash
# Locate BaseResponse definition and its to_dict method
echo "Searching for BaseResponse class:"
rg -n "class BaseResponse" -A 50
# Locate to_dict/serialization logic in BaseResponse
echo
echo "Searching for to_dict methods or usage of dataclass_config in serialization:"
rg -n "to_dict" -A 20
# Locate dataclass_config helper
echo
echo "Searching for dataclass_config definition:"
rg -n "def dataclass_config" -A 20Length of output: 108006
Review ThinkProvider defaults and serialization
The ThinkProvider class now defines type and model as Optional[str] = None without any exclude metadata. Unlike temperature, these fields will be serialized as "type": null and "model": null when unset—potentially breaking API calls that expect valid provider identifiers or previous non-null defaults.
Please address this change by either:
- Restoring sensible defaults (e.g.
default="open_ai"anddefault="gpt-4o-mini") to match prior behavior, or - Adding
metadata=dataclass_config(exclude=lambda f: f is None)to bothtypeandmodelso they’re omitted when not explicitly set.
Examples currently override both fields, but the default Think() instance will send null. Confirm against the API spec and update accordingly.
🤖 Prompt for AI Agents (early access)
In deepgram/clients/agent/v1/websocket/options.py around lines 158 to 167, the ThinkProvider class has type and model fields changed to Optional[str] without exclude metadata, causing them to serialize as null when unset. To fix this, either restore their previous default values (e.g., default="open_ai" for type and default="gpt-4o-mini" for model) or add metadata=dataclass_config(exclude=lambda f: f is None) to both fields to omit them from serialization when None. Verify the API spec to choose the correct approach and ensure default Think instances do not send null values for these fields.
Proposed changes
Types of changes
What types of changes does your code introduce to the community Python SDK?
Put an
xin the boxes that applyChecklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.Further comments
Summary by CodeRabbit
.gitignoreto exclude generated audio and chat log files.requestslibrary to example dependencies.