Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AssemblyAI Plugin #687

Merged
merged 45 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
ef6a5b5
Add AssemblyAI STT plugin
cch41 Jul 5, 2024
dad69a7
Merge changes from PR 419 into oconnoob/add-assemblyai-plugin
ryan-assemblyai Aug 30, 2024
afb7cf1
add error resolution details to ValueError in STT initialization
ryan-assemblyai Aug 30, 2024
147bebc
use session property rather than function in STT
ryan-assemblyai Aug 30, 2024
aa2cf67
fix type hint for buffer size in STT
ryan-assemblyai Aug 30, 2024
ddd8106
replace type hint union with `Optional`
ryan-assemblyai Aug 30, 2024
8b478a6
make encoding type hint specify possible values
ryan-assemblyai Aug 30, 2024
2e8cde5
word boost type hint: `str` -> `List[str]`
ryan-assemblyai Aug 30, 2024
c53f254
`"Assembly AI"` -> `"AssemblyAI"`
ryan-assemblyai Aug 30, 2024
ab67cc4
fix buffer size type hint
ryan-assemblyai Aug 30, 2024
5bdaa64
add check for mono audio
ryan-assemblyai Aug 30, 2024
194dda9
moved buffer variables from instance to `_run_ws` method
ryan-assemblyai Aug 30, 2024
72adccd
corrected logic error and improved readability
ryan-assemblyai Aug 30, 2024
6958e9c
update `assemblyai.stt.STT` to use new `STTCapabilities`
ryan-assemblyai Sep 5, 2024
4fc4e3c
remove token in favor of api key
ryan-assemblyai Sep 5, 2024
fcd08ca
add catch-all for unexpected data types in `SpeechStream`
ryan-assemblyai Sep 5, 2024
93d130b
update AAI plugin README
ryan-assemblyai Sep 5, 2024
158b431
add assemblyai to agents README
ryan-assemblyai Sep 5, 2024
669e9a8
remove mentions of `token`
ryan-assemblyai Sep 5, 2024
7e861b9
`Assembly AI` -> `AssemblyAI`
ryan-assemblyai Sep 5, 2024
55f4bb4
remove arguments/attributes redundant with `STTOptions`
ryan-assemblyai Sep 5, 2024
5ba8be0
remove unused function
ryan-assemblyai Sep 5, 2024
65ad305
fix definition of `STT.session`
ryan-assemblyai Sep 5, 2024
dd8ec5c
add
ryan-assemblyai Sep 18, 2024
7253fd6
Merge branch 'main' into oconnoob/add-assemblyai-plugin
ryan-assemblyai Nov 4, 2024
1180bfb
address PR comments
ryan-assemblyai Nov 4, 2024
60776cc
satisfy interface changes to base class
ryan-assemblyai Nov 4, 2024
9d2b4b6
ruff formatting
ryan-assemblyai Nov 5, 2024
a48a464
add plugin to README
ryan-assemblyai Nov 5, 2024
34ec612
change parameter name to avoid naming conflict
ryan-assemblyai Nov 5, 2024
22ea13f
add `package.json`
ryan-assemblyai Nov 6, 2024
4042aaf
import logger and remove extraneous method from `__init__.py`
ryan-assemblyai Nov 6, 2024
bf1c787
use `push_frame` where possible (DRY)
ryan-assemblyai Nov 6, 2024
37d2ea7
add end_of_input and override `end_input`
ryan-assemblyai Nov 8, 2024
295dc90
comment out
ryan-assemblyai Nov 8, 2024
f4a87d9
implement feedback
ryan-assemblyai Nov 8, 2024
7d0987b
`_main_task` -> `_task` and optimize loop
ryan-assemblyai Nov 8, 2024
32f52d7
update install_plugins_editable.sh to include aai
ryan-assemblyai Nov 8, 2024
d6fbb5b
remove extraneous code
ryan-assemblyai Nov 8, 2024
2c9c6dc
add post init to type check encoding
ryan-assemblyai Nov 8, 2024
330c3f5
change exception raise to log
ryan-assemblyai Nov 8, 2024
df8d664
add handling for all AAI message types
ryan-assemblyai Nov 8, 2024
98d62ed
Revert "remove extraneous code"
ryan-assemblyai Nov 8, 2024
5be5dbe
fix formatting
ryan-assemblyai Nov 12, 2024
1572f64
ruff format
ryan-assemblyai Nov 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ The following plugins are available today:
| Plugin | Features |
| ---------------------------------------------------------------------------------- | ------------------------------------------- |
| [livekit-plugins-anthropic](https://pypi.org/project/livekit-plugins-anthropic/) | LLM |
| [livekit-plugins-assemblyai](https://pypi.org/project/livekit-plugins-assemblyai/) | STT |
| [livekit-plugins-azure](https://pypi.org/project/livekit-plugins-azure/) | STT, TTS |
| [livekit-plugins-deepgram](https://pypi.org/project/livekit-plugins-deepgram/) | STT |
| [livekit-plugins-cartesia](https://pypi.org/project/livekit-plugins-cartesia/) | TTS |
Expand All @@ -65,7 +66,6 @@ The following plugins are available today:
| [livekit-plugins-openai](https://pypi.org/project/livekit-plugins-openai/) | LLM, STT, TTS, Assistants API, Realtime API |
| [livekit-plugins-silero](https://pypi.org/project/livekit-plugins-silero/) | VAD |


## Documentation and guides

Documentation on the framework and how to use it can be found [here](https://docs.livekit.io/agents)
Expand Down
1 change: 1 addition & 0 deletions livekit-plugins/install_plugins_editable.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ if [[ -z "$VIRTUAL_ENV" ]]; then
exit 1
fi

pip install -e ./livekit-plugins-assemblyai --config-settings editable_mode=strict
pip install -e ./livekit-plugins-azure --config-settings editable_mode=strict
pip install -e ./livekit-plugins-cartesia --config-settings editable_mode=strict
pip install -e ./livekit-plugins-deepgram --config-settings editable_mode=strict
Expand Down
13 changes: 13 additions & 0 deletions livekit-plugins/livekit-plugins-assemblyai/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# LiveKit Plugins AssemblyAI

Agent Framework plugin for AssemblyAI. Currently supports Streaming Speech-to-Text.

## Installation

```bash
pip install livekit-plugins-assemblyai
```

## Pre-requisites

You'll need to specify an AssemblyAI API Key. It can be set as environment variable: `ASSEMBLYAI_API_KEY`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from .log import logger
from .stt import STT, SpeechStream
from .version import __version__

__all__ = [
"STT",
"SpeechStream",
"logger",
"__version__",
]

from livekit.agents import Plugin


class AssemblyAIPlugin(Plugin):
def __init__(self):
super().__init__(__name__, __version__, __package__)


Plugin.register_plugin(AssemblyAIPlugin())
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import logging

logger = logging.getLogger("livekit.plugins.assemblyai")
Loading
Loading