Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update turn detector model; more robust to missing terminal punctuation #1565

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jeradf
Copy link
Contributor

@jeradf jeradf commented Feb 27, 2025

we've seen that streaming STT sometimes omits terminal punctuation from true turn endings, which almost always results in a false negative from the turn detector model.

i've retrained the model to be more robust to these STT mistakes. eval metrics below

                        tpr    tnr
v1.2.0 with puncs     0.988  0.904
v1.2.0 without puncs  0.218  0.899

                        tpr    tnr
v1.2.1 with puncs     0.988  0.875
v1.2.1 without puncs  0.780  0.907

Copy link

changeset-bot bot commented Feb 27, 2025

🦋 Changeset detected

Latest commit: 7812d2c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
livekit-plugins-turn-detector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant