Skip to content

feat(speaker-diarization): add streaming Sortformer diarization - #31

Open
sumimakito wants to merge 8 commits into
mainfrom
feat/sd-sortformer
Open

sumimakito wants to merge 8 commits into
mainfrom
feat/sd-sortformer

Conversation

@sumimakito

Copy link
Copy Markdown
Member

This pull request adds streaming Sortformer diarization to Sherpaw:

  • Label up to eight speakers every 10 ms with NVIDIA Nemotron 3 Diarization on WebGPU, from @sherpaw/speaker-diarization/sortformer. It also marks when two people talk at once.
  • Choose 1.04 s or 0.64 s latency. A benchmark on the device recommends the one that keeps up with live audio: on an Apple M4, the model runs about 2.7 times and 1.9 times faster than real time.
  • Turn the probabilities into speaker turns. A turn starts where the new speaker becomes more likely than the previous one, so the first word of a quick reply no longer joins the previous turn.
  • Download the model with a script that stores its weights as int8: 104 MB instead of about 400 MB, with nearly the same output.
  • In the demo, a Nemotron page shows the turns live and how far each step is behind real time.

Next: publishing the int8 model on Hugging Face, so that the deployed demo can load it.

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

✅ Deploy to Cloudflare Workers (Preview) for sandbox ready!

Name Link
🔍 Latest deploy log https://github.com/moeru-ai/sherpaw/actions/runs/38079427263
😎 Deploy Preview https://02b04a4f-moeru-ai-sherpaw.moeru-ai.workers.dev
🚀 Pull Request Preview https://pr-31-moeru-ai-sherpaw.moeru-ai.workers.dev

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant