Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any plans to incorporate shallow fusion e.g. with a n-gram language model for decoding? #1232

Open
JakobHavtorn opened this issue Jan 29, 2025 · 0 comments

Comments

@JakobHavtorn
Copy link

One of the better options to adapt whisper to an unknown vocabulary (i.e. specialised terms) is to use shallow fusion to combine an external language model with whisper when decoding.

The external language model can then be trained/tuned using text-only data. In the simplest case, it would be an n-gram language model trained using Whisper's subword units as vocabulary.

However, shallow fusion is not yet available in faster-whisper nor in CTranslate2. Are there any plans to do this, or is this known to not be feasible?

@JakobHavtorn JakobHavtorn changed the title Any plans to incorporate decoding assisted by n-gram language model decoding? Any plans to incorporate shallow fusion e.g. with a n-gram language model for decoding? Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant