You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One of the better options to adapt whisper to an unknown vocabulary (i.e. specialised terms) is to use shallow fusion to combine an external language model with whisper when decoding.
The external language model can then be trained/tuned using text-only data. In the simplest case, it would be an n-gram language model trained using Whisper's subword units as vocabulary.
However, shallow fusion is not yet available in faster-whisper nor in CTranslate2. Are there any plans to do this, or is this known to not be feasible?
The text was updated successfully, but these errors were encountered:
JakobHavtorn
changed the title
Any plans to incorporate decoding assisted by n-gram language model decoding?
Any plans to incorporate shallow fusion e.g. with a n-gram language model for decoding?
Jan 29, 2025
One of the better options to adapt whisper to an unknown vocabulary (i.e. specialised terms) is to use shallow fusion to combine an external language model with whisper when decoding.
The external language model can then be trained/tuned using text-only data. In the simplest case, it would be an n-gram language model trained using Whisper's subword units as vocabulary.
However, shallow fusion is not yet available in
faster-whisper
nor in CTranslate2. Are there any plans to do this, or is this known to not be feasible?The text was updated successfully, but these errors were encountered: