Maybe a better way forward than fixing whisper (VibeVoice-ASR)

[User-3090]

https://huggingface.co/microsoft/VibeVoice-ASR

https://www.youtube.com/watch?v=BYPlfLQm0CQ&t=1441s

[cjpais]

This is a pretty big model and not really suited to most people without GPU's or lower end GPU's

[User-3090]

I guess the reality is that most people with low end hardware use SuperWhisper with their online models anyways. Parakeet is nice for what it is, but it's just not good enough for day to day usage. For casual conversations it might be okay, but it falls apart with technical terms or medical terms or languages other than English.

Nvidia also has these Canary models. Did you ever look into those? I think those can be run on a 12GB GPU at least.

I think Whisper Large V3 Turbo with Vulkan backend is still very close and runs well on my AMD Ryzen 7 7840HS mini PC. It just barely works in handy. The advantage of the Whisper family is that there are many fine-tuned model variants for specific languages or particular industries.

I don't know what way forward. Parakeet isn't quite ready for the real world. Whisper is broken. VibeVoice probably overkill. I think fixing Whisper and looking into the the Canary models might be the most promising, what do you think?

[cjpais]

Supporting large models at the moment is not a priority. There are relatively few complaints that I have heard on the whole relative to the number of people using the app

There are plenty of people who want free alternatives.

[intech]

@User-3090 @cjpais I encountered the same issue and implemented a canary architecture along with model support #1022