Adding multilingual support via fine-tuning code

[loretoparisi]

While both tiny and base model achieve impressive performances 💯 in terms of WER on academics datasets when compared to the "standard-de-facto" (e.g. Whisper), in a real world scenery a multi-lingual model would address more use cases.
Fine-tune train code could eventually enable support by the community.

[evmaki]

Yes, I think releasing some fine-tuning code would be beneficial. I can add it to our TODO list.

[bil-ash]

@evmaki Please provide the code for finetuning, would be beneficial for extending to other languages.

[zalastone]

Korean and Mandarin will boost their popularity in ASR use cases . it's great work guys

[pprobst]

+1

[wredan]

Results seem promising. Hope to have the multilingual support or some fine-tune code soon, thank you for your work :)

[bil-ash]

Any updates on this?

[loretoparisi]

The reason why this (multilingual support) is extremely important can been see from this simple chart related to whisper fine-tune for downstream tasks of transcription (WER) and translation (BLEU) and the correlation to the amount (in hours) of audio transcribed or translated respectively. By example while Spanish (ES) exhibits the Best WER (2.5) for speech recognition it has the Lowest BLEU score (24) for translation, while German (DE) has balanced performance (WER: 4, BLEU: 35), or - viceversa Portuguese (PT) shows the High BLEU score (39) for translation but a relatively low WER (4) for speech recognition, etc.

Approximative data derived from Robust Speech Recognition via Large-Scale Weak Supervision