VITS how to add new languages

[saibharani]

I have a VITS model trained in English, French and Hindi languages and I want to add extra languages. can anyone give instruction about how to add languages?

[erogol]

1. Run python TTS/tts/utils/text/phonemizers/__init__.py to see what languages are supported by the current set of phonemizer.
   You will see something like

{'lb': 'gruut', 'ru-ru': 'gruut', 'en-gb': 'espeak', 'sw': 'espeak', 'de-de': 'gruut', 'ar': 'espeak', 'sv-se': 'gruut', 'fa': 'espeak', 'nl': 'espeak', 'fr-fr': 'espeak', 'cs-cz': 'gruut', 'en-us': 'espeak', 'it-it': 'gruut', 'zh-cn': 'zh_cn_phonemizer', 'es-es': 'gruut', 'pt': 'espeak', 'af': 'espeak', 'am': 'espeak', 'an': 'espeak', 'as': 'espeak', 'az': 'espeak', 'ba': 'espeak', 'bg': 'espeak', 'bn': 'espeak', 'bpy': 'espeak', 'bs': 'espeak', 'ca': 'espeak', 'cmn': 'espeak', 'cs': 'espeak', 'cy': 'espeak', 'da': 'espeak', 'de': 'espeak', 'el': 'espeak', 'en-029': 'espeak', 'en-gb-scotland': 'espeak', 'en-gb-x-gbclan': 'espeak', 'en-gb-x-gbcwmd': 'espeak', 'en-gb-x-rp': 'espeak', 'eo': 'espeak', 'es': 'espeak', 'es-419': 'espeak', 'et': 'espeak', 'eu': 'espeak', 'fa-latn': 'espeak', 'fi': 'espeak', 'fr-be': 'espeak', 'fr-ch': 'espeak', 'ga': 'espeak', 'gd': 'espeak', 'gn': 'espeak', 'grc': 'espeak', 'gu': 'espeak', 'hak': 'espeak', 'hi': 'espeak', 'hr': 'espeak', 'ht': 'espeak', 'hu': 'espeak', 'hy': 'espeak', 'hyw': 'espeak', 'ia': 'espeak', 'id': 'espeak', 'is': 'espeak', 'it': 'espeak', 'ja': 'espeak', 'jbo': 'espeak', 'ka': 'espeak', 'kk': 'espeak', 'kl': 'espeak', 'kn': 'espeak', 'ko': 'espeak', 'kok': 'espeak', 'ku': 'espeak', 'ky': 'espeak', 'la': 'espeak', 'lfn': 'espeak', 'lt': 'espeak', 'lv': 'espeak', 'mi': 'espeak', 'mk': 'espeak', 'ml': 'espeak', 'mr': 'espeak', 'ms': 'espeak', 'mt': 'espeak', 'my': 'espeak', 'nb': 'espeak', 'nci': 'espeak', 'ne': 'espeak', 'om': 'espeak', 'or': 'espeak', 'pa': 'espeak', 'pap': 'espeak', 'pl': 'espeak', 'pt-br': 'espeak', 'py': 'espeak', 'quc': 'espeak', 'ro': 'espeak', 'ru': 'espeak', 'ru-lv': 'espeak', 'sd': 'espeak', 'shn': 'espeak', 'si': 'espeak', 'sk': 'espeak', 'sl': 'espeak', 'sq': 'espeak', 'sr': 'espeak', 'sv': 'espeak', 'ta': 'espeak', 'te': 'espeak', 'tn': 'espeak', 'tr': 'espeak', 'tt': 'espeak', 'ur': 'espeak', 'uz': 'espeak', 'vi': 'espeak', 'vi-vn-x-central': 'espeak', 'vi-vn-x-south': 'espeak', 'yue': 'espeak', 'en': 'espeak', 'ja-jp': 'ja_jp_phonemizer'}

2. If you target language is not in the list, you need to write your own phonemizer under TS/tts/utils/text/phonemizers. If it is the case let me know. Or you can simply train with the graphemes.

3. Assuming you have your language above, set these in the model config accordingly (I assume english)

use_phonemes=True,
phonemizer="espeak",
phoneme_language="en",

4. If you use graphemes,

use_phonemes=False,

5. If you have a different alphabet. Set the CharactersConfig

char_config = CharactersConfig(...)
config = ModelConfig(...
   characters=char_config
   ...
)

Briefly this is it. If you're stuck somewhere check the source code, example recipes. If all fails, post here.

[francois-vz]

Thanks, great answer! :)

[Divesh-M]

HI @erogol , am working on this VITS TTS model, i have to train telugu language which is not present in the phonemizer list, can you please guide here. How to create our own phonemizer and train the model for specific language.

Thanks in advance.

[Rakshitha-Ummadisetti]

Hi @Divesh-M . I've recently trained VITS with telugu dataset.
There are a couple of changes you need to make

1. In symbols.py include telugu symbols all matras and alphabet
2. Espeak-ng supports telugu. Make necessary changes to run the preprocess.py
3. If you are using single GPU please make changes in train.py to make it work for telugu dataset

If you have any particular issues do email [email protected]

[Divesh-M]

@saibharani did you try to train VITS model with hindi language, if yes can you please instruct how to train it.

Thanks in advance

[bharani-y]

I used the original VITS implementation here as it supports espeak better and the number of modifications required is less, change the dataloader to fit your data structure and use espeak-ng to convert hindi text to phoneme, you can select between LJspeech (Single speaker) config or VCTK (Multi speaker) config based on your dataset

[Divesh-M]

I addded hindi phonemizer in code, but when i trying to train the model. it throws error "[!] train_step() retuned None outputs. Skipping training step."

Data Loader Initialization config :

DataLoader initialization
| > Tokenizer:
| > add_blank: False
| > use_eos_bos: False
| > use_phonemes: True
| > phonemizer:
| > phoneme language: hi
| > phoneme backend: hi_phonemizer
| > Number of instances : 531
| > Preprocessing samples
| > Max text length: 288
| > Min text length: 19
| > Avg text length: 118.19585687382298
|
| > Max audio length: 514113.0
| > Min audio length: 47799.0
| > Avg audio length: 223526.23540489643
| > Num. instances discarded samples: 0
| > Batch group size: 0.

[AssassinsLament]

Hi,

Can someone point me out to a written guide on how to train a different/new language that is not in the list?

Things about this language that might help, this language is a tonal language with classifiers, class nouns, adverbs, reciprocal verbs, phrasal verbs and others. Besides, this language's words consists of monosyllable, disyllable and polysyllable. Basically, words can be written similarly with the exception of the tone marker (at the end of the word) that defines how it is said and what it's meaning is. Is there a similar language that I can use as an example?

Thank you in advanced.