Issues with xtts_v2 batched inference

[Rakshith12-pixel]

I am working with xtts_v2

I have implemented batched inference on xtts_v2. However, if each text(sentence) in the batch is of different lenght, then I face the following issues:

1) when we look at XTTS_MODEL.gpt.generate(cond_latents=batch_gpt_cond_latent, text_inputs=batch_text_inputs), which are the audio codes, there are some tokens that gets repeated (though they are not gpt.stop_audio_token). This problem occurs whenever there's an unequal length-batch.

2) Sometimes the audio length is way too long, in a way that is not proportionate to the sentence length.

3) As a result of these things, Gibberish sound is being generated in the output audio, only when unequal length-examples in the batch is provided. Otherwise, the model performs well.

Even the xtts_demo.py performs single example feed-forward even if we give longer sentence after splitting it.

I'd be very helful if anyone could explain why these issues happen and help me implement a proper batched inference for xtts_v2 model

[2Bye]

@Rakshith12-pixel Hello! Could you write me? I think i can help you.

My email: [email protected]

[sushant-samespace]

@2Bye @Rakshith12-pixel did you guys find anything on this ? Thanks !