ONNX Inference Issue

[ashissamal]

Thank you for the excellent work and sharing this implementation.

I tried to convert to ONNX and did the inference . However I have below issue/challenges. Appreciate any valuable suggestions .

ONNX input dimension remains fixed, As a result we need to pad additional Ids to the phoneme array. In the existing code, it replicates the phoneme till the ONNX input size length. This in turn creates repeated audios of the same content. Is there any specific Id, I can pad to avoid unwanted audio at the end. OR Is there a way to pass dynamic length phoneme array to ONNX model . Please clarify if I'm missing anything here and how to avoid this.

[roatienza]

The duration prediction was left out in the demo version. It has been restored in the pytorch2.0 version.

Sadly, there is an upsample operator that is not supported w/ dynamic axis. The workaround is to use a fixed size via --onnx-insize then trim the wav prediction using the predicted duration.

[ashissamal]

Thank you for the suggestions. I could create a static dimension input and convert to ONNX format. However, the inference time on the ONNX created server (x86_64, 16 core, 2.40GHz, Linux) is <60ms and the target inference platform(Android + ARM 64 v8, 4 core, 1.6GHz) is 10sec, hugely different. Is there anything to be done differently to improve the latency in the target platform.

P.S. Both servers are running the inference in CPU.