Faster inference

[jiayng01]

Are there ways to improve the inference speed, preferably without retraining the whole model? Is there a reason why DDIM is not used or how should I go about integrating it?

[ymingxie]

Thanks for your interest in our work. We used DDPM following the base work MDM. If you'd like to improve the inference speed, you can reduce n_guide_steps in the classifier guidance.
If you want to integrate DDIM, you can follow the implementation in MDM. OmniControl used the same code base as MDM so it should be easy to bring the DDIM-related code back.

Thanks

[jiayng01]

Thank you for the response. I'd also like to ask how the validity of a spatial control signal $\sigma_{nj}$ is determined; is this the part?

[ymingxie]

Yes, the code shown in this screenshot determined the joints (spatial sparsity) we want to control (non-zero number). The mask_hint determined the temporal sparsity (https://github.com/neu-vi/OmniControl/blob/main/diffusion/gaussian_diffusion.py#L474)