Multi-view version checkpoint/code

[wowo0709]

Thank you for your great work!

I noticed that the paper discusses a multi-view version of Pixal3D, but the current repository seems to provide only the single-image inference code and checkpoint. Do you have any plans to release the checkpoint and inference code for the multi-view TRELLIS.2-based model?

Looking forward to your response :)

[LDYang694]

Thanks for your interest!

The currently released version is based on TRELLIS.2, mainly because this version achieves better overall geometry and texture quality. The released checkpoint has not been specifically fine-tuned for multi-view inputs.

That said, from an implementation perspective, extending Pixal3D to the multi-view setting is relatively straightforward and does not require substantial changes to the existing code. Specifically, features from multiple input views can be back-projected into a unified 3D space, and the resulting feature volumes can then be aggregated. This setting requires the input views to have known and reasonably accurate camera parameters.

The currently released checkpoint also has a certain degree of generalization ability, so it can serve as a basis for multi-view extension. We will also consider further supporting the multi-view version in the future.

[samuele-bortolato]

Even adding multiview using only global features and cross attention (and using the back-projection for only one main image) could give the model more information and improve the reconstruction without requiring accurate camera poses and parameters

[gogogogzb]

非常感谢您的工作，我这边想问一下您这里是否能提供一下处理多图输入的模型权重以及推理的代码，如果没有基于Trellis2的模型，是否有论文中描述的模型呢？ 或者能提供一份基于Trellis2的处理多图输入的推理代码吗？ 非常感谢