Reproducing the quantitative results on RealEstate10K

[XiangZ-0]

Hi,

Thank you for releasing the amazing ViewCrafter project! I hope to use ViewCrafter as a baseline method in my own project due to its great performance. However, when testing ViewCrafter on the RealEstate10K dataset, I found that the viewpoint of output frames will have a slight mismatch with the viewpoint of the input point cloud renders, which degrades the computed metrics. I also tested that on the boy example and found similar results as shown below (I used the 25 frames 320x512 model). Could you help me with this? Or could you tell me how to correctly do the evaluation on RealEstate10K? Thanks a lot!

[Drexubery]

Hi, thanks for your interest in our work!

We do found that there may be cases where the generated views and the input point cloud renders are not 100% aligned, since the model needs to learn to correct the inaccurate geometry and cannot fully trust the input. We recommend using the 576x1024, 25 frames model for better visual results, although it may not fully resolve such slight misalignment.

For the evaluation on RealEstate10K, you can refer to the answer here #29 (comment) to generate paired point cloud render and GT video for evaluation.

[XiangZ-0]

Thank you for the reply! Yes, I understand that sometimes the point cloud renders contain inaccurate geometry. For the evaluation, the GT sequences are also generated by the point clouds estimated by DUSt3R instead of the original frames in RealEstate10K. Is my understanding correct?

[LiHaodong0217]

nope,gt is ground true from dataset file