Inference with COLMAP processed results

[bell-one]

First of all, Thanks for share your code with great paper,

I think this initial planar mapping will be very useful with another multi view image process,

and I'd like to use your initial planar information with neural radiance fields,
so I tried some edits for COLMAP processed results, with iphone demo reference.

But I cannot get expected results at all,
I tried with NeRF/llff/room dataset which has sufficient planar surfaces,

like this, but as results, I got like follows,

I think there should be problem with pre-processing of colmap data
and these errors propagated to coordinate systems, scaling, intrinsic params or projection matrix.

Can you help me to apply colmap dataset to your project?

[ymingxie]

Thank you for your interest in our work! I will take a look soon ...

[ymingxie]

Hey, I checked the llff data and found that there seems to be a large domain gap between ScanNet (the dataset we trained the model) and llff. The differences mainly lie in the fact that the geometry in llff dataset is far away from the camera, while in ScanNet, the camera usually scans the scene, so the surface won't be that far. We predefined a local volume in PlanarRecon for each fragment, so the surface (about 4 meters far away from the camera) is excluded.

It's not easy to directly adopt the PlanarRecon with ScanNet pretrained model on NeRF dataset. It would be better to finetune the model on such a dataset but it seems that gt plane on NeRF dataset is hard to obtain... I don't have a good idea for now :(

[kwea123]

@ymingxie thank you for the insight, can you still provide the script to process colmap data anyway? so that we can try on other types of data, not only llff.

Thank you

[bell-one]

@ymingxie
Thanks for the review, I'm sure there must be a gap in the data domain. I'll do some fine-tuning as you suggested, or I'll make another attempt to apply the callmap processing result.
@kwea123
I made some initial code edit for use colmap data in here,
If you are interested, I think it would be nice to make it together.