This (work in progress) project is a pipeline to reconstruct a 3D model of a face starting from RGBD scans.
However, additional steps required to actually use this model (e.g., retopology) are not part of this project. The goal of this project is producing a rough model based on acquired data to use instead of a 3D sculpture.
This project is built on top of Open3D and is basically an application of their pipeline.
The first step is to capture the input frames.
The realsense-read-bag.py script provides a way to extract the needed data
from a bag file created with the Intel® RealSense™ Viewer or SDK.
The essential files are aligned RGB and depth frames, and a camera.json with
intrinsic parameters.
In addition to that, the script creates an overlapped view of the frame for
manual selection of the frames.
From my tests, it is better to use just a reduced number of cherry-picked good frames, rather than trying to automatically deal with all the frames.
The align program will load all the frames in memory (and also on the GPU), so
it isn't suited for running with all the frames of a scan.
After choosing a few frames, you should align them roughly. This replaces the global alignment step of Open3D's pipeline.
The advantage of doing it manually is that you'll be able to choose the final placement already.
When you have an initial alignment, you can refine it with ICP.
You can select the reference point cloud and the one to align with the checkboxes and then click on the align button.
Only one-to-one alignments are supported currently.
Finally, the point clouds are merged with Truncated Signed Distance Function.
You can choose the frames to merge with the checkboxes, and then click on the merge button.
You can then export both the merged point cloud and the triangle mesh generated from it in any format supported by Open3D.
This project is built upon Open3D.
Building it isn't trivial, so you'll have to follow their instructions, or download their prebuilt binaries, or install it from a package manager.
We use some of Open3D's dependencies directly:
It should be possible to configure CMake to look for them from Open3D, but I haven't tried it, yet (my Open3D build is linked to the libraies provided by my system, and in general I've built the project only on Debian testing).
In addition to that, the align program depends on the following libraries,
that are either included in the tree, or as git submodules:
If needed, they will be built as static libraries.
While this project’s code is dedicated to the public domain (you can refer to the Zero-Clause BSD license, if needed), the depdencies have their own licensing terms.