CurveFusion: Reconstructing Thin Structures from RGBD Sequences

Lingjie Liu1,2*   Nenglun Chen1*   Duygu Ceylan3   Christian Theobalt4   Wenping Wang1   Niloy J. Mitra2

1University of Hong Kong  2University College London  3Adobe Research  4Max Planck Institute for Informatics

*joint first authors

Siggraph Asia 2018 (Conditionally accepted)




Abstract

We introduce CurveFusion, the first approach for high quality scanning of thin structures at interactive rates using a handheld RGBD camera. Thin filament-like structures are mathematically just 1D curves embedded in R3, and integration-based reconstruction works best when depth sequences (from the thin structure parts) are fused using the object's (unknown) curve skeleton. Thus, using the complementary but noisy color and depth channels, CurveFusion first automatically identifies point samples on potential thin structures and groups them into bundles, each being a group of a fixed number of aligned consecutive frames. Then, the algorithm extracts per-bundle skeleton curves using L1 axes, and aligns and iteratively merges the L1segments from all the bundles to form the final complete curve skeleton. Thus, unlike previous methods, reconstruction happens via integration along a data-dependent fusion primitive, i.e., the extracted curve skeleton. We extensively evaluate CurveFusion on a range of challenging examples, different scanner and calibration settings, and present high fidelity thin structure reconstructions previously just not possible from raw RGBD sequences.

paper thumbnail

Paper

CurveFusion  2018











Video

 


Acknowledgement

We thank our reviewers for their invaluable comments. We thank Hui Huang, Amy Tabb and Zheng Wang for their great help with the testing and validation of our work. We also thank Jiatao Gu, Cheng Lin, Zhiming Cui, Runnan Chen, Maria Lam, Paul Guerrero, Elizabeth Schildge for their help. This work was partially funded by the ERC Starting Grant SmartGeometry (StG-2013-335373), the Research Grant Council of Hong Kong (GRF 17210718), a Google Faculty award, a UCL visiting student program, and gifts from Adobe.