SmartBoxes for Interactive Urban Reconstruction

ACM Transactions on Graphics (Proceedings SIGGRAPH 2010)

Liangliang Nan¹     Andrei Sharf¹     Hao Zhang²   Daniel Cohen-Or³    Baoquan Chen¹  

¹Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China
²School of Computing Science, Simon Fraser University, Canada
³School of Computer Science, Tel Aviv University, Israel

Figure 1: A large-scale urban scene is reconstructed in details from a noisy and sparse LiDAR scan using SmartBoxes interactively. Close-up views show the reconstructed details of 3D architectural structures.

Abstract

We introduce an interactive tool which enables a user to quickly assemble an architectural model directly over a 3D point cloud acquired from large-scale scanning of an urban scene. The user loosely defines and manipulates simple building blocks, which we call SmartBoxes, over the point samples. These boxes quickly snap to their proper locations to conform to common architectural structures. The key idea is that the building blocks are smart in the sense that their locations and sizes are automatically adjusted on-the-fly to fit well to the point data, while at the same time respecting contextual relations with nearby similar blocks. SmartBoxes are assembled through a discrete optimization to balance between two snapping forces defined respectively by a data-fitting term and a contextual term, which together assist the user in reconstructing the architectural model from a sparse and noisy point cloud. We show that a combination of the user’s interactive guidance and high-level knowledge about the semantics of the underlying model, together with the snapping forces, allows the reconstruction of structures which are partially or even completely missing from the input.

Results

Figure 2: From a typically poor scan, we reconstruct simple SmartBoxes to form window railings and balconies (left) through snapping and grouping. Next, one column (middle) is reconstructed through a drag-and-drop of the grouped compound balcony and window. Reconstruction of the whole facade (right) can then be achieved in a matter of seconds by dragging the whole column, grouped into a compound SmartBox, and snapping to sparse regions. Note the irregularities, e.g., in the scales of the columns and their spacing.

Figure 3: Effects of the data-fitting and contextual forces on reconstruction, where a compound balcony-window SmartBox (shown in the red square) is dragged to snap to a noisy and sparse point cloud (top row). Second row: data-fitting force only. Third row: contextual force only. Bottom row: combining two forces so that inconsistencies in the data are disambiguated by the contextual force.

Figure 4: Grouping several boxes (left) triggers an automatic repair of a few forms of inconsistencies, e.g., intersections, resulting in a more regularized and well-aligned compound box. (right).

Figure 5: Reconstruction of a building containing cylinders, along with other complex architectural structures.

Figure 6: Additional reconstruction results using SmartBoxes. From left to right: real photograph, LiDAR scan, 3D reconstruction, and its textured version for a visual comparison with the photograph.

Acknowledgements

We thank the anonymous reviewers for their valuable suggestions. This work was supported in part by National Natural Science Foundation of China (60902104), National High-tech R&D Program of China (2009AA01Z302), CAS Visiting Professorship for Senior International Scientists, CAS Fellowship for Young International Scientists, Shenzhen Science and Technology Foundation (GJ200807210013A), and the Natural Sciences and Engineering Research Council of Canada (No. 611370).

BibTex