SmartBoxes for Interactive Urban Reconstruction

ACM Transactions on Graphics (Proceedings SIGGRAPH 2010)

Liangliang Nan1     Andrei Sharf1     Hao Zhang2   Daniel Cohen-Or3    Baoquan Chen1  

1Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China
2School of Computing Science, Simon Fraser University, Canada
3School of Computer Science, Tel Aviv University, Israel


Figure 1: A large-scale urban scene is reconstructed in details from a noisy and sparse LiDAR scan using SmartBoxes interactively. Close-up views show the reconstructed details of 3D architectural structures.



We introduce an interactive tool which enables a user to quickly assemble an architectural model directly over a 3D point cloud acquired from large-scale scanning of an urban scene. The user loosely defines and manipulates simple building blocks, which we call SmartBoxes, over the point samples. These boxes quickly snap to their proper locations to conform to common architectural structures. The key idea is that the building blocks are smart in the sense that their locations and sizes are automatically adjusted on-the-fly to fit well to the point data, while at the same time respecting contextual relations with nearby similar blocks. SmartBoxes are assembled through a discrete optimization to balance between two snapping forces defined respectively by a data-fitting term and a contextual term, which together assist the user in reconstructing the architectural model from a sparse and noisy point cloud. We show that a combination of the user’s interactive guidance and high-level knowledge about the semantics of the underlying model, together with the snapping forces, allows the reconstruction of structures which are partially or even completely missing from the input.



Figure 2: From a typically poor scan, we reconstruct simple SmartBoxes to form window railings and balconies (left) through snapping and grouping. Next, one column (middle) is reconstructed through a drag-and-drop of the grouped compound balcony and window. Reconstruction of the whole facade (right) can then be achieved in a matter of seconds by dragging the whole column, grouped into a compound SmartBox, and snapping to sparse regions. Note the irregularities, e.g., in the scales of the columns and their spacing.


Figure 3: Effects of the data-fitting and contextual forces on reconstruction, where a compound balcony-window SmartBox (shown in the red square) is dragged to snap to a noisy and sparse point cloud (top row). Second row: data-fitting force only. Third row: contextual force only. Bottom row: combining two forces so that inconsistencies in the data are disambiguated by the contextual force.


Figure 4: Grouping several boxes (left) triggers an automatic repair of a few forms of inconsistencies, e.g., intersections, resulting in a more regularized and well-aligned compound box. (right).


Figure 5: Reconstruction of a building containing cylinders, along with other complex architectural structures.

(a) Handling of a nearly symmetry facades with some irregularity.
(b) Handling of a more complex building structure containing more irregularities.
(c) Grouping and emphasis on contextual force during drop-and-drop allow the reconstruction to deal with large-scale missing data.

Figure 6: Additional reconstruction results using SmartBoxes. From left to right: real photograph, LiDAR scan, 3D reconstruction, and its textured version for a visual comparison with the photograph.

Paper [5M PDF]
Video [120M mov]
Live demo [52M wmv]
Slides [13M pptx]
Data [95M zip]


We thank the anonymous reviewers for their valuable suggestions. This work was supported in part by National Natural Science Foundation of China (60902104), National High-tech R&D Program of China (2009AA01Z302), CAS Visiting Professorship for Senior International Scientists, CAS Fellowship for Young International Scientists, Shenzhen Science and Technology Foundation (GJ200807210013A), and the Natural Sciences and Engineering Research Council of Canada (No. 611370).



  title = {SmartBoxes for Interactive Urban Reconstruction},
  author = {Liangliang Nan and Andrei Sharf and Hao Zhang and Daniel Cohen-Or and Baoquan Chen},
  journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH 2010)},
  volume = {29},
  number = {4},
  pages = {Article 93},
  year = {2010}