A Search-Classify Approach for Cluttered Indoor Scene Understanding

ACM Transactions on Graphics (Proceedings SIGGRAPH Asia 2012)

Liangliang Nan1   Ke Xie1  Andrei Sharf2  

1SIAT, China
2Ben Gurion University, Israel

Figure 1: A raw scan of a highly cluttered indoor scene is given (left). Applying our search-classify method, we segment the scene into meaningful objects (middle: chairs (blue) and tables (purple)), followed by a template deform-to-fit reconstruction (right).



We present an algorithm for recognition and reconstruction of scanned 3D indoor scenes. 3D indoor reconstruction is particularly challenging due to object interferences, occlusions and overlapping which yield incomplete yet very complex scene arrangements. Since it is hard to assemble scanned segments into complete models, traditional methods for object recognition and reconstruction would be inefficient. We present a search-classify approach which interleaves segmentation and classification in an iterative manner. Using a robust classifier we traverse the scene and gradually propagate classification information. We reinforce classifiation by a template fiting step which yields a scene reconstruction. We deform-to-fit templates to classified objects to resolve classfication ambiguities. The resulting reconstruction is an approximation which captures the general scene arrangement. Our results demonstrate successful classification and reconstruction of cluttered indoor scenes, captured in just few minutes.



Figure 2: A zoom into the cluttered region of Figure 1 (left), reveal that accurate segmentation and classification are challenging, even for human perception. We initially oversegment the scene (mid-left) and search-classify meaningful objects in the scene (mid-right), that are reconstructed by templates (right) overcoming the high clutter.


Figure 3: Two examples of final classification likelihoods and template fitting values. Mid-left figure shows classified scene with likelihood values per object. Mid-right figure shows three best fitting templates to each object with average matching distance.

Paper [17M pdf]
Slides [13M pptx]
Data & Viewer [290M RAR]


We thank the reviewers for their valuable comments and suggestions. This work was supported in part by NSFC (61272327, 61003190, 61232011), National 863 Program (2012AA011802, 2011AA010503), Guangdong Science and Technology Program (2011B050200007) and Shenzhen Science and Technology Foundation (JC201005270340A, CXB201104220029A, JC201005270329A).



  title = {A Search-Classify Approach for Cluttered Indoor Scene Understanding},
  author = {Liangliang Nan and Ke Xie and Andrei Sharf},
  journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2012)},
  volume = {31},
  number = {6},
  year = {2012}