Irène Apra
3D data integration into a Convolutional Neural Network (CNN) on RGB aerial images to improve the semantic segmentation of roof superstructures on buildings
The thesis is written in cooperation with the faculty of mechanical engineering of the Technical University of Munich where a pipeline is being developed at the chair of automotive engineering to assess the PV-potential of buildings in Bavaria. The current state includes two Convolution Neural Networks (CNN) assessing the roof geometrical potential for PV installation. The first network segments a roof into 16 azimuth classes; the second detects installations on the roof, called “superstructures”, as single- or multi-classes (ladder, chimney…). The wider pipeline considers the physical suitability of roofs, as well as the technical and economical potentials for PV installation.
The motivation of the thesis is to improve the geometrical potential assessment of roofs by extending the existing network(s) with 3D data. Despite little research conducted on CNN for RGB-Height (Z) data, it has recently been established that Z information added to a CNN can improve the accuracy for classifying aerial imagery.
Based on these results and the existing work at the chair, we will explore the following question: How can 3D data be best integrated into a CNN for RGB aerial images to improve the semantic segmentation of roof superstructures on buildings? Various types of height data and their combination with the segmentation results of the second network will be explored, as well as their integration to the existing architecture and the evolution of the results accuracy.
Supervisors: Ken Arroyo Ohori + Giorgio Agugiaro