Automatic Building Structure Extraction from Aerial Photographs using Transformers

Abstract

This thesis explores a method for automatically extracting three-dimensional, geolocated building roof structure representations from non-orthorectified aerial photos, with a view to automate time intensive tasks that are currently handled manually. A two stage framework for 3D roof structure inference from aerial photos is proposed. A transformer-based deep learning framework is used to extract the roof structure of buildings from non-orthorectified aerial photos, and a multi-view triangulation method is used to transform the extracted planar graphs to a 3D representation of the building roof without the use of a digital surface model. Additionally, a preprocessing method for producing training data for the roof structure extraction model using aerial images and existing 3D building graph representations from a national database is developed. The proposed framework is shown to successfully infer building roof structure graphs, achieving an F1-score of 0.77 on the test set, and a set of inferred graphs are successfully transformed to 3D with a mean average corner reconstruction error of 0.43 meters. The results are promising, and automatic extraction and 3D reconstruction is viable, but the process requires further development.