Deep Learning of Oriented Bounding Box Regression Networks for Ship Detection in Optical Satellite Images

Abstract

Maritime surveillance is important for management of maritime traffic and to prevent activities like illegal fishing, hazardous cargo transportation, piracy, and smuggling of goods and humans. Remote sensing is frequently used for positioning vessels that are not transmitting via the Automatic Identification System (AIS). Modern optical remote sensing instruments provide high-resolutional imagery, allowing for advanced analyses of the Earth’s surface. Human operators are trained to recognize different structures and objects in satellite images, resulting in precise scene analyzes. This endeavor is, however, time consuming and expensive, and the earth observation community is continuously researching how to effectively and precisely automate this process. For ship detection in remote sensing images, state-of-the-art architectures are based on deep neural networks. This thesis contributes data, experiments and architectures that are based on deep neural networks. Recognizing vessel heading may be useful for assessing its intentions, and is an interesting topic in the application field which will be studied in this thesis. This is obtained by deep learning of regression networks that assign rotated bounding boxes to detected vessels. A data set with high-resolutional SuperView optical satellite imagery and rotatable bounding box annotations is contributed by this thesis. Experiments on five reference object detectors are performed, giving results on the reliability of running ship detection services on SuperView images. Scenes of high object density are studied. Explicitly, experimental results on the newly proposed Object Detection with Grouped Instances (ODGI) (Royer and Lampert, 2020) show slightly increased performance when utilizing grouped object instances, compared to equivalent models that use individual object instances. The novel Oriented YOLOv2 and Oriented Tiny YOLO neural network architectures, extending from YOLOv2 (Redmon and Farhadi, 2017) and Tiny YOLO, recognize object rotation and provide a more accurate shape description than the predecessors. These are used in the novel Oriented Object Detection with Grouped Instances (OODGI) pipeline, extending from the newly proposed ODGI (Royer and Lampert, 2020), to utilize object groupings while providing positioning, shape and rotation predictions. An additional error analysis of 11 reference and novel neural network architectures is supplemented to study model sensitivities. Experiments on performance consistency of deep neural network architectures when the amount of training data is limited reveal that resulting precision varies over different training sessions. This variation is discussed to be induced by stochasticity in weight initialization and batch selection. The experimental results indicate that Faster R-CNN has the highest precision. However, ODGI is three times faster and has competitive precision. The novel models proposed in this thesis successfully describe positioning, shape and orientation of ships, although OODGI needs some amendment.