Exploring the possibility of using a deep learning based pipeline for seal detection and classification from high-resolution optical satellite images

Abstract

In this thesis, we investigate the possibility of using a deep learning based pipeline for detection and classification of seals in high-resolution optical satellite images. The pipeline includes a seal detector which uses a UNet-3+ architecture and a clustering module for point level prediction, and an image level convolutional neural network (CNN) seal classifier. Areal images targeting the seal whelping grounds of the East coast of Greenland during the month of March in 2022 was used for this analysis. For the purpose of testing the pipeline aerial images were downsampled to 0,3m spatial resolution to emulates high resolution satellite images. The aerial image dataset was accompanied by expert labeled annotations of 1866 seal placements. A manual investigation of the images used was done resulting in another 696 annotated seals added to the dataset.

Using red, green and blue (RGB) images with spatial resolution of 0,3m the seal detector was able to achieve a F1-score of 0,77 with a precision of 0,80 and recall of 0,75. With the same data the seal classifier was able to separate adult and pup seals with an accuracy of 96.3% and between harp and hooded seals with an accuracy of 82.8%.

From the test with different combination of the RGB channels. The best performing single channel was the blue channel and the best performing two channel combination was the red-blue for both the detector and the classifier.

In this work we evaluated two different methods for making the detector work with pan-sharpened images: one trained on 0,3m resolution RGB images (without pan-sharpening) and one trained directly on pan-sharpened images. These two methods were tested on a range of degraded spatial resolution for the color channels. Both methods show a clear drop in performance with decreased spatial resolution. The results also show that for a spatial resolutions finer then 0,8m, the model trained on the original RGB model outperformed the model trained on the pan-sharpened images. However, at the coarser resolution (0,8m and lower), the model trained on the pan-sharpened images performed better.

In this work we also explored the possibility of skipping the pan-sharpening by just linearly upsampling the color channels and concatenate it with the panchromatic channels to make a 4 channel input. From this we found that it generally worked better then when using pan-sharpening. It also showed that the F1-score steadily dropped until reaching a spatial resolution of approximately 1,2m, where it flattened out indicating that at that point the network mostly rely on the panchromatic channel. It was also investigated if the performance of the seal detector could be increased by adding a variable to the seal classifier to filter out false positives. We saw a increase in the F1-score for all the models tested, but the increase was less for the models with an already high precision score.

From the test done the best performance was achieved using the RGB images. The lower spatial resolution for the color channels compared to the panchromatic meant that the best method to utilize the optical data was to linearly upsample the color channels to match the pixel resolution of the panchromatic channel. This method outperformed using pan-sharpening and has the added benefit of being more flexible on the color channels/spectral bands which can be used.

In this thesis we have demonstrated that detection and classification of seals using high resolution optical satellite images is possible. One of the biggest limitations for use of high resolution optical satellite images for this task is its availability. One of the biggest improvements in satellite technology which would increase the performance for detection would be increasing the spatial resolution of the color channels/multi-spectral bands.