Advancing Deep Learning for Marine Environment Monitoring

Abstract

Marine environment monitoring has become increasingly significant due to the excessive exploitation of oceans, which detrimentally impacts ecosystems. Deep learning provides an effective monitoring approach by automating the analysis of vast amounts of observed image data, enabling stakeholders to make informed decisions regarding fishing quotas or conservation efforts.

The success of deep learning is often attributed to its capacity to extract relevant features from data, without the need for handcrafted rules or heuristics. However, this capability is not without limitations, as the intricate feature extraction process of deep learning-based systems poses fundamental challenges.

A lack of annotated data presents an inherent challenge for deep learning. The widespread success of deep learning has primarily relied on the ample availability of annotated data, while deep learning models encounter difficulties when learning from limited annotations. However, obtaining annotated data is expensive, particularly in the context of marine environment monitoring, as it is often a manual process demanding the expertise of domain specialists.

Another challenge of deep learning is a lack of explainability. The black-box nature of deep learning models can make it difficult to understand how they arrive at their decisions. This hinders their adoption in critical decision-making processes, as stakeholders may be hesitant to trust models whose decision-making rationale is not transparent or interpretable.

To address the challenges and further advance deep learning methodologies, this thesis proposes three novel deep learning methods, highlighting marine environment monitoring as an application domain. The dependence on annotated data is addressed through two novel semi-supervised methods demonstrated in different image tasks. The central operational logic in both methods entails alternating between supervised learning and unsupervised deep clustering within a single network, merging data structure uncovered through unsupervised clustering with a small amount of ground-truth class information. Both methods employ multi-frequency echosounder data to demonstrate their effectiveness in marine environment monitoring, outperforming conventional approaches.

Moreover, a new explainable deep learning method is proposed to address the lack of explainability. This method generates explanations for its decisions while adhering to user-centered explanation principles, such as minimality, sufficiency, and interactivity. The information-bottleneck framework provides a theoretical ground to pursue minimality and sufficiency, while interactivity is accomplished by integrating additional domain knowledge into the training process, enabling the generated explanations to evolve accordingly. The method is validated using a variety of marine image datasets, encompassing multi-frequency echosounder data and seal pup images on sea ice.

While the monitoring of marine environments is a significant focus, the primary aim of the thesis is to contribute to the advancements of deep learning methodologies. As such, the proposed methods are designed to be generic and possess the potential for broader applicability across various domains. We believe that the methods presented in this thesis hold the promise of fostering a more effective, user-centered, and transparent approach to deep learning, as well as facilitating our efforts to preserve the marine environment and promote sustainable ocean stewardship.