Deep Image Clustering with Tensor Kernels and Unsupervised Companion Objectives
AuthorTrosten, Daniel Johansen
Deep image clustering is a rapidly growing branch of machine learning and computer vision, in which deep neural networks are trained to discover groups within a set of images, in an unsupervised manner. Deep neural networks have proven to be immensely successful in several machine learning tasks, but the majority of these advances have been in supervised settings. The process of labeling data for supervised applications can be extremely time-consuming, or even completely infeasible in many domains. This has led researchers to shift their focus towards the deep clustering field. However, this field is still in its infancy, meaning that it includes several open research questions, regarding e.g. the design and optimization of the algorithms, the discovery of meaningful clusters, and the initialization of model parameters. In an attempt to address some of these open questions, a new algorithm for deep image clustering is developed in this thesis. The proposed Deep Tensor Kernel Clustering (DTKC) consists of a convolutional neural network (CNN), which is trained to reflect a common cluster structure at the output of all its intermediate layers. Encouraging a consistent cluster structure throughout the network has the potential to guide it towards meaningful clusters, even though these clusters might appear to be nonlinear in the input space. The cluster structure is enforced through the idea of companion objectives, where separate loss functions are attached to each of the layers in the network. These companion objectives are constructed based on a proposed generalization of the Cauchy-Schwarz (CS) divergence, from vectors to tensors of arbitrary rank. Generalizing the CS divergence to tensor-valued data is a crucial step, due to the tensorial nature of the intermediate representations in the CNN. Furthermore, an alternate initialization strategy based on self-supervised learning, is also employed. To the author's best knowledge, this is the first attempt at using this particular self-supervised learning approach to initialize a deep clustering algorithm. Several experiments are conducted to thoroughly assess the performance of the proposed DTKC model, with and without self-supervised pre-training. The results show that the models outperform, or perform comparable to, a wide range of benchmark algorithms from the literature.
PublisherUiT Norges arktiske universitet
UiT The Arctic University of Norway
The following license file are associated with this item:
Showing items related by title, author, creator and subject.
Baadshaug, Ole (Master thesis; Mastergradsoppgave, 2018-06-29)Moving icebergs represent a major problem for shipping, as well as for oil and gas installations in ice infested waters. To be able to take actions against hazardous icebergs, it is necessary to develop models for prediction of iceberg drift trajectories. Many models have been developed in order to do so, using different approaches. These approaches can be divided into two main categories, dynamic ...
Schulz, Jörn (Doctoral thesis; Doktorgradsavhandling, 2013-12-18)The use of statistical shape analysis in medical settings has increased during the last decades. This thesis presents contributions to three major topics of statistical shape analysis with application to medical problems. These topics are: the modeling of the shape by a geometrical model, the study of rotational shape deformations and the comparison of shapes between populations. Paper I presents ...
Johansen, Thomas A. Haugland (Master thesis; Mastergradsoppgave, 2016-12-08)The key objectives in this thesis are; the study of GPU-accelerated eigenvalue decomposition in an effort to uncover both benefits and pitfalls, and then to investigate and facilitate a future GPU implementation of the symmetric QR algorithm with permutations. With the current trend of having ever larger datasets both in terms of features and observations, we propose that GPU computation can help ...