Machine Learning-based Classification, Detection, and Segmentation of Medical Images

Abstract

Gastrointestinal tract (GI) cancers are among the most common types of cancers worldwide. In particular, colorectal cancer (CRC) is the most lethal in terms of number of incidences and mortality (third most common cause of cancer and the second common cause of cancer-related deaths). Colonoscopy is the gold standard for screening patients for CRC. During the colonoscopy, gastroenterologists examine the large bowel, detect precancerous abnormal tissue growths like polyps and remove them through the scope if necessary. Although colonoscopy is considered the gold standard, it is an operator-dependent procedure. Previous research has shown large missing rates for GI abnormalities, e.g., polyp miss detection is around 22\%-28\%. Early detection of GI lesions and cancers at the curable stage can help reduce the mortality rate. The development of automated, accurate, and efficient methods for the detection of the GI cancers could benefit both gastroenterologists and patients. In addition, if integrated into screening programs, an automatic analysis could improve overall GI endoscopy quality.

The medical field is becoming more interdisciplinary, and the importance of medical image data is increasing rapidly. Medical image analysis can play a central role in disease detection, diagnosis, and treatment. With the increasing number of medical images, there is enormous potential to improve the screening quality. Deep learning (DL), in particular, convolutional neural network (CNN) based models have tremendous potential to automate and enhance the medical image analysis procedure and provide an accurate diagnosis. The automated analysis of the medical images could reduce the burden of the medical experts and provide quality and accessible healthcare to a larger population. In medical imaging, classification, detection, and semantic segmentation tasks are crucial for clinical practice. The development of accurate and efficient computer aided diagnosis (cadx) or computer aided detection (cade) models can help to identify the abnormalities at an early stage and can act as a third eye for the doctors.

To this end, we have studied and designed machine learning (ML) and DL based architectures for GI tract disease classification, detection, and segmentation. Our designed architectures can classify different types of GI tract findings and abnormalities accurately with high performance. Our contribution towards the development of CADe models for automated polyp detection showed improved performance. Out of three different medical imaging tasks, semantic segmentation of medical imaging data plays a significant role in extracting meaningful information from images by classifying each pixel and segmenting it by class. Using the GI case scenario, we have mainly worked on polyp segmentation and proposed and evaluated different automated polyp segmentation architectures. We have also built architectures for surgical instrument segmentation that showed high performance and real-time speed.

We have collected, annotated, and released several open-access datasets such as HyperKvasir, KvasirCapsule, PolypGen, Kvasir-SEG, Kvasir-instrument, and KvasirCapsule-SEG in collaboration with hospitals in Norway and abroad to address the lack of datasets in the field. We have devised several medical image segmentation architectures (for example, ResUNet++, DoubleU-Net, and ResUNet + CRF + TTA) that provided improved results with the publicly available datasets. Beside that, we have also designed architectures that have the capability of segmenting polyps in real-time with high frame per second (FPS) (for example, ColonSegNet, NanoNet, PNS-Net, and DDANet). Moreover, we performed extensive studies on the generalizability of our models on public datasets, and by creating a dataset consisting of data from different hospitals, we allow multi-center cross dataset testing. Our results prove that proposed DL based CADx systems might be of great assistance to clinicians in the future.