Simultaneous Semantic and Instance Segmentation for Colon Nuclei Identification and Counting

Abstract

Nucleus segmentation and classification within the Haematoxylin and Eosin stained histology images is a key component in computer-aided image analysis, which helps to extract features with rich information for cellular estimation and following diagnosis. Therefore, it is of great relevance for several downstream computational pathology applications In this work, we address the problem of automatic nuclear segmentation and classification. Our solution is to cast as a simultaneous semantic and instance segmentation framework, and it is part of the Colon Nuclei Identification and Counting (CoNIC) Challenge. Our framework is a carefully designed ensemble model. We first train a semantic and an instance segmentation model separately, where we use as backbone HoverNet and Cascade Mask-RCNN models. We then ensemble the results with a customized Non-Maximum Suppression embedding algorithm. From our experiments, we observe that the semantic segmentation part can achieve an accurate class prediction for the cells whilst the instance information provides a refined segmentation. We enforce a robust segmentation and classification result through our customized embedding algorithm. We demonstrate, through our visual and numerical experimental, that our model outperforms the provided baselines by a large margin. Our solution ranked as the $4^{th}$ solution on the Grand Challenge CoNIC 2022.