Li, Hao; Masood, Anum. (2026).听.听The Visual Computer, 42(9), 371.听
This study addresses the challenge of聽automatic polyp segmentation, the process of identifying and outlining polyps in colonoscopy images. While many artificial intelligence (AI) models perform well on the data they were trained on, they often struggle when tested on images from different hospitals or datasets because polyps can vary widely in size, shape, color, and appearance. Differences in image quality, such as glare from reflected light, blur, fluid, and changes in lighting during colonoscopy, add to this challenge. To improve performance across different datasets, the researchers developed聽HAT-SAM3, an adaptation of the聽Segment Anything Model 3 (SAM 3), a large pre-trained AI model originally designed for image segmentation. HAT-SAM3 uses a training technique called聽highlight-aware training (HAT), which helps the model better handle bright reflections commonly seen in endoscopy images without changing how the model operates during testing. When evaluated on five publicly available datasets, HAT-SAM3 achieved an average聽Dice score(a measure of how closely the model鈥檚 predicted outlines match the true polyp boundaries) of 0.884 and produced the best reported results on four of the five datasets. In additional testing, where the model was trained on one dataset and evaluated on three completely separate datasets without further training, HAT-SAM3 consistently outperformed the strongest previously reported method. These results suggest that adapting a pre-trained foundation model with endoscopy-specific training techniques can improve the accuracy and reliability of automated polyp segmentation across a wide range of clinical datasets. The researchers have also made their code publicly available to support future research.
Fig 1: Motivation of HAT-SAM3. A task-specific polyp segmenter can perform well when training and test images come from the same visual distribution, but degrade under cross-domain changes in illumination, blur, specular reflection, and polyp appearance. HAT-SAM3 uses a broad segmentation prior with endoscopy-aware adaptation to improve cross-domain polyp segmentation