LuGSAM: a novel framework for integrating text prompts to Segment Anything Model (SAM) for segmentation tasks of ICU chest x-rays

Segmenting lung regions in ICU Chest X-rays (CXR’s) is vital for diagnosing lung-related disorders, but existing methods require extensive annotations or training on large datasets. We present LuGSAM, a novel framework that integrates text prompts with the Segment Anything Model (SAM) for segmentation tasks, enhancing precision and adaptability in clinical settings. Our approach combines Grounding DINO, a zero-shot object detector using textual prompts (e.g., "right lobe"), and Meta AI’s SAM. Grounding DINO generates bounding boxes based on word-level prompts. These bounding boxes serve as an input to SAM, to generate precise segmentation masks. To further improve accuracy, we propose an iterative bounding box adjustment algorithm that refines object detections through multiple iterations. The Vision Transformer huge (Vit-h) variant of SAM achieved the highest overlap score (IoU = 0.95) for right lung segmentation. Grounding DINO demonstrated high detection accuracy for prompts like “right lung” with a confidence score of 0.58. The Binarized Predicted IoU (BPIoU) metric showed significant improvements in segmentation quality, making this framework a promising tool for clinical applications.

Duke Scholars

Author Rishi Kamaleswaran Trauma, Acute, and Critical Care Surgery