LocSAM3
LocSAM3: Box-Supervised Adaptation of SAM3 for Text-Only Chest X-Ray Segmentation
Kirellos Nashed1, Mina Heinein1, Mina Youssef1, Tamer Basha1, Hasan Md Tusfiqur Alam2, Abdulrahman Mohamed Selim2, 3, Omair Shahzad Bhatti2, Daniel Sonntag2, 3
1 Department of Systems and Biomedical Engineering, Faculty of Engineering, Cairo University, Giza, Egypt 2 German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany 3 University of Oldenburg, Oldenburg, Germany
Accepted as an Oral Presentation at MSB EMERGE 2026 (MICCAI Workshop)
Supplementary Material & Extended Ablation Studies
This project website provides the extended empirical analyses, sensitivity plots, and baseline comparisons that could not be included in the main manuscript due to strict page limitations.
1. Box-Prompt Dropout Sensitivity Analysis ($r$-Sweep)
In Section 4.4 of the main text, we discuss the core mechanism of LocSAM3: decoupling the bounding box’s role as a supervision target from its role as an inference prompt.
To isolate and quantify the effect of this mechanism, we evaluated the model’s text-only inference performance across a complete sweep of box-prompt inclusion rates ($r \in {0.0, 0.2, 0.4, 0.6, 0.8, 1.0}$) during training. The table below demonstrates the clear performance peak at $r=0.6$, highlighting the severe performance degradation that occurs when the model is either never exposed to spatial prompts ($r=0.0$) or entirely dependent on them ($r=1.0$).
| Bounding Box Inclusion Ratio ($r$) | Left Lung (mIoU) | Right Lung (mIoU) | Heart (mIoU) | OVERALL (mIoU) |
|---|---|---|---|---|
| 0.0 (Always text-only) | 0.8474 | 0.8481 | 0.6703 | 0.7886 |
| 0.2 | 0.8492 | 0.8502 | 0.6810 | 0.7935 |
| 0.4 | 0.8485 | 0.8521 | 0.6987 | 0.7998 |
| 0.6 (LocSAM3 Peak) | 0.8462 | 0.8508 | 0.7131 | 0.8034 |
| 0.8 | 0.8447 | 0.8489 | 0.7023 | 0.7986 |
| 1.0 (Always boxed) | 0.8222 | 0.7619 | 0.0154 | 0.5331 |
2. Additive Loss Component Ablation
Conducting a full-scale ablation across the entire MIMIC-CXR dataset for every loss combination is computationally prohibitive. To isolate the relative contributions of our objective function components, we conducted a targeted additive ablation study utilizing a representative 5% subset of the training data.
This controlled experiment quantifies the step-wise performance gains of adding Generalized IoU and Presence losses to the foundational classification and bounding box regression base.
| Category | Samples | Base (mIoU) | Base (Dice) | + gIoU (mIoU) | + gIoU (Dice) | + pres (mIoU) | + pres (Dice) |
|---|---|---|---|---|---|---|---|
| Overall Pathology | 2217 | 0.4187 | 0.5227 | 0.4293 | 0.5336 | 0.4401 | 0.5526 |
| Overall Anatomy | 330 | 0.5555 | 0.6112 | 0.5701 | 0.6248 | 0.5717 | 0.6311 |
Note: Base = Box + Classification Loss. + gIoU adds Generalized IoU. + pres adds Binary Cross-Entropy Presence Loss.
3. Extended Baseline Comparisons
During our initial investigations, we evaluated several additional baselines, including native SAM3, pure text-native vision-language models, and MedSAM3. However, due to the extreme domain shift of thoracic radiography (characterized by overlapping 2D projections and grayscale spatial clutter), these models exhibited catastrophic spatial collapse under text-only inference. We selected Medical-SAM3 as the exclusive baseline in our manuscript because it shares our exact foundational architecture (SAM3). This ensures a matched-architecture control, proving that the performance jump is driven entirely by our domain adaptation strategy and box-prompt dropout mechanism, rather than underlying differences in model parameters.
Medical-SAM3’s failure for Text-Only inference is primarily a localization failure, not purely a mask-decoder failure. In Table 2 of the manuscript, Medical-SAM3 performs much better when given a bounding box at inference than under text-only inference; for pathology, its mIoU increases from 0.085 with text-only to 0.400 with text+box. This suggests that when spatial guidance is supplied, the model can often refine the prompted region into a reasonable mask. However, without the spatial prompt, the model struggles to map a text concept such as “effusion” or “pneumonia” to the correct radiographic region. We believe this is amplified in chest X-rays because thoracic anatomy is highly overlapping, abnormalities are often diffuse, and many concepts share similar low-contrast appearances.
To visualize this prompt dependency, the chart below illustrates the catastrophic spatial collapse of the baseline when the bounding box is removed, contrasting sharply with LocSAM3’s robust text-only retention.
