[BibTeX] [RIS]
SeMLI-RAG: Semantics-Guided Multimodal Late Interaction for Retrieval-Augmented Radiology Report Generation
Type of publication: Article
Citation:
Publication status: Published
Journal: Association for Computational Linguistics
Volume: Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Year: 2026
Month: October
Pages: P. 54040-54055
DOI: 10.18653/2026.emnlp-main.2379
Abstract: The consultation of reference cases in clinical radiology reporting motivates integrating mul timodal retrieval-augmented generation (RAG) into automated systems. However, traditional RAG approaches employ single-vector embed Coarse e.g., PromptMRG Global vector Selective injection Fine-grained SeMLI-RAG (Ours) Fine-grained retriever Selective injection Post-hoc validation Adaptive Selection dings for indexing, resulting in loss of gran ular information. Moreover, a semantic gap exists whereby visually similar radiographs of ten encode distinct pathologies. In this paper, we propose a Semantics-guided Multimodal Late Interaction RAG (SeMLI-RAG) frame work for radiology report generation. Our ap proach employs a weight-based late interaction strategy for retrieval to capture fine-grained ra diographic features. Furthermore, we introduce an image-adaptive RAG to selectively leverage retrieved information, mitigating performance degradation caused by retrieval-induced noise. Experiments on three radiology benchmarks demonstrate that our framework yields con sistent improvements over the backbone and RAG baselines across multiple evaluation di mensions. Further analysis shows that improv ing retrieval granularity and retrieval robustness can both benefit radiology report generation
Keywords:
Authors He, Jiawei
Feng, Weibo
Ferdowsi, Sohrab
Yazdani, Anthony
Zhang, Boya
Cinar-Koras, Osman Alperen
Khattab, Sameh
Corbeil, Jean-Philippe
Müller, Henning
Teodoro, Douglas
Added by: []
Total mark: 0
Attachments
  • 2026_SeMLI-RAG Semantics-Guide...
Notes
    Topics