[BibTeX] [RIS]
SeMLI-RAG: Semantics-Guided Multimodal Late Interaction for Retrieval-Augmented Radiology Report Generation
Tipo de publicação: Artigo
Citação:
Publication status: Published
Journal: Association for Computational Linguistics
Volume: Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ano: 2026
Mês: October
Páginas: P. 54040-54055
DOI: 10.18653/2026.emnlp-main.2379
Resumo: The consultation of reference cases in clinical radiology reporting motivates integrating mul timodal retrieval-augmented generation (RAG) into automated systems. However, traditional RAG approaches employ single-vector embed Coarse e.g., PromptMRG Global vector Selective injection Fine-grained SeMLI-RAG (Ours) Fine-grained retriever Selective injection Post-hoc validation Adaptive Selection dings for indexing, resulting in loss of gran ular information. Moreover, a semantic gap exists whereby visually similar radiographs of ten encode distinct pathologies. In this paper, we propose a Semantics-guided Multimodal Late Interaction RAG (SeMLI-RAG) frame work for radiology report generation. Our ap proach employs a weight-based late interaction strategy for retrieval to capture fine-grained ra diographic features. Furthermore, we introduce an image-adaptive RAG to selectively leverage retrieved information, mitigating performance degradation caused by retrieval-induced noise. Experiments on three radiology benchmarks demonstrate that our framework yields con sistent improvements over the backbone and RAG baselines across multiple evaluation di mensions. Further analysis shows that improv ing retrieval granularity and retrieval robustness can both benefit radiology report generation
Palavras-chave:
Autores He, Jiawei
Feng, Weibo
Ferdowsi, Sohrab
Yazdani, Anthony
Zhang, Boya
Cinar-Koras, Osman Alperen
Khattab, Sameh
Corbeil, Jean-Philippe
Müller, Henning
Teodoro, Douglas
Adicionado por: []
Total mark: 0
Anexos
  • 2026_SeMLI-RAG Semantics-Guide...
Notas
    Tópicos