SeMLI-RAG: Semantics-Guided Multimodal Late Interaction for Retrieval-Augmented Radiology Report Generation
| Tipo de publicação: | Artigo |
| Citação: | |
| Publication status: | Published |
| Journal: | Association for Computational Linguistics |
| Volume: | Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing |
| Ano: | 2026 |
| Mês: | October |
| Páginas: | P. 54040-54055 |
| DOI: | 10.18653/2026.emnlp-main.2379 |
| Resumo: | The consultation of reference cases in clinical radiology reporting motivates integrating mul timodal retrieval-augmented generation (RAG) into automated systems. However, traditional RAG approaches employ single-vector embed Coarse e.g., PromptMRG Global vector Selective injection Fine-grained SeMLI-RAG (Ours) Fine-grained retriever Selective injection Post-hoc validation Adaptive Selection dings for indexing, resulting in loss of gran ular information. Moreover, a semantic gap exists whereby visually similar radiographs of ten encode distinct pathologies. In this paper, we propose a Semantics-guided Multimodal Late Interaction RAG (SeMLI-RAG) frame work for radiology report generation. Our ap proach employs a weight-based late interaction strategy for retrieval to capture fine-grained ra diographic features. Furthermore, we introduce an image-adaptive RAG to selectively leverage retrieved information, mitigating performance degradation caused by retrieval-induced noise. Experiments on three radiology benchmarks demonstrate that our framework yields con sistent improvements over the backbone and RAG baselines across multiple evaluation di mensions. Further analysis shows that improv ing retrieval granularity and retrieval robustness can both benefit radiology report generation |
| Palavras-chave: | |
| Autores | |
| Adicionado por: | [] |
| Total mark: | 0 |
|
Anexos
|
|
|
Notas
|
|
|
|
|
|
Tópicos
|
|
|
|
|
