MeddeX: A Retrieval-Augmented LLaMA-3 System with Gemini Vision for Evidence-Backed Medical Report Analysis

This paper presented MeddeX, a modular medical report analysis system combining a QLoRA fine-tuned LLaMA-3 (8B) model with RAG, a Gemini Vision multimodal OCR layer, and a suite of robustness and clinical-utility features developed through iterative engineering. Beyond the core sum-marization and Q&A pipeline, MeddeX introduces garbage-text detection for corrupted PDF handling, report-scoped retrieval for precise contextual grounding, an automated longitudinal comparison engine for tracking patient health trends, and a lazy backfill pipeline for transparent processing of legacy documents. Quantitative evaluation demonstrates consistent improvements over the base LLaMA3-Med42-8B model across all five metrics, with a particularly strong BLEU improvement of +71.4%, validating the domain fine-tuning approach. Together, these contributions position MeddeX as a comprehensive, production-oriented clinical AI system.

[1]     J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186. [2]     J. Lee et al., “BioBERT: A pre-trained biomedical language representa-tion model for biomedical text mining,” Bioinformatics, vol. 36, no. 4, pp. 1234–1240, 2020. [3]     M. Lewis et al., “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,” in Proc. ACL, 2020, pp. 7871–7880. [4]     I. Beltagy, M. E. Peters, and A. Cohan, “Longformer: The Long-Document Transformer,” arXiv:2004.05150, 2020. [5]     P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in Proc. NeurIPS, 2020, pp. 9459–9474. [6]     H. Touvron et al., “LLaMA: Open and Efficient Foundation Language Models,” arXiv:2302.13971, 2023. [7]     T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” in Proc. NeurIPS, 2023. [8]     Gemini Team, Google, “Gemini: A Family of Highly Capable Multi-modal Models,” arXiv:2312.11805, 2023.