A Hybrid Retrieval-Augmented Generation Architecture for Biomedical Question Answering Using FastAPI, ChromaDB, and Adaptive Threshold-Based Response Routing

This paper presented the design of a modular biomedical question-answering system built around a FastAPI service layer, a hybrid retrieval pipeline combining ChromaDB-based semantic search with BM25 lexical search, a cross-encoder reranking stage, and an adaptive threshold-based mechanism for routing between grounded and fallback generation modes. The architecture separates ingestion, retrieval, reranking, and generation into independent, replaceable services, and supports incremental knowledge base construction from both uploaded PDF documents and PubMed abstracts. By explicitly distinguishing between queries that can be answered from retrieved evidence and those that cannot, the system aims to reduce the risk of presenting ungrounded content as though it were evidence-based, addressing a concern that is not fully resolved by hybrid retrieval alone. The immediate next step for this work is the completion of controlled empirical evaluation using the metrics and table structures defined in Sections XI and XII, followed by threshold calibration against labeled data. Longer-term extensions toward knowledge-graph integration, multimodal retrieval, and federated search offer a path toward broader clinical and research applicability.

[1] Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., Schaekermann, M., Wang, A., Amin, M., Lachgar, S., Mansfield, P., Prakash, S., Green, B., Dominowska, E., Arcas, B. A., Tomasev, N., Liu, Y., Wong, R., Semturs, C., Mahdavi, S. S., Barral, J., Webster, D., Corrado, G. S., Matias, Y., Azizi, S., Karthikesalingam, A., & Natarajan, V. (2023). Towards expert-level medical question answering with large language models. arXiv preprint arXiv:2305.09617. [2] Asai, A., Zhong, Z., Chen, D., Koh, P. W., Zettlemoyer, L., Hajishirzi, H., & Yih, W. (2024). Reliable, adaptable, and attributable language models with retrieval. arXiv preprint arXiv:2403.03187. [3] Tonmoy, S. M. T. I., Zaman, S. M. M., Jain, V., Rani, A., Rawte, V., Chadha, A., & Das, A. (2024). A comprehensive survey of hallucination mitigation techniques in large language models. arXiv preprint arXiv:2401.01313. [4] Alonso, I., Oronoz, M., & Agerri, R. (2024). MedExpQA: Multilingual benchmarking of large language models for medical question answering. arXiv preprint arXiv:2404.05590. [5] Xiong, G., Jin, Q., Lu, Z., & Zhang, A. (2024). Benchmarking retrieval-augmented generation for medicine. arXiv preprint arXiv:2402.13178. [6] Robertson, S., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4), 333–389. [7] Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020). BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4), 1234–1240. https://doi.org/10.1093/bioinformatics/btz682 [8] Jin, Q., Kim, W., Chen, Q., Comeau, D. C., Yeganova, L., Wilbur, W. J., & Lu, Z. (2023). MedCPT: Contrastive pre-trained transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval. Bioinformatics, 39(11), btad651. [9] Ma, X., Yang, Z., & Zhang, H. (2020). A hybrid first-stage retrieval model for biomedical literature. Proceedings of the CEUR Workshop on Biomedical Information Retrieval. https://ceur-ws.org/Vol-2696/paper_92.pdf [10] Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474.