Beyond Embedding-Based Search: A Comparative Study of Vector and Vectorless Retrieval in RAG

This paper presents a conceptual comparison of vector-based dense retrieval and vectorless sparse retrieval within Retrieval-Augmented Generation pipelines. Sparse methods deliver exceptional computational efficiency and keyword pre-cision for structured domains and latency-sensitive deploy-ments. Dense embedding-based retrieval enables semantic matching essential for paraphrased and multilingual queries. Neither approach dominates universally; optimal choice de-pends on query distribution, corpus characteristics, and infras-tructure constraints. Hybrid and adaptive retrieval architectures that combine both signals represent the most robust solution for general-purpose production RAG systems. Empirical benchmarks con-sistently show hybrid configurations outperforming individual retrievers across precision, recall, and hallucination metrics, with the infrastructure overhead justified by quality improve-ments in most application contexts [4]. Future research should focus on learned retriever selec-tion via lightweight classifiers, progressive hybrid indexing for dynamic corpora, and embedding-free graph retrieval for structured domains [9]. Long-term, end-to-end differentiable RAG pipelines jointly optimizing retrieval and generation objectives represent a promising direction for further quality gains [15]. Immediate engineering priorities include deploying query-based routing classifiers and evaluating cascade retrieval architectures in production-grade systems. Beyond the immediate engineering horizon, the broader challenge of aligning retrieval objectives with human evalu-ation criteria remains open. Automated retrieval metrics such as NDCG and MRR correlate imperfectly with user satisfac-tion in deployed conversational systems. Developing richer evaluation protocols that capture not only factual correctness but also response coherence, citation transparency, and user trust calibration will be essential as RAG systems take on increasingly consequential roles in information access and decision support [16].

[1]     P. Lewis, E. Perez et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in Advances in Neural Information Processing Systems (NeurIPS), 2020.   [2]     S. Robertson and H. Zaragoza, “The Probabilistic Relevance Framework: BM25 and Beyond,” Foundations and Trends in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009.   [3]     N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” in Proceedings of EMNLP, 2019.   [4]     K. Gomez, F. Silberzahn et al., “Blended RAG: Hybrid Sparse–Dense Retrieval for Retrieval-Augmented Generation,” arXiv:2404.07220, 2024.   [5]     S. Gupta, R. Ranjan et al., “A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Fu-ture Directions,” arXiv:2410.12837, 2024. [6]     V. Karpukhin, B. Og˘uz et al., “Dense Passage Retrieval for Open-Domain Question Answering,” in Proceedings of EMNLP, 2020.   [7]     P. Zhao, H. Zhang et al., “Retrieval-Augmented Generation: A Compre-hensive Survey,” arXiv:2312.10997v1, 2023.   [8]     H. Peng et al., “Retrieval-Augmented Generation for AI-Generated Content: A Survey,” arXiv:2402.19473, 2024.   [9]     J. Maghakian et al., “Embedding-Free RAG,” in Findings of EMNLP, 2025.   [10]    C. Xiong et al., “Accelerating Inference of Retrieval-Augmented Gen-eration via Sparse Context Selection,” arXiv:2405.16178, 2022.   [11]    K. MoR, “Better Handling Diverse Queries with a Mixture of Retriev-ers,” arXiv:2506.15862, 2025.   [12]    N. Thakur, N. Reimers, A. Ru¨ckle´, A. Srivastava, and I. Gurevych, “BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Infor-mation Retrieval Models,” in Proc. NeurIPS Datasets and Benchmarks Track, 2021.   [13]    T. Formal, B. Piwowarski, and S. Clinchant, “SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking,” in Proc. 44th Interna-tional ACM SIGIR, pp. 2288–2292, 2021.   [14]    T. Formal, C. Lassance, B. Piwowarski, and S. Clinchant, “SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval,” arXiv:2109.10086, 2021.   [15]    Y. Gao, Y. Xiong et al., “Retrieval-Augmented Generation for Large Language Models: A Survey,” arXiv:2312.10997, 2023.   [16]    S. Es, J. James, L. Espinosa-Anke, and S. Schockaert, “Ragas: Auto-mated Evaluation of Retrieval Augmented Generation,” in Proc. EACL (Demo Track), pp. 150–158, 2024.