Enhancing RAG Systems: A Survey of Optimization Strategies for Performance and Scalability

Enhancing RAG Systems: A Survey of Optimization Strategies for Performance and Scalability

Nishanth S1* and Swetha S1

1*Department of ISE, RV College of Engineering, Mysore Road, Bengaluru, 560059, Karnataka, India.

*Corresponding author(s). E-mail(s): nishanths.is20@rvce.edu.in; Contributing authors: shwetha.ise@rvce.edu.in;

Abstract

Retrieval Augmented Generation (RAG) systems offer significant advancements in natural language processing by combining large language models (LLMs) with external knowledge sources to improve factual accuracy and contextual relevance. However, the computational complexity of RAG pipelines presents challenges in terms of efficiency and scalability. This research paper conducts a comprehensive survey of optimization techniques across four key areas: tokenizer performance, encoder performance, vector database search strategies, and LLM agent integra- tion. The study explores approaches to accelerate tokenization using Rust-based implementations and investigates hardware optimizations like CUDA and Ten- sor cores to boost encoder efficiency. Additionally, it delves into algorithms and indexing strategies for efficient vector database searches and examines meth- ods for optimising the interaction between retrieved knowledge and LLM agents. By analysing recent research and evaluating various optimization strategies, this paper aims to provide valuable insights into enhancing the performance and practicality of RAG systems for real-world applications.

Keywords: Retrieval-Augmented Generation, Performance optimization, Tokenization, WordPiece, Rust, FastTokenizer,Encoding, GPU, CUDA, Tensor Cores, Parallel Processing, Vector Databases, Indexing, HNSW, FAISS, DiskANN, LLM Agents, Prompt Engineering, Retriever