Retrieval-Augmented Generation for Semantic Querying of PDF Documents Using Open-Source Language Models

Due to their dependence on static, pre-trained knowledge, large language models (LLMs) frequently experience hallucination, producing outputs that appear confident but are factually incorrect. This restriction is lessened by retrieval-augmented generation (RAG), which grounds responses in outside context. A document-based RAG system that uses the open-source reasoning model DeepSeek and is locally deployed using Ollama is presented in this paper. A Chroma vector database is used to store the PDF documents that users upload after they have been parsed, segmented, and embedded using LangChain. When a query is received, the system uses DeepSeek to produce grounded responses and retrieve semantically relevant content. The system may optionally call the web-based search API Tavily if no appropriate context is found. Streamlit, which provides an easy-to-use interface for document upload, parameter tuning, and chat interaction, is used to deploy the application. The system enhances reliability and decreases hallucinations in knowledge-intensive tasks by integrating contextual retrieval, local inference, and optional fallback.