Instant Question Answering System for Pdf Documents Using Local Retrieval-Augmented Generation
Mrs. P. B. Khandekar1,
1, Professor, Sir Visvesvaraya Institute of Technology Nashik.
Abstract -
The exponential growth of digital documents, particularly research papers, legal contracts, technical reports and textbooks, has made manual information retrieval increasingly time-consuming and inefficient. Traditional keyword-based search and existing cloud-dependent PDF question-answering tools suffer from high latency, privacy risks, recurring costs and inability to operate offline. This paper proposes PDF-Insight 360, a fully local, open-source Retrieval-Augmented Generation (RAG) framework that enables instant, accurate and privacy-preserving natural-language question answering on arbitrary PDF documents using only consumer-grade hardware.
The proposed system integrates PyMuPDF for high-fidelity text extraction, recursive character-based chunking with overlap, all-MiniLM-L6-v2 sentence transformer for embedding generation, Chroma persistent vector database and a quantized Llama-3-70B-Instruct (8-bit/4-bit GGUF) model executed via Ollama/llama.cpp. A novel two-stage retrieval mechanism combined with persistent embedding storage ensures zero-shot indexing: the first query on a previously unseen multi-hundred-page document completes in under 2.4 seconds, while subsequent queries consistently achieve 180–240 ms response latency. Extensive evaluation on 50 real-world PDFs (average 127 pages) comprising research articles, legal agreements and technical manuals demonstrates 92.3 % semantic accuracy with complete data privacy and zero external API dependency.
PDF-Insight 360 establishes that state-of-the-art document intelligence previously exclusive to proprietary cloud platforms can be realized entirely offline on standard laptops, offering significant implications for researchers, legal professionals, students and organizations handling sensitive documents.
Keywords: Retrieval-Augmented Generation, Local LLM, Offline Document Intelligence, Privacy-Preserving RAG, Chroma DB, Sentence Transformers, Llama-3, Consumer-Hardware AI
