Trust-Aware Agentic RAG: A Guardrailed, Self-Evaluating Multi-Agent Architecture for Enterprise Knowledge Assistants

We presented TA-ARAG, a reference architecture and evaluation harness that treats trust as a first-class, measurable property in enterprise agentic RAG. Three numerical headlines summarise the contribution. First, on quality: faithfulness 0.894 ± 0.002, well above the 0.6–0.7 vanilla RAG floor, and the v1.1 RRF [8] + cross-encoder [7] score-fusion lifts context precision by +12.0 pp without regression on the other three Ragas [6] metrics. Second, on trust: a same-session controlled ablation shows the measurable advantage over naive RAG [9] is almost entirely the Presidio [2] PII guardrail — +80 pp PII, +40 of +42 aggregate probes, with the upstream content filter contribution proving non-reproducible across sessions. Third, on engineering: three defects found only by the eval suite were worth +5–16 pp per metric, validating eval-driven development as an architectural feedback mechanism, not a final QA gate.

[1] LangChain Inc., “LangGraph — stateful multi-actor applications with LLMs,” https://github.com/langchain-ai/langgraph, 2026. [2] Microsoft, “Presidio — PII detection and anonymisation toolkit,” https://microsoft.github.io/presidio/, 2026. [3] OpenTelemetry Authors, “OpenTelemetry: an observability framework for cloud-native software,” https://opentelemetry.io, 2026. [4] Z. Yang et al., “HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering,” Proc. EMNLP, 2018, pp. 2369–2380. [5] P. Islam et al., “FinanceBench: A New Benchmark for Financial Question Answering,” arXiv:2311.11944, 2023. [6] S. Es et al., “Ragas: Automated Evaluation of Retrieval-Augmented Generation,” Proc. EACL (Demo Track), 2024. [7] J. Chen et al., “BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation,” arXiv:2402.03216, 2024. [8] G. V. Cormack et al., “Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods,” Proc. SIGIR, 2009, pp. 758–759. [9] P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” Proc. NeurIPS, 2020. [10] S. Robertson and H. Zaragoza, “The Probabilistic Relevance Framework: BM25 and Beyond,” Foundations and Trends in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009.