Business Analytics through Schema-Aware Query Validation

This paper presented Business Analytics through Schema-Aware Query Validation, a self-reflective, schema-aware natural language querying system for business analytics. By combining adaptive schema-chunk retrieval, iterative SQL refinement via three reflection tokens (ISVALID, ISALIGNED, ISCORRECT), and a business context validation gate, the proposed system enables non-technical users to query enterprise databases in plain English with high reliability. Evaluation on a 150-query business benchmark shows 76.4% execution accuracy and a 98.9% valid SQL rate, with the schema validation layer reducing schema-error queries by 61% over a zero-shot baseline. The hybrid chart selector achieves 89.3% visualization recom-mendation accuracy, with the LLM fallback stage contributing most significantly for ambiguous multi-measure result schemas. The primary limitations of the current system are •  Latency: High latency on extra hard queries (avg. 5.1 s), which arises from multiple LLM calls within the reflection loop. •  Evaluation Context: Evaluation on synthetic rather than real enterprise data, which may not capture the full complexity of production schemas. •  Benchmark Gaps: Absence of direct comparison with published Text-to-SQL systems on the Spider benchmark. Future work will explore three directions. First, caching partial query plans across sessions to reduce latency for repeated or similar queries. Second, extending schema retrieval to databases with 500+ tables by introducing hierarchical retrieval strategies. Third, evaluating the proposed system on the Spider benchmark [10] to enable direct comparison with DIN-SQL [9], ReFoRCE [7], and SHARE [8], and incorporating user feedback signals to fine-tune reflection token thresholds over time.

[1]     A. Alparslan, “The role of accuracy and validation effectiveness in conversational business analytics,” IEEE Access, vol. 13, pp. 29279–29291, 2025. [2]     S. Chu and J. Liu, “Based on BERT-GPT-GNN converged architecture: intelligent generation engine for complex SQL queries in business intelligence,” Discov. Artif. Intell., Springer, 2025. [3]     P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal,A.   Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Adv. Neural Inf. Process. Syst., vol. 33, pp. 9459–9474, 2020. [4]     K. Hu, M. A. Bakker, S. Li, T. Kraska, and C. Hidalgo, “VizML: A machine learning approach to visualization recommendation,” in Proc. CHI Conf. Human Factors Comput. Syst., pp. 1–12, 2019. [5]     A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi, “Self-RAG: Learning to retrieve, generate, and critique through self-reflection,” in Proc. Int. Conf. Learn. Representations (ICLR), 2024. [6]     R. I. Saveliev and M. V. Dendiuk, “Self-reflective retrieval-augmented generation (Self-RAG) in analytical systems,” in Proc. Int. Sci.-Prac. Conf. Forestry Educ. Sci., 2024. [7]     M. Deng, A. Ramachandran, C. Xu, L. Hu, Z. Yao, A. Datta, and H. Zhang, “ReFoRCE: A Text-to-SQL agent with self-refinement, consensus enforcement, and column exploration,” in ICLR Workshop VerifAI, 2025. [8]     G. Qu, J. Li, B. Qin, X. Li, N. Huo, C. Ma, and R. Cheng, “SHARE: An SLM-based hierarchical action correction assistant for Text-to-SQL,” in Proc. Annu. Meeting Assoc. Comput. Linguistics (ACL), 2025. [9]     M. Pourreza and D. Rafiei, “DIN-SQL: Decomposed in-context learning of text-to-SQL with self-correction,” in Adv. Neural Inf. Process. Syst., vol. 37, 2024. [10]    T. Yu, R. Zhang, K. Yang, M. Yasunaga, D. Wang, Z. Li, J. Ma, I. Li,Q. Yao, S. Roman, Z. Zhang, and D. Radev, “Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL,” in Proc. Conf. Empirical Methods Natural Lang. Process. (EMNLP), pp. 3911–3921, 2018.