From Black Box to Audit Trail: Implementing Explainable Agentic AI and Human-in-the-Loop Governance for High-Risk Enterprise Decisions

This study has demonstrated that the transition from rule-based automation to agentic AI in enterprise IT Service Management fundamentally alters the risk profile of operational decision-making. As autonomous agents increasingly execute end-to-end actions across infrastructure, HR, and procurement workflows, traditional governance mechanisms based on static approvals and post-hoc audits become insufficient. The findings presented across architectural analysis, case studies, and quantitative evaluation confirm that unconstrained autonomy, rather than model inaccuracy alone, is the primary source of operational and governance risk. The HITL 2.0 Framework allows for the safe and scalable deployment of Autonomous AI in Enterprise IT Service Management through the integration of Adaptive Governance into Workflows. In order to provide the appropriate level of oversight, while minimizing the reduction in autonomous workflow performance risk-aware guardrails, explainable decisions, and centralized monitoring are used to apply selective human oversight at high-risk decision-making points. Patterns of sustained improvement in human override rates and trust have been empirically demonstrated, and therefore, integrated governance architectures do not constrain AI-driven performance; they enhance it.

1.                J. S. Turgunov, M. O. Rakhimova, B. K. Saidov, and D. S. Kholmatova, “Human-in-the-loop systems for ethical AI,” Faculty of Mathematical Modeling, Namangan State University, Namangan, Uzbekistan, Dec. 8, 2020. [Online]. Available: https://www.researchgate.net/profile/Dharmasena-Sd/publication/393802734_HUMAN-IN-THE-LOOP_SYSTEMS_FOR_ETHICAL_AI/links/687bb6837d202419e84d6540/HUMAN-IN-THE-LOOP-SYSTEMS-FOR-ETHICAL-AI.pdf 2.                J. Frenette, “Ensuring human oversight in high-performance AI systems: A framework for control and accountability,” World Journal of Advanced Research and Reviews, vol. 20, no. 2, pp. 1507–1516, 2023, doi: 10.30574/wjarr.2023.20.2.2194. [Online]. Available: https://d1wqtxts1xzle7.cloudfront.net/124100264/Ensuring_Human_Oversight_in_High_Performance_AI_Systems_A_Framework_for_Control_and_Accountability-libre.pdf 3.                P. Enyiorji, “Human-centered responsible AI product development lifecycles: Merging participatory design, stakeholder alignment, and risk modeling for equitable digital financial service delivery,” International Journal of Science and Engineering Applications, vol. 11, no. 12, pp. 452–468, 2022, doi: 10.7753/IJSEA1112.1067. [Online]. Available: https://ijsea.com/archive/volume11/issue12/IJSEA11121069.pdf 4.                A.-L. K. Lindström, “Scalable cloud automation for SAP ecosystems: Ethical AI and predictive risk management using machine learning,” International Journal of Research in Engineering, Technology and Management, Nov. 10, 2023, doi: 10.15662/IJRPETM.2023.0606007. [Online]. Available: https://ijrpetm.com/index.php/IJRPETM/article/view/184 5.                A. Chan et al., “Harms from increasingly agentic algorithmic systems,” in Proc. ACM Conf. Fairness, Accountability, and Transparency (FAccT ’23), 2023, pp. 651–666, doi: 10.1145/3593013.3594033. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/3593013.3594033 6.                M. Saarela, V. Heilala, P. Jääskelä, A. Rantakaulio, and T. Kärkkäinen, “Explainable student agency analytics,” IEEE Access, vol. 9, pp. 137444–137459, Sep. 29, 2021, doi: 10.1109/ACCESS.2021.3116664. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9552837 7.                D. Ogbu, “Agentic AI in computer vision domain – recent advances and prospects,” International Journal of Research Publication and Reviews, vol. 4, Dec. 2023, pp. 5102–5120, doi: 10.55248/gengpi.5.1124.3309. [Online]. Available: https://www.researchgate.net/publication/386292786_Agentic_AI_in_Computer_Vision_Domain_-Recent_Advances_and_Prospects 8.                G. Papagni, J. de Pagter, S. Zafari, M. Filzmoser, and S. T. Koeszegi, “Artificial agents’ explainability to support trust: Considerations on timing and context,” AI & Society, vol. 38, pp. 947–960, 2023, published online Jun. 27, 2022, doi: 10.1007/s00146-022-01462-7. [Online]. Available: https://link.springer.com/article/10.1007/s00146-022-01462-7 9.                A. Dodda, “AI governance and security in fintech: Ensuring trust in generative and agentic AI systems,” Asian Journal of Applied Economics and Development, vol. 1, no. 1, pp. 1––, Dec. 2, 2023. [Online]. Available: https://aajed.com/index.php/aajed/article/view/6 10.             B. Abedin, C. Meske, I. Junglas, F. Rabhi, and H. R. Motahari-Nezhad, “Designing and managing human-AI interactions,” Information Systems Frontiers, vol. 24, pp. 691–697, Jul. 18, 2022, doi: 10.1007/s10796-022-10313-1. [Online]. Available: https://link.springer.com/article/10.1007/s10796-022-10313-1