Runtime Monitoring and Risk-Adaptive Control for LLM Agents

AgentGuard shows how lexical monitoring, composite risk scoring, and adaptive control can be assembled in a small, inspectable runtime prototype. In the archived simulations, the main run records F1=0.844 and the ten-seed mean is 0.791. The equal-weight ablation also shows how strongly the decision depends on the security factor. At the same time, the evidence sets firm limits on the claim: lexical transformations sharply reduce detection, recall on the internal benchmark is modest, labels overlap with SRMP output, and latency is simulated rather than measured. The present contribution is therefore an auditable research baseline, not a deployment-ready security claim. AgentGuard preserves event-level records and exposes each stage of its control logic, making it possible to investigate failure rather than treating the monitor as a black box. The next phase should introduce independently adjudicated labels, public agent benchmarks, semantic and trace-based monitors alongside the lexical rules, measured enforcement overhead, and an explicit analysis of how false interventions affect task completion. That evidence is needed before the prototype can be assessed as an operational runtime defence.

[1] OWASP GenAI Security Project, “LLM01:2025 Prompt Injection,” OWASP Top 10 for LLM Applications, 2024. [2] M. Mazeika et al., “HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal,” in Proc. ICML, 2024. [3] M. Andriushchenko et al., “AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents,” in Proc. ICLR, 2025. [4] Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents,” in Findings of ACL, 2024. [5] H. Zhang et al., “Agent Security Bench: Formalizing and Benchmarking Attacks and Defenses in LLM-Based Agents,” in Proc. ICLR, 2025. [6] C. Autio et al., Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, 2024. [7] OWASP GenAI Security Project, State of Agentic AI Security and Governance, ver. 2.01, 2026. [8] Y. Mou et al., “ToolSafe: Enhancing Tool Invocation Safety of LLM-Based Agents via Proactive Step-Level Guardrail and Feedback,” arXiv:2601.10156, 2026. [9] “Runtime Policy Enforcement for MCP-Based LLM Agents,” Electronics, 2026. [10] A. H. Müller et al., “Classifier Calibration: A Survey on How to Assess and Improve Predicted Class Probabilities,” Machine Learning, vol. 112, pp. 3211–3260, 2023.