Graphzero Agent: Monte Carlo Graph Search as Reasoning-As-A-Service for General Agentic Systems

Current agentic AI systems, predominantly based on the Reason-Act (ReAct) paradigm, execute single-trajectory reasoning with limited backtracking, no principled exploration, and no reuse of repeated subproblems. Empirical analyses reveal brittleness to prompt perturbations and catastrophic performance degradation on tasks exceeding 15–120 steps. We propose GraphZero Agent, a reasoning architecture that adapts AlphaZero’s policy/value-guided Monte Carlo Tree Search (MCTS) to Monte Carlo Graph Search (MCGS) over general agentic belief-states. GraphZero Agent generalizes the search space from trees to Directed Acyclic Graphs, enabling transposition merging of semantically equivalent reasoning states via dense embedding similarity and Approximate Nearest Neighbor search. Intermediate states are scored by Process Reward Models (PRMs) trained without human annotation through a self-rewarding rollout pipeline that produces Process Preference Models (PPMs) from pairwise trajectory comparisons. Beyond single-session inference, a dual-loop Empirical-MCTS architecture provides continuous non-parametric learning: a local loop refines the LLM’s generation policy within a search session via contrastive meta-prompting (PE-EMP), while a global loop distills winning trajectories into reusable heuristics across sessions. The system is deployed as Reasoning-as-a-Service (RaaS) via Model Context Protocol (MCP) or OpenAPI, enabling any agentic orchestration framework to invoke deep deliberative search with independent scaling and LLM hot-swapping.