A Multi-Agent Scientific Audit Platform Powered by Vision-Language Models for Detecting P-Hacking in Clinical Research

With the expansion of clinical research, there is a need for reviewers for users to review statistical evidence to see if it backs up the claims in the text, methods and figures. Manual auditing is vital, but software that unpacks the content of a document, infers content from the process, explains the results and checks numerical relationships can bolster it. We have proposed a multi-agent Scientific Audit Platform in which PDF parsing, reasoning with text using an LLM and reasoning with figures using a VLM collaborates. Findings are shared through a Coordinator for tasks to be divided among specialist agents and there is Cross-Verification and Skeptic Review to ensure findings are backed by the original paper. The final report will show all observations and their source evidence. All 47 controlled reporting errors were identified on the component results and the statistical checker was in close agreement with statcheck. The result for PDF text extraction was 0.991 recall, while personally, we obtained 0.944 for a table; recall of numbered figures was the same as for 72 of 74. The outcome-switching screening resulted in F1=0.30 which was not superior to the accuracy because the never-changing accuracy rate baseline. They denote their reported sub-tasks and shouldn't be reported as LLM/VLM accuracy. The LLM and the VLM play a key role in the system, with the LLM responsible for extracting the text and context, and the VLM responsible for extracting the visual claims from the charts. Source text, layout and reproducible calculations are provided by the parser and deterministic statistical routines. We're not trying to state that a language or vision model does everything, this is not a hybrid. Future research should focus on a model-specific evaluation of the LLM and VLM that is replicable, by setting the version of the model and prompt for the experiment, providing examples of the charts to humans and measuring the accuracy of grounding the source, the extraction of the values from the charts and the interpretation of the trends on the top chart. Furthermore, it should facilitate outcome match between registry and texts, assess consistency between figures and texts, conduct a multi-agent ablation study, quantify the agreement between reviewers and publish a complete dataset manifest.

J. P. Simmons, L. D. Nelson, and U. Simonsohn, “False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant,” Psychological Science, vol. 22, no. 11, pp. 1359–1366, 2011. [2]               B. C. Kalpélbé et al., “Vision language models in medicine,” arXiv preprint arXiv:2503.01863, 2025. [3]               T. Berkane et al., “Self-ensembling vision-language models for chart data extraction,” arXiv preprint arXiv:2605.27298, 2026. [4]               V. Winland, “What is CrewAI?,” IBM Developer, 2024. [5]               G. Prasad, “Comparing AI agent frameworks: CrewAI, LangGraph and BeeAI,” IBM Developer, 2024. [6]               National Institutes of Health, “PubMed Central Open Access Subset,” 2025. [7]               OpenAI, “Vision API Documentation,” 2024. [8]               PyMuPDF Documentation, “Table extraction from scientific PDFs,” 2024. [9]               M. G. Head et al., “Approaches to detect p-hacking,” PLOS ONE, 2023. [10]             Google DeepMind, “Gemini Technical Report,” 2024.