ScrapeFlow: An AI-Augmented Visual Web Scraping and Data Workflow Platform

Building a complete web data pipeline today means choosing between two incompatible options: API orchestrators like n8n and Zapier offer visual DAG editors but no native browser automation, while scraping frameworks such as Apify provide headless browser control at the cost of requiring devel-oper expertise [4] [5]. Prior academic work on web automation scripting [1], natural-language parameterization [2], and hierar-chical scraping visualization [3] advanced individual concerns yet remained dependent on brittle DOM-structural selectors and did not converge visual workflow design, browser execution, and AI-driven extraction into a single system. To close this gap, we built ScrapeFlow—an open-source platform on Next.js 16 that brings together a typed-handle DAG editor, a phase-based Puppeteer execution engine with merge-point-aware branching and three loop variants, and a provider-agnostic AI layer covering seven LLM backends including local Ollama. A capability evaluation against eight commercial platforms across ten feature dimensions shows that ScrapeFlow is the only system satisfying all dimensions simultaneously, and a case study demonstrates an end-to-end pipeline from AI-powered data extraction through sentiment analysis to conditional webhook delivery. Taken together, these results suggest that semantic web orchestration is a practical design approach—one that gives non-technical users access to enterprise-grade control-flow without writing a single line of code.