An Large Language Models Based Autonomous Browser Agent

This paper introduced an LLM-based Autonomous Browser Agent intended to automate intricate web-based tasks through natural language directives. The system combines Large Language Models with real browser control using the ReAct (Reasoning and Acting) framework. This lets it intelligently, adaptively, and completely complete tasks in dynamic web environments.   The proposed system shows that it can do a lot of different things with little help from the user, like web browsing, multimedia control, messaging, job and internship applications, and data extraction. Adding support for multiple languages, voice input and output, and profile-based auto-fill makes the app much easier to use and more accessible. The system also does a good job of handling real-world problems like needing to log in and not being able to access content by using pause-and-resume execution and error recovery methods. The proposed solution offers a cohesive platform that integrates reasoning, adaptability, and usability, eliminating the need for programming expertise or established workflows, in contrast to conventional automation tools and current AI-based systems. This makes it great for people who aren't tech-savvy and for use in the real world. The system does add some latency because it uses LLM-based reasoning and relies on external APIs, but it is much more flexible and intelligent than other methods. The project shows how LLM-driven autonomous agents could change the way people use the web. This could lead to more advanced, user-focused automation systems in the future.

[1]              Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, Jirong Wen (2024) A Survey on Large Language Model Based Autonomous Agents. Frontiers of Computer Science. https://link.springer.com/article/10.1007/s11704-024-40231-1 [2]              Yao, S., et al.,“ReAct: Synergizing Reasoning and Acting in Language Models.”arXiv Research Paper. https://arxiv.org/abs/2210.03629 [3]              Tom B. Brown, Benjamin Mann, Nick Ryder et al. (2020) Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems (NeurIPS) 33:1877-1901.https://arxiv.org/abs/2005.14165 [4]              Shunyu Yao, Dian Yu, Jeffrey Zhao et al. (2023) Tree of Thoughts: Deliberate Problem Solving with Large Language Models. arXiv preprint arXiv:2305.10601. https://arxiv.org/abs/2305.10601 [5]              Shunyu Yao, Howard Chen, John Yang, Karthik Narasimhan (2022) WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents. arXiv preprint arXiv:2207.01206. https://arxiv.org/abs/2207.01206 [6]              Microsoft (2022) Playwright: Fast and Reliable End-to-End Testing for Modern Web Apps. Software Documentation. https://playwright.dev/python/docs/intro [7]              Oriol Vinyals, Igor Babuschkin et al. (2022) WebShop: Towards Scalable RealWorld Web Interaction with Grounded Language Agents. arXiv preprint arXiv:2207.01206.  [8]              Daniel Jurafsky, James H. Martin (2009) Speech and Language Processing, 2nd Edition. Pearson.  [9]              Ashish Vaswani, Noam Shazeer et al. (2017) Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS)  [10]           Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova (2019) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACLHLT.  [11]        Lawrence R. Rabiner, Biing-Hwang Juang (1993) Fundamentals of Speech Recognition. Prentice Hall.