<div align="center">
<!-- title -->
# Awesome Web Agents
<p align="center">
<a href="https://awesome.re" target="_blank">
<img src="https://awesome.re/badge.svg">
</a>
<a href="https://twitter.com/steeldotdev" target="_blank">
<img src="https://img.shields.io/twitter/follow/steeldotdev.svg?logo=twitter">
</a>
<a href="https://discord.gg/steel-dev" target="_blank">
<img src="https://img.shields.io/static/v1?label=&message=Join%20the%20discord&color=mediumslateblue">
</a>
<!-- <a href="https://github.com/steel-dev/awesome-web-agents/actions/workflows/lint.yaml" target="_blank">
<img src="https://github.com/steel-dev/awesome-web-agents/actions/workflows/lint.yaml/badge.svg">
</a>-->
</p>
<!-- subtitle -->
A curated list of tools, frameworks, and resources for building AI agents that can browse and interact with the web.
</div>
<h2>About Steel</h2>
<!-- image -->
<a href="https://steel.dev" target="_blank" rel="noopener noreferrer">
<img src="steel_hero.png" />
</a>
<!--lint disable double-link-->
Steel is an [open-source](https://github.com/steel-dev/steel-browser) browser API built specifically for AI agents. We make it easy to build AI applications that can effectively interact with the web.
✨ Get started for free [here](https://app.steel.dev).
<!--lint enable double-link-->
<!-- description -->
<!-- TOC -->
<h2>Contents</h2>
<!--lint disable awesome-list-item-->
- [Awesome Web Agents](#awesome-web-agents)
- [Autonomous Web Agents](#autonomous-web-agents)
- [Computer-use Agents](#computer-use-agents)
- [AI Web Automation Tools](#ai-web-automation-tools)
- [Dev Tools](#dev-tools)
- [AI Web Scrapers/Crawlers](#ai-web-scraperscrawlers)
- [Web Search \& Query Tools](#web-search--query-tools)
- [Benchmarks \& Research](#benchmarks--research)
- [Tutorials \& Guides](#tutorials--guides)
- [Archive](#archive)
- [Interested in implementing Steel?](#interested-in-implementing-steel)
- [Join the Community](#join-the-community)
- [Contributing](#contributing)
- [Contributors](#contributors)
<!--lint enable awesome-list-item-->
<!-- CONTENT -->
<!--
## Featured (new releases)
- [Apple](https://apple.com) - Apple as a placeholder.
- [Opera Agentic Feature](https://techcrunch.com/2025/03/03/opera-announces-a-new-agentic-feature-for-its-browser/) - Opera announces a new agentic feature for its browser, showcasing innovative web agent integration.
-->
## Autonomous Web Agents
AI agents that autonomously navigate and interact with the web through a user-friendly interface. (a.k.a Browser Agents)
- [OpenAI Operator](https://openai.com/index/introducing-operator/) - OpenAI's AI agents that can browse the web for you.
- [Browser-Use](https://www.browser-use.com) - SOTA agent and framework that makes the web LLM-friendly. 
- [Skyvern-AI](https://www.skyvern.com/) - Framework to automate browser-based workflows. 
- [Google Project Mariner](https://deepmind.google/technologies/project-mariner/) - A research prototype exploring the future of human-agent interaction, starting with your browser.
- [Sentience API](https://www.sentienceapi.com) - A tool for building more deterministic and explainable web agents using semantic geometry on web content.
- [Runner H](https://www.hcompany.ai/) - State-of-the-art AI agent that helps automate complex, cumbersome, multi-step tasks without repetitive manual input.
- [WebVoyager (Agent)](https://github.com/MinorJerry/WebVoyager) - Vision-enabled web agent. 
- [AgentGPT](https://github.com/reworkd/AgentGPT) - Deploy autonomous AI agents in your browser. 
- [Agent-E](https://github.com/EmergenceAI/Agent-E) - Agent & framework with HTML DOM distillation. 
- [Manus](https://manus.im/) - A general AI agent that can execute long running tasks across tools like browsers, terminals, and text editors.
- [doBrowser](https://www.dobrowser.io) - An AI-powered Chrome extension that understands natural language and takes actions in your browser on your behalf.
- [WebSurfer (Autogen)](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.web_surfer.html#autogen_ext.agents.web_surfer.MultimodalWebSurfer) - MultimodalWebSurfer is a multimodal agent that can search the web and visit web pages. 
- [Magentic-One](https://www.microsoft.com/en-us/research/articles/magentic-one-a-generalist-multi-agent-system-for-solving-complex-tasks/) - A generalist multi-agent system for solving complex tasks including surfing the web via Autogen's MultimodalWebSurfer.
- [Harpa.ai](https://harpa.ai/) - An AI-powered Chrome extension & browser agent that understands natural language and takes actions on your behalf.
- [Yutori](https://yutori.com/) - A multi-agent system that executes browser-based tasks in parallel given a natural language prompt.
- [rtrvr.ai](https://www.rtrvr.ai/) - AI web agent Chrome extension that autonomously does tasks, scrapes to Sheets, and calls APIs with prompts in your own browser.
- [Nanobrowser](https://nanobrowser.ai) - An open-source & local-first AI web agent Chrome extension with flexible LLM options and multi-agent system. 
- [Browserable](https://browserable.ai) - An open-source & self-hostable browser automation library for AI agents. 
- [Tongyi WebAgent](https://github.com/Alibaba-NLP/WebAgent) - WebAgent for information seeking built by Tongyi Lab, Alibaba Group. 
- [Openwork](https://github.com/accomplish-ai/openwork) - An MIT-licensed, open alternative to Anthropic's Cowork built with Opencode and dev-browser. Supports multiple LLM providers for launching computer-use agents to automate browser workflows. 
- [Dassi](https://www.dassi.ai/) - An AI coworking agent in your browser that automates tasks, navigates pages, and works with files and 2000+ apps from a side panel.
- [Caesar](https://github.com/jasonzliang/caesar-agent) - Autonomous research agent that traverses the web to build a knowledge graph, then synthesizes an answer through adversarial redrafting. 
- [SLICC](https://www.sliccy.com) - Browser-based agent runtime with a POSIX shell, virtual filesystem, and Git that automates web apps through the user's existing signed-in browser sessions and runs sandboxed sub-agents in parallel. 
- [Browsentic](https://browsentic.com) - Open-source browser extension that runs Claude Code, Codex or Antigravity from a side panel in the user's logged-in browser, and also works as an MCP server. 
- [CUGA](https://github.com/cuga-project/cuga-agent) - IBM Research's open-source agent that runs tasks in the browser (Playwright or a Chrome extension), over APIs, or both in one task. 
- [qz](https://github.com/alibaba/qz-browser-agent) - Open-source browser agent from Alibaba that runs as a Chrome extension in the user's signed-in browser, with a CLI and MCP server so coding agents such as Claude Code and Codex can hand it web tasks. 
### Computer-use Agents
- [Anthropic Computer Use](https://www.anthropic.com/news/3-5-models-and-computer-use) - Computer use agent that can control your browser.
- [Self-Operating Computer Framework](https://github.com/OthersideAI/self-operating-computer) - A framework to enable multimodal models to operate a computer. 
- [Highlight](https://highlightai.com/) - Desktop activity layer that helps models understand your workflow and complete tasks faster.
- [OpenInterpreter](https://github.com/openinterpreter/open-interpreter) - An open-source CLI based agent that can write & execute code as well as control your browser. 
- [UI-TARS](https://github.com/bytedance/UI-TARS?tab=readme-ov-file) - A GUI agent model designed to interact seamlessly with GUIs using human-like perception, reasoning, and action capabilities. 
## AI Web Automation Tools
Tools, frameworks and libraries that translate natural language instructions into web interactions.
- [Asteroid.ai](https://asteroid.ai/) - Hosted browser agents for SMEs to automate complex workflows. 
- [PulsarRPA](https://github.com/platonai/pulsarRPA) - AI-powered browser automation for data extraction. 
- [VimGPT](https://github.com/ishan0102/vimGPT) - Experimental project using GPT-4 Vision to browse the web via the Vimium extension. 
- [Cekura.io](https://www.cekura.io/) - An AI browser agent that helps companies maintain up-to-date documentation.
- [Dex by Dexterity](https://getdexterity.com/) - An AI coworker embedding into and controlling your browser.
- [Autobrowser](https://chromewebstore.google.com/detail/autobrowser/lebgieachadpombahcacicmebckokcjl) - A free, experimental Chrome extension that leverages Claude Computer Use to automate tasks in your browser.
- [Bytebot](https://bytebot.ai) - AI-powered scraping automations that evolve with your target sites.
- [Runcopycat](https://www.runcopycat.com/) - A no-code browser automation platform that turns screen recordings into reusable automated workflows.
- [Bardeen.ai](https://bardeen.ai) - A Chrome extension that enables AI-powered browser automations, allowing users to automate tasks and workflows directly within the browser.
- [Starizon.ai](https://starizon.ai/) - Browser assistant for web task automation.
- [BrowserGPT](https://browsegpt.ai/) - Browser extension for page summaries and Q&A.
- [Browse.ai](https://www.browse.ai/) - Chrome extension webscraping that can leverage AI for structured data extraction.
- [Deta.surf](https://deta.surf/) - An integrated platform that combines a browser, file manager, and AI assistant with browser-level context.
- [Comet by Perplexity](https://www.perplexity.ai/comet) - An AI-powered browser by Perplexity. Not much more details out yet.
- [Dia Browser](https://www.diabrowser.com/) - AI-first web browser envisioned by The Browser Company (Arc).
- [Reworkd](https://reworkd.ai) - No-code web data extraction solution using agentic AI.
- [Onpiste](https://onpiste.work) - Chrome extension that uses AI to control and read webpages, including auto summaries, web automation, scraping, and MCP support.
- [Komos](https://www.komos.ai/) - AI browser workflow automation platform that turns recorded web tasks into reusable runs with API triggers, schedules, credentials, logs, and human review.
- [agent-qa](https://github.com/vostride/agent-qa) - Self-improving QA harness for natural-language web test runs with execution memory and self-healing actions. 
### Dev Tools
- [Steel.dev](https://steel.dev) - Open-source headless browser API built specifically for AI agents and apps. 
- [Atlas](https://github.com/steel-dev/atlas) - Open-source deep research harness for building cited web research agents with ledger-based coverage audits, pluggable search providers, and Steel-backed browser fetches. 
- [Omniparser](https://microsoft.github.io/OmniParser/) - Tool for parsing GUIs for vision based agents. 
- [LaVague](https://www.lavague.ai/) - Framework for natural language web automation. 
- [LangChain Playwright Toolkit](https://python.langchain.com/docs/integrations/tools/playwright/#use-within-an-agent) - Toolkit integration with AI agents.
- [Browserbase](https://browserbase.com) - A headless browser API for AI workflows.
- [Stagehand](https://www.stagehand.dev/) - AI web browsing framework. 
- [Tarsier](https://github.com/reworkd/tarsier) - Vision utilities library for web interaction agents. 
- [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT) - Experimental agent for task completion and web browsing. 
- [TinyFish](https://www.tinyfish.ai) - Remote web agents that execute tasks on any website and return structured JSON via a single API call. 
- [Bytebot](https://github.com/bytebot-ai/bytebot) - Containerized computer use agent framework with a virtual desktop environment. 
- [Lumen](https://github.com/omxyz/lumen) - Vision-first browser agent with self-healing deterministic replay. Screenshot → model → action loop over CDP, multi-provider (Anthropic, Google, OpenAI), action caching for zero-token reruns. 
- [Webagent-cloud](https://webagent.cloud) - Open-source API for browser agents to automate repetitive workflows. Works with multiple browsers/LLM providers, and minimizes costs with a self-healing action cache. 
- [BrowserTrace](https://github.com/aaronlab/browsertrace) - Local-first trace viewer for debugging Playwright, Browser Use, Stagehand, and other web-agent runs with redacted shareable exports. 
- [Notte](https://notte.cc) - Browser infrastructure for AI agents with managed sessions, an agent runtime, and credential vault and persona authentication primitives. 
- [invisible-playwright](https://github.com/feder-cr/invisible_playwright) - Playwright wrapper for a stealth-patched Firefox 150 build. Drop-in replacement returning native Playwright Browser objects; spoofing happens in C++ source with no JS-level overrides. 
- [Webwright](https://github.com/microsoft/Webwright) - Browser agent framework from Microsoft Research where the agent writes and runs Playwright scripts in a terminal workspace; supports OpenAI, Anthropic, and OpenRouter backends. 
- [BrowserAct](https://github.com/browser-act/skills) - Browser automation CLI and skills for AI agents to operate real browsers, manage sessions, support human handoff, and capture screenshots and evidence. 
- [Agent Browser Shield](https://github.com/pixiebrix/agent-browser-shield) - Browser extension that sits between an AI agent and the page, stripping prompt injection, masking PII/credentials, and removing dark patterns before content reaches the model. 
- [HUD](https://github.com/hud-evals/hud-python) - Open-source SDK for building browser and computer-use RL environments to evaluate and train web agents, with task-based verifiable rewards runnable as evals or RL training across any model. 
- [Webfuse](https://www.webfuse.com) - Configurable web proxy and browser-as-a-service for deploying and operating AI agents in a sandbox layer on top of any third-party website, using client-side extensions and without source-code access.
- [Webcmd](https://github.com/agentrhq/webcmd) - Self-learning browser infrastructure for AI agents that records how a site is navigated, then compiles that context into deterministic CLI adapters and reusable sitemap memory for later runs. 
- [Hermes Connector](https://github.com/CorsenAI/hermes-connector) - Unofficial open-source Chrome extension and local companion that let Hermes Agent control only explicitly attached tabs in a user's Chrome session. 
- [Clawbrowser](https://clawbrowser.ai) - Open-source managed Chromium runtime for AI agents that keeps per-profile fingerprints, proxy routing, cookies, and storage isolated and exposes each profile over a standard CDP endpoint for Playwright, Puppeteer, or other CDP clients. 
- [FSB (Full Self Browsing)](https://github.com/fullselfbrowsing/FSB) - Open-source Chrome extension and MCP server that lets MCP clients such as Claude Code, Codex, and Cursor drive the user's signed-in Chromium browser by reading the live DOM instead of screenshots. 
- [Browser Use Cloud](https://docs.browser-use.com/cloud/quickstart) - Hosted stealth Chromium browsers for AI agents with residential proxies, CAPTCHA solving, persistent profiles and live view, driven over CDP by Playwright, Puppeteer, Stagehand or the Browser-Use agent.
## AI Web Scrapers/Crawlers
Web crawlers & scrapers that leverage AI to navigate websites and extract content.
- [Firecrawl](https://www.firecrawl.dev/) - API for AI agents to search, scrape, crawl, and interact with the web, returning Markdown or structured JSON. Has an official MCP server and agent skills. 
- [Crawl4AI](https://crawl4ai.com) - Open-source LLM Friendly Web Crawler & Scraper. 
- [ScrapeGraphAI](https://scrapegraphai.com/) - Python scraper based on AI. 
- [WebAgent (OpenAgents)](https://github.com/xlang-ai/OpenAgents) - The web-browsing agent module of the OpenAgents platform (HKU). Enables autonomous navigation of websites via natural language, as part of a larger multi-modal agent framework. 
- [Expand.ai](https://www.expand.ai/) - Turns any website into a type-safe API you can rely on.
- [LLM Scraper](https://github.com/mishushakov/llm-scraper) - Uses LLMs for intelligent scraping and content understanding. 
- [Plasmate](https://github.com/plasmate-labs/plasmate) - Open-source headless browser engine for AI agents. Compiles HTML to Semantic Object Model (SOM) with 17.5x token compression. 13 MCP tools. First browser tool on the MCP Registry. Rust, Apache-2.0. 
- [SpiderCreator](https://github.com/carlosplanchon/spidercreator) - Create complex Playwright spiders with natural language prompts. 
- [DataLens](https://datalens.uk) - AI web data agent that plans browser scraping tasks, extracts structured web data, and exposes MCP tools for dataset workflows.
- [Context.dev](https://www.context.dev/) - Web APIs and MCP tools for search, scraping, crawling, schema-based extraction, document parsing, monitoring, and batch jobs.
- [ScrapingBee](https://www.scrapingbee.com/) - Web scraping API with JavaScript rendering, proxy rotation and plain-English AI extraction, plus a CLI, Agent Skills and a remote MCP server for AI agents.
## Web Search & Query Tools
Utilities that help agents search the web or query web data via natural language.
- [AgentQL](https://www.agentql.com/) - A query language and toolkit that makes the web AI-ready. 
- [SerpAPI](https://serpapi.com/) - Search API that provides Google Search results for your agents.
- [Serper.dev](https://serper.dev/) - Performant and cost effective search API that provides Google Search results for your agents.
- [Jina.ai](https://jina.ai/) - Neural search platform for web data.
- [Exa.ai](https://exa.ai) - The fastest and most accurate web search API for AI agents.
- [Not Human Search](https://nothumansearch.ai) - Search engine that indexes 1,750+ agent-first tools ranked by agentic readiness. Available as an MCP server with tools for searching, scoring, and monitoring agent infrastructure.
- [Superhighway](https://superhighway.walls.sh) - Web search API for AI agents with five tools (search, news, images, scrape, research); agents pay per call in USDC via the x402 protocol, or use a free API key.
- [Zoom Search](https://github.com/goofrey/zoom-search) - Open-source web search and evidence tool for AI agents with query rewriting, source-domain zoom-in, structured sourced outputs, and MCP and LangGraph integrations. 
- [SocialCrawl](https://www.socialcrawl.dev) - Unified social and commerce data API covering 50+ platforms as one GET and one JSON schema.
- [Keenable](https://keenable.ai) - Web search API and hosted MCP server with its own index, returning ranked results and page content as markdown for AI agents. 
## Benchmarks & Research
Datasets, benchmarks, and notable research efforts for evaluating and advancing web-capable AI agents.
- [Web Agent Leaderboard](https://leaderboard.steel.dev) - Leaderboard compiling AI agent products and their performance on widely used WebVoyager benchmarks. 
- [Web Games by Convergence](https://webgames.convergence.ai/) - A collection of challenges designed for testing general-purpose web-browsing AI agents. 
- [Bananalyzer](https://github.com/reworkd/bananalyzer) - An open-source evaluation framework for web-based AI agents. 
- [Mind2Web](https://osu-nlp-group.github.io/Mind2Web) - A large-scale dataset for generalist web agents. 
- [World of Bits: An Open-Domain Platform for Web-Based Agents](https://proceedings.mlr.press/v70/shi17a/shi17a.pdf) - OpenAI's research paper that introduces World of Bits: a platform where agents complete tasks on the internet by performing low-level keyboard and mouse actions.
- [MiniWoB++](https://miniwob.farama.org) - A classic suite of 104 mini web browser tasks in a synthetic environment. It is an extension of the OpenAI MiniWoB benchmark. 
- [WebTaskBench](https://github.com/plasmate-labs/plasmate-benchmarks) - 51-URL benchmark comparing HTML vs Markdown vs SOM representations for AI agents. Measures token efficiency, latency, and accuracy across GPT-4o and Claude Sonnet 4. 
- [WebArena](https://webarena.dev) - A realistic, self-hostable web environment for autonomous agents. Includes official leaderboard tracking agent performance. 
- [WebCanvas](https://github.com/iMeanAI/WebCanvas) - An online evaluation framework for dynamic web environments. Tests agents on live websites. 
- [WebGPT](https://openai.com/research/webgpt) - OpenAI's browser-assisted question-answering research project.
- [WebShop](https://webshop-pnlp.github.io) - A simulated e-commerce shopping environment with 1.18M real Amazon products. 
<!--lint ignore double-link-->
- [WebVoyager (Benchmark)](https://arxiv.org/abs/2401.13919) - Vision-enabled benchmark for real-world website interaction with large multimodal models. 
- [WorkArena](https://github.com/ServiceNow/WorkArena) - A suite of 33 browser-based tasks for enterprise "knowledge worker" scenarios. 
- [BrowserGym by ServiceNow](https://github.com/ServiceNow/BrowserGym) - A gym environment for web task automation. 
- [TimeWarp](https://timewarp-web.github.io) - A benchmark on historical versions of web UI.
- [ClawBench](https://github.com/reacher-z/ClawBench) - 283 everyday tasks (V1 153 + V2 130) on 163 live production websites across 15 categories. Two-stage scoring (final HTTP-request interception + LLM judge) blocks only the write request so real sites stay clean. Public leaderboard with 5-layer execution traces (recording, action log, request log, agent messages, interception). 
- [Next-Gen CAPTCHAs](https://github.com/MetaAgentX/NextGen-CAPTCHAs) - Scalable CAPTCHA defense framework and benchmark (ICML 2026) for evaluating multimodal web GUI agents. 
- [WebsiteBench](https://github.com/TIGER-AI-Lab/WebsiteBench) - Benchmark from TIGER-AI-Lab that tests whether agents can rebuild full-stack websites by exploring them in a browser. 
## Tutorials & Guides
Resources for learning how to build, deploy, or utilize AI web agents.
- [LangGraph WebVoyager Tutorial](https://langchain-ai.github.io/langgraph/tutorials/web-navigation/web_voyager/) - Tutorial demonstrating how to build a web navigation agent using LangGraph Agents, Vision Models, and Web Voyager.
- [Build an AI Browser Agent](https://dzone.com/articles/build-ai-browser-agent-llms-playwright-browser-use) - Step-by-step guide to create an AI that browses the web using Playwright and the Browser-Use library.
- [Install & Run Browser-Use Locally](https://aleksandarhaber.com/install-and-run-browser-use-ai-agents-locally-using-ollama/) - Instructions on installing the open-source Browser-Use agent with a local LLM.
- [Build a Browser Agent with DeepSeek](https://dev.to/nodeshiftcloud/build-a-browser-use-agent-with-deepseek-a-step-by-step-guide-2n59) - Walks through deploying a Browser-Use web UI agent powered by the DeepSeek model on a cloud VM.
## Archive
Historical or inactive projects are tracked in [ARCHIVE.md](ARCHIVE.md).
<!-- END CONTENT -->
<!--lint disable double-link-->
<!--lint ignore no-heading-punctuation-->
## Interested in implementing Steel?
Feel free to reach out at [team@steel.dev](mailto:team@steel.dev?subject=Hello%20from%20github!) or on [Discord](https://discord.gg/steel-dev).
Steel is an [open-source](https://github.com/steel-dev/steel-browser) browser API built specifically for AI agents. Get started for free [here](https://app.steel.dev).
<!--lint enable double-link-->
## Join the Community
<!-- list people worth following on social sites (Twitter, LinkedIn, GitHub, YouTube etc.) -->
<!--lint disable double-link-->
- Follow [@steeldotdev](https://x.com/steeldotdev) on X.
- Join the [Discord community](https://discord.gg/steel-dev).
- Feel free to reach out to us at [team@steel.dev](mailto:team@steel.dev?subject=Hello%20from%20github!)
<!--lint enable double-link-->
## Contributing
[Contributions of any kind welcome, just follow the guidelines](CONTRIBUTING.md)!
### Contributors
[Thanks goes to these contributors](https://github.com/steel-dev/awesome-web-agents/graphs/contributors)!