agent-browser

user-invocable	false
name	agent-browser
description	Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. USE FOR: - "open a website", "go to this URL", "navigate to", "打开网页", "访问这个网站" - "fill out a form", "click a button", "submit this form", "填写表单", "点击按钮" - "take a screenshot", "截图", "capture this page", "record this session" - "scrape data from a page", "extract information", "抓取数据", "爬取页面" - "test this web app", "automate browser actions", "测试网页", "iOS 模拟器测试" - "login to a site", "automate browser actions", "自动化浏览器", "保存登录状态" - Any task requiring programmatic web interaction REPLACES: Manual browser interactions, Selenium scripts REQUIRES: - agent-browser CLI installed (`brew install agent-browser` or via npm) - Playwright browsers installed (`agent-browser install`) - For iOS: macOS with Xcode, Appium (`npm install -g appium && appium driver install xcuitest`)
allowed-tools	Bash(agent-browser:*)

Pattern: Tool Wrapper (Google ADK)

Browser Automation with agent-browser

When to Use

Use when the user wants to interact with any website programmatically — filling forms, clicking buttons, extracting data, taking screenshots, or testing web apps. Also the go-to tool for iOS simulator testing and video recording of browser sessions.

Don't use when: The user only needs to read text from a page (use web_fetch instead — zero token cost). For AI-autonomous task completion, prefer browser-use run "task".

Prerequisites

1. Install agent-browser: check with agent-browser --version
2. Install Playwright browsers: agent-browser install (first time only)
3. For iOS testing: macOS + Xcode + npm install -g appium && appium driver install xcuitest
4. For proxy/geo features: configure proxy settings in agent-browser config

Core Workflow

Every browser automation follows this pattern:

1. Navigate: agent-browser open <url>
2. Snapshot: agent-browser snapshot -i (get element refs like @e1, @e2)
3. Interact: Use refs to click, fill, select
4. Re-snapshot: After navigation or DOM changes, get fresh refs

agent-browser open https://example.com/form
agent-browser snapshot -i
# Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Submit"

agent-browser fill @e1 "[e-mail]"
agent-browser fill @e2 "password123"
agent-browser click @e3
agent-browser wait --load networkidle
agent-browser snapshot -i  # Check result

Essential Commands

# Navigation
agent-browser open <url>              # Navigate (aliases: goto, navigate)
agent-browser close                   # Close browser

# Snapshot
agent-browser snapshot -i             # Interactive elements with refs (recommended)
agent-browser snapshot -i -C          # Include cursor-interactive elements (divs with onclick, cursor:pointer)
agent-browser snapshot -s "#selector" # Scope to CSS selector

# Interaction (use @refs from snapshot)
agent-browser click @e1               # Click element
agent-browser fill @e2 "text"         # Clear and type text
agent-browser type @e2 "text"         # Type without clearing
agent-browser select @e1 "option"     # Select dropdown option
agent-browser check @e1               # Check checkbox
agent-browser press Enter             # Press key
agent-browser scroll down 500         # Scroll page

# Get information
agent-browser get text @e1            # Get element text
agent-browser get url                 # Get current URL
agent-browser get title               # Get page title

# Wait
agent-browser wait @e1                # Wait for element
agent-browser wait --load networkidle # Wait for network idle
agent-browser wait --url "**/page"    # Wait for URL pattern
agent-browser wait 2000               # Wait milliseconds

# Capture
agent-browser screenshot              # Screenshot to temp dir
agent-browser screenshot --full       # Full page screenshot
agent-browser pdf output.pdf          # Save as PDF

Common Patterns

Form Submission

agent-browser open https://example.com/signup
agent-browser snapshot -i
agent-browser fill @e1 "Jane Doe"
agent-browser fill @e2 "[e-mail]"
agent-browser select @e3 "California"
agent-browser check @e4
agent-browser click @e5
agent-browser wait --load networkidle

Authentication with State Persistence

# Login once and save state
agent-browser open https://app.example.com/login
agent-browser snapshot -i
agent-browser fill @e1 "$USERNAME"
agent-browser fill @e2 "$PASSWORD"
agent-browser click @e3
agent-browser wait --url "**/dashboard"
agent-browser state save auth.json

# Reuse in future sessions
agent-browser state load auth.json
agent-browser open https://app.example.com/dashboard

Data Extraction

agent-browser open https://example.com/products
agent-browser snapshot -i
agent-browser get text @e5           # Get specific element text
agent-browser get text body > page.txt  # Get all page text

# JSON output for parsing
agent-browser snapshot -i --json
agent-browser get text @e1 --json

Parallel Sessions

agent-browser --session site1 open https://site-a.com
agent-browser --session site2 open https://site-b.com

agent-browser --session site1 snapshot -i
agent-browser --session site2 snapshot -i

agent-browser session list

Visual Browser (Debugging)

agent-browser --headed open https://example.com
agent-browser highlight @e1          # Highlight element
agent-browser record start demo.webm # Record session

Local Files (PDFs, HTML)

# Open local files with file:// URLs
agent-browser --allow-file-access open file:///path/to/document.pdf
agent-browser --allow-file-access open file:///path/to/page.html
agent-browser screenshot output.png

iOS Simulator (Mobile Safari)

# List available iOS simulators
agent-browser device list

# Launch Safari on a specific device
agent-browser -p ios --device "iPhone 16 Pro" open https://example.com

# Same workflow as desktop - snapshot, interact, re-snapshot
agent-browser -p ios snapshot -i
agent-browser -p ios tap @e1          # Tap (alias for click)
agent-browser -p ios fill @e2 "text"
agent-browser -p ios swipe up         # Mobile-specific gesture

# Take screenshot
agent-browser -p ios screenshot mobile.png

# Close session (shuts down simulator)
agent-browser -p ios close

Requirements: macOS with Xcode, Appium (npm install -g appium && appium driver install xcuitest)

Real devices: Works with physical iOS devices if pre-configured. Use --device "<UDID>" where UDID is from xcrun xctrace list devices.

Ref Lifecycle (Important)

Refs (@e1, @e2, etc.) are invalidated when the page changes. Always re-snapshot after:

• Clicking links or buttons that navigate
• Form submissions
• Dynamic content loading (dropdowns, modals)

agent-browser click @e5              # Navigates to new page
agent-browser snapshot -i            # MUST re-snapshot
agent-browser click @e1              # Use new refs

Semantic Locators (Alternative to Refs)

When refs are unavailable or unreliable, use semantic locators:

agent-browser find text "Sign In" click
agent-browser find label "Email" fill "[e-mail]"
agent-browser find role button click --name "Submit"
agent-browser find placeholder "Search" type "query"
agent-browser find testid "submit-btn" click

Quick Start

# Open a page and interact with it
agent-browser open https://example.com/form
agent-browser snapshot -i
agent-browser fill @e1 "[e-mail]"
agent-browser click @e2
agent-browser wait --load networkidle
agent-browser screenshot result.png

Examples

Example 1: Fill and Submit a Form

User says: "Fill out the contact form on example.com with my email" Steps:

agent-browser open https://example.com/contact
agent-browser snapshot -i
# Output: @e1 [input name="email"], @e2 [input name="message"], @e3 [button] "Send"
agent-browser fill @e1 "[e-mail]"
agent-browser fill @e2 "Hello, I have a question."
agent-browser click @e3
agent-browser wait --load networkidle
agent-browser snapshot -i  # Verify success

Output: Snapshot showing confirmation message Reply: "Done! The form has been submitted. The page now shows a confirmation message."

Error Handling

Error	Cause	Solution
Error: Browser not found	Playwright not installed	Run agent-browser install to install browsers
@e1 ref not found	Page changed, refs invalidated	Run agent-browser snapshot -i again to get fresh refs
Navigation timeout	Page takes too long to load	Use agent-browser wait --load networkidle with longer timeout
Element not clickable	Element hidden or off-screen	Use agent-browser scroll down 500 then retry
appium: command not found	Appium not installed for iOS	Run npm install -g appium && appium driver install xcuitest
Session expired	Browser session closed	Re-run agent-browser open <url> to start fresh

Deep-Dive Documentation

Reference	When to Use
references/commands.md	Full command reference with all options
references/snapshot-refs.md	Ref lifecycle, invalidation rules, troubleshooting
references/session-management.md	Parallel sessions, state persistence, concurrent scraping
references/authentication.md	Login flows, OAuth, 2FA handling, state reuse
references/video-recording.md	Recording workflows for debugging and documentation
references/proxy-support.md	Proxy configuration, geo-testing, rotating proxies

Ready-to-Use Templates

Template	Description
templates/form-automation.sh	Form filling with validation
templates/authenticated-session.sh	Login once, reuse state
templates/capture-workflow.sh	Content extraction with screenshots

./templates/form-automation.sh https://example.com/form
./templates/authenticated-session.sh https://app.example.com/login
./templates/capture-workflow.sh https://example.com ./output