browser

name	browser
description	Browse, navigate, click, fill forms, screenshot, or scrape web pages with a stealth (Camoufox) browser.

Browser

Camoufox (an anti-detection Firefox that spoofs its fingerprint in C++ below JS) driven over raw WebDriver BiDi. Accessibility-tree snapshots with numbered refs (e1, e2) for the ergonomic path, and click(x, y) + screenshots for everything accessibility can't see. The helpers are Python, short, and agent-editable. When something is missing, write it.

Stealth is structural, not a flag: Camoufox is always fingerprint-spoofed, and headless leaks nothing (unlike stock Chromium). Each profile gets one coherent fingerprint, stable across restarts, different across profiles.

Setup: SETUP.md

Search first

Before inventing an approach to a site, check domain-skills/<host>/ for saved recipes and interaction-skills/ for reusable mechanics (dialogs, iframes, shadow DOM, uploads, scrolling, dropdowns). When you open or navigate to a URL, the CLI prepends a banner listing any matching recipes. Read them.

# List everything we know about a site
ls ~/agent/skills/browser/domain-skills/amazon/
# Or search broadly
rg "<selector or keyword>" ~/agent/skills/browser/domain-skills/ ~/agent/skills/browser/interaction-skills/

Two ways to drive the browser

1. Bash subcommands (ref-based, agent-ergonomic)

browser launch                          # once per session (fetches Camoufox on first use)
browser open "https://example.com"      # opens a new tab and prints a snapshot with e1, e2 refs
browser click e5                        # click the ref
browser type e3 "hello" --submit        # type and press Enter
browser snapshot --interactive          # only interactive elements
browser screenshot --path /tmp/s.png    # PNG of current viewport

Every action command returns an updated snapshot. Use refs from the most recent snapshot only; a fresh snapshot invalidates older refs, and navigating away invalidates them all.

2. Python stdin mode (programmatic, multi-step flows)

browser <<'PY'
goto("https://news.ycombinator.com")
wait_for_load()
print(page_info())
stories = js("[...document.querySelectorAll('.athing .titleline a')].map(a => ({title: a.innerText, url: a.href}))")
print(stories[:5])
PY

All helpers.py primitives are pre-imported: goto, new_tab, switch_tab, list_tabs, ensure_real_tab, click (coordinate), type_text, press_key, scroll, screenshot, page_info, set_viewport, js, bidi (raw escape hatch), drain_events, pending_dialog, http_get, fetch_navigate, wait, wait_for_load, wait_for_text, wait_for_url, upload_file, iframe_target. Plus ref-based variants: click_ref, type_ref, hover_ref, scroll_to_ref.

Command reference

# Session
browser launch                                    # fetch (first time) + launch Camoufox, headless
browser launch --mode screenshot                  # ... and report back with screenshots, not the a11y tree
browser launch --user-data-dir ~/.browser/work    # reuse / isolate a profile (own fingerprint)
browser connect http://192.168.1.10:9222          # attach to the user's own Chrome (CDP), even over a tunnel
browser connect ws://192.168.1.10:9222/session    # attach to a remote Camoufox BiDi endpoint
browser mode screenshot                           # switch perception: a11y | screenshot | both
browser stop                                      # stop this session
browser stop-all                                  # stop everything
browser sessions                                  # list active sessions
browser doctor                                    # report Camoufox install + session health

# Navigation
browser open "URL"                                # new tab + navigate + snapshot
browser navigate "URL"                            # current tab
browser reload / back / forward

# Reads
browser snapshot [--interactive]                  # accessibility tree with e1/e2 refs
browser screenshot [--path PATH] [--full-page] [--webp] [--region X,Y,W,H] [--quality N]
browser pdf [--path PATH]
browser evaluate "document.title"                 # run JS in the page
browser bidi "browsingContext.getTree"            # raw WebDriver BiDi escape hatch
browser bidi "storage.getCookies" '{"filter":{"domain":"example.com"}}'
browser http-get "https://api.example.com/v1/x"   # no browser, pure HTTP
browser fetch "URL" --navigate-first              # render through the stealth browser, return text

# Actions on refs
browser click e5 [--double|--right]
browser click --at 320 180                        # coordinate click (goes through shadow DOM)
browser type e3 "text" [--submit] [--slowly]
browser press Enter
browser press a --modifiers Control               # Ctrl+A
browser hover e2
browser scroll --down 500 / --up 300 / e7

# Tabs
browser tabs
browser focus <context_id>
browser close <context_id>

# Waits
browser wait --text "Welcome"
browser wait --url "**/dashboard"
browser wait --time 2000
browser wait --load-state load

Screenshots

Screenshots are costly in context: prefer --webp (much smaller than PNG) and --region to clip to the part that matters. Use PNG only when you need lossless output (e.g. pixel-diffing UI state). Camoufox captures PNG and JPEG natively; --webp is encoded as JPEG.

Perception: a11y tree or screenshots

You pick how the browser reports back after each action, once per session:

browser mode a11y             # default: every action returns the accessibility tree with e1/e2 refs
browser mode screenshot       # every action returns a screenshot path instead (work from the image)
browser mode both             # return both
browser mode                  # print the current mode
browser launch --mode screenshot   # pick the mode at launch instead of switching after

In a11y mode, drive with refs (browser click e5). In screenshot mode, read /tmp/vesta-browser-view.png, find the target's pixel, and use coordinate actions (browser click --at X Y). browser snapshot and browser screenshot always work regardless of mode when you want the other view on demand.

Refs vs coordinates

Use refs (e1, e2) first. They're semantic, survive layout changes within a snapshot, and work on 95% of pages. Refs come from the accessibility snapshot, which computes each element's role and accessible name with the full W3C accname algorithm.

Drop to click(x, y) / browser click --at X Y when:

• The target is inside a shadow DOM or cross-origin iframe (input-level click passes through)
• The accessibility tree is misleading or the element has no ARIA role
• You're following a screenshot-based flow (read pixel, click pixel, re-screenshot to verify)

input.performActions dispatches a real pointer event at that viewport point regardless of DOM structure.

When stealth isn't enough (escalation)

Camoufox stealth handles the large majority of sites. When one still blocks you, escalate in this order, most-preferred first:

1. Stealth (default). Just browser launch. Try this first, always.
2. Handover (primary fallback). If a site gates on account trust (sign-in walls, banking, locked tenants) rather than fingerprint, hand your headed browser to the user to sign in once; the session persists in the shared profile and you resume automating. One command does it: browser handover start --url "<sign-in URL>" registers the public route itself and returns a ready-to-send user_url (send the user that link, not web_port). See interaction-skills/handover.md.
3. Remote-control the user's own browser (last resort). Only when you specifically need their logged-in Chrome, drive it over a tunnel with browser connect. See interaction-skills/remote-control.md.

More

Occasional topics live in their own files so this one stays lean:

• interaction-skills/advanced-usage.md : extending helpers, multi-session, the raw BiDi escape hatch, how stealth works, contributing back
• interaction-skills/ : reusable mechanics (dialogs, iframes, shadow DOM, uploads, tabs)

Troubleshooting

• no Camoufox for this session: run browser launch first, or set VESTA_BROWSER_BIDI_WS to a remote BiDi endpoint.
• daemon did not come up: check /tmp/vesta-browser-<session>.log for the reason, and browser doctor for install/session state.
• First launch is slow: Camoufox (~650 MB) is fetched and cached under ~/.cache/camoufox/<version>/ on first browser launch; subsequent launches are instant.
• Bot detection / blocked: browser screenshot to see the page. Camoufox is already stealthy, so a block is usually account-trust, geo/IP, or a CAPTCHA, so try handover.
• Stale refs: take a fresh browser snapshot after navigation or major DOM change.

Auth-gated / heavy-JS pages

• Heavy-JS auth pages (Google, Zoom, Apple idmsa/dev.apple, and similar SPAs) HANG goto/wait_for_load for the full timeout: these SPAs keep network connections open so the load event never fires cleanly, yet the page almost always loaded underneath. After a goto that times out, do NOT retry it and NEVER call wait_for_load() on these. Run a SEPARATE short call (timeout 30 browser) that reads page_info() and inspects the DOM directly (no navigation, no wait_for_load). Cap every browser call at timeout 40-55 so a hang costs seconds, not minutes. Long blocking browser calls during a live exchange make the agent go unresponsive, so keep them short.
• Login-form fills need care: some sign-in widgets live in an iframe (e.g. Apple's aid-auth-widget-iFrame, same-origin so contentDocument works). A JS value-set often does NOT satisfy the form's own validation (it re-renders back to empty after "Verifying..."), and real keystrokes (type_text) can DOUBLE a field that already holds a value, so CLEAR it first. These flows also commonly gate on device-2FA or a passcode that only the user's phone has, so browser automation frequently cannot finish them: prefer the official API or app path.