browserclaw

name	browserclaw
description	Scrape browser traffic with Playwright → infer API endpoints → generate Python client + MCP tools artifacts.
agent_scope	Any AI agent (Codex, Claude Code, OpenClaw, Hermes, AO)

browserclaw

Inspect browser-driven API traffic and convert captured patterns into reusable artifacts — a Python client for programmatic control and MCP tool definitions for tool-calling agents.

Scope

Any public website — not limited to "the app" or "the auth handshake." browserclaw captures traffic from any URL you can reach in Chrome/Chromium. The "no login bypass" rule is about authentication (don't bypass logins/CAPTCHAs), NOT about which sites you can browse. For sites where you're already authenticated in Chrome, use --storage-state (or browserclaw cookies decrypt --db <chrome-cookies-db> + browserclaw cookies inject --cookies <out.json>) to reuse your existing session cookies. This works for any site where Chrome has cookies: GCP Console, Slack, GitHub, Firebase Console, etc.

What it does

1. Capture — Playwright + Chromium records HTTP traffic to a HAR file
2. Infer — Extract API endpoints (method, URL template, query keys, body keys) from HAR
3. Enrich — Optionally call an LLM to add natural-language descriptions
4. Generate — Emit catalog.json, generated_client.py, and mcp_tools.json

Prerequisites

# Python 3.11+, pip
python -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'
python -m playwright install chromium

Environment variables (optional, for LLM enrichment):

• ANTHROPIC_API_KEY (provider: anthropic)
• OPENAI_API_KEY (provider: openai)
• GEMINI_API_KEY (provider: gemini)

Capture flow

Option A — Manual (interactive, recommended for auth-gated sites)

browserclaw capture \
  --url https://app.example.com/dashboard \
  --output generated/example/capture.har \
  --manual

The tool opens a Chromium window. You log in and browse as usual. Press Enter when done. The HAR (with embedded request/response bodies) is saved.

Option B — Headless (non-interactive, no stdin needed)

browserclaw capture \
  --url https://app.example.com \
  --output generated/example/capture.har \
  --headless \
  --wait-after-load 5

--headless always runs non-interactively. Combine with --steps for scripted navigation.

Option C — Scripted steps (JSON)

[
  {"action": "goto", "url": "https://app.example.com/login"},
  {"action": "fill", "selector": "input[name=email]", "value": "[e-mail]"},
  {"action": "fill", "selector": "input[name=password]", "value": "secret"},
  {"action": "click", "selector": "button[type=submit]"},
  {"action": "wait_for_url", "value": "**/dashboard"},
     
     

Supported actions: goto, click, fill, press, wait_for_timeout, wait_for_url, eval.

browserclaw capture \
  --url https://app.example.com \
  --output generated/example/capture.har \
  --steps steps.json

Option D — Authenticated capture (injected headers)

For Firebase-backed SPAs or sites requiring a Bearer token:

browserclaw capture \
  --url https://app.example.com \
  --output capture.har \
  --headless \
  --extra-headers Authorization="Bearer $FIREBASE_TOKEN"

Mint a Firebase test token with the Admin SDK:

import firebase_admin
from firebase_admin import credentials, auth
cred = credentials.Certificate('service-account.json')
firebase_admin.initialize_app(cred)
token = auth.create_custom_token('user-uid').decode()
print(f"Bearer {token}")

Use --eval '{"action":"eval","value":"window.testAuthBypass={enabled:true}"}' to activate JS test-mode flags in the browser.

Option E — LLM-planned steps

browserclaw capture \
  --url https://app.example.com \
  --output generated/example/capture.har \
  --goal "Log in, navigate to invoices, open the first one" \
  --provider anthropic \
  --model claude-sonnet-4-20250514

LLM produces a deterministic step plan. No login or CAPTCHA bypass — you still authenticate manually.

Option F — One-shot (capture + infer + generate)

browserclaw reverse \
  --url https://app.example.com \
  --output-dir generated/example \
  --manual

Runs capture, infer, and generate in sequence. Emits:

• generated/example/capture.har
• generated/example/catalog.json
• generated/example/generated_client.py
• generated/example/mcp_tools.json

Infer flow

browserclaw infer \
  --har generated/example/capture.har \
  --output generated/example/catalog.json

Optionally enrich with an LLM for natural-language descriptions:

browserclaw infer \
  --har generated/example/capture.har \
  --output generated/example/catalog.json \
  --site example.com \
  --goal "Document the invoice list and detail endpoints" \
  --provider anthropic \
  --model claude-sonnet-4-20250514

Generate flow

browserclaw generate \
  --catalog generated/example/catalog.json \
  --output-dir generated/example

Artifact reference

capture.har

Raw Playwright HAR — full request/response bodies embedded. Contains all HTTP methods: GET, POST, PUT, PATCH, DELETE, and any others observed.

catalog.json

Structured endpoint catalog. Each entry:

{
  "name": "get_feed_updates",
  "method": "GET",
  "url_template": "https://api.example.com/v1/feed/{id}",
  "host": "api.example.com",
  "query_keys": ["limit", "offset", "filter"],
  "request_body_keys": [],
  "response_header_keys": ["content-type", "x-request-id"],
  "sample_status_codes": [200, 401],
  "sample_content_types": ["application/json"],
  "description": "Inferred from 3 captured requests."

generated_client.py

A requests.Session-based Python client. Example:

"""Generated by browserclaw from capture.har."""

import requests


class BrowserClawClient:
    def __init__(self, session: requests.Session | None = None):
        self.session = session or requests.Session()

    def get_feed_updates(self, limit=None, offset=None, filter=None):
        """Inferred from 3 captured requests."""
        url = "https://api.example.com/v1/feed".format()
        params = {"limit": limit, "offset": offset, "filter": filter}
        params = {key: value for key, value in params.items() if value is not None}
        payload = {}
        payload = {key: value for key, value in payload.items() if value is not None}
        response = self.session.request(
            "GET",
            url,
            params=params or None,
            json=payload or None,
        )
        response.raise_for_status()
        return response.json() if response.content else None

     ():
        
        url = .(id_0)
        params = {}
        params = {key: value  key, value  params.items()  value   }
        payload = {: title, : body}
        payload = {key: value  key, value  payload.items()  value   }
        response = .session.request(
            ,
            url,
            params=params  ,
            json=payload  ,
        )
        response.raise_for_status()
         response.json()  response.content  

mcp_tools.json

MCP tool definitions for tool-calling agents:

{
  "schema_version": "2025-03-26",
  "site": "example.com",
  "tools": [
    {
      "name": "get_feed_updates",
      "description": "Inferred from 3 captured requests.",
      "inputSchema": {
        "type": "object",
        "properties": {
          "limit": {"type": "string", "description": "Inferred parameter for limit"},
          "offset": {"type": "string", "description": "Inferred parameter for offset"}
        },
          
         
      
       
         
         
      
    
  

Using the generated Python client

from generated_client import BrowserClawClient

client = BrowserClawClient()

# Set auth (cookies or token from manual login)
client.session.cookies.set("session", "your-session-cookie")
# or
client.session.headers["Authorization"] = "Bearer your-token"

# List resources
updates = client.get_feed_updates(limit=10, offset=0)
print(updates)

# Create a resource
result = client.create_feed_item(title="Hello", body="World content")
print(result)

Using MCP tools in an agent

Register the mcp_tools.json with your agent's MCP server, or parse it and wire each tool to a handler that calls generated_client.py.

When the browser path fails — Python GraphQL escape hatch

browserclaw cookies inject into headless Chromium does not always carry the session across to the SPA's dashboard (cookies are bound to device UUIDs, Cloudflare clearances, etc.). When the headless Chromium lands on a login page despite 500+ cookies injected, do NOT declare the site unreachable. Walk this ladder:

1. Decrypt the source DB cookies (Aside, Chrome, etc.) with browserclaw cookies decrypt. Read the file: json.load(open(file))['cookies'] is the Playwright storage_state format.
2. Inspect for the actual session marker — session_id, cf_clearance, csrftoken. If document.cookie in the browser shows only analytics/CSRF cookies but the DB has session_id, the DB cookies are stale OR the active page never inherited them.
3. Call the internal API directly with Python:

   import json, requests
   blob = json.load(open('/tmp/cookies.json'))
   s = requests.Session()
   for c in blob['cookies']:
       s.cookies.set(c['name'], c['value'], domain=c['domain'], path=c.get('path','/'))
   csrf = s.cookies.get('csrftoken', domain='.example.com')
   s.headers.update({
       'X-CSRFToken': csrf,
       'Origin': 'https://app.example.com',
       'Referer': 'https://app.monarch.com/',  # exact referer the SPA sends
       'Content-Type': 'application/json',
       'User-Agent': 'Mozilla/5.0 (compatible)',
   })
   r = s.post('https://api.example.com/graphql', json={"query":"{ me { id } }"})
   print(r.status_code, r.text[:200])
   

4. Worked Monarch example (2026-07-22): 537 Aside cookies extracted → CSRF token AQT26Ys6dtFUZd3Kd5sWF9cQjq6rVlyD + Origin: https://app.monarch.com + Referer: https://app.monarch.com/ → returned . Then returned ~12 account balances including the $1,862.06 EOD balance on account ending 6425 (the Chase checking that was -$2,585.24 overdrawn the day before).

Anti-pattern: declaring the dashboard inaccessible after browserclaw cookies inject into headless Chromium shows the login page. The dashboard may be unreachable BUT the API often is. Always try the API path before asking the user for credentials.

Guardrails

• No auth bypass — You authenticate manually during capture. The tool does not attempt to bypass login, MFA, or CAPTCHA.
• No stealth — Captured traffic includes real user-agent, cookies, and headers. The site sees a real browser session.
• Authorized use only — Inspect only sites you have permission to reverse-engineer.
• No browser extension — All capture runs through Playwright + Chromium directly.
• Review generated code — Client stubs and MCP tools are inferred from traffic, not audited APIs. Validate before production use.