gsd-browser

# Browser Automation with gsd-browser ## Critical Rules 1. **The daemon auto-starts on browser commands.** `daemon health` only reports state; it does not start a session. Use `daemon start` only when you want to pre-warm or verify daemon lifecycle explicitly. 2. **Always re-snapshot after page changes.** Refs are versioned (`@v1:e1`). After navigation, form submission, or dynamic content loading, old refs are stale. Run `gsd-browser snapshot` to get fresh refs. 3. **Use `--json` when parsing output.** Use text mode when reading output yourself. Use `--json` when you need to extract values programmatically (e.g., checking assertion results, parsing snapshot refs). 4. **Positional args have no flag prefix.** Commands like `click`, `type`, `hover` take positional args — do NOT add `--selector`. See exact syntax in command reference below. 5. **Use `batch` for atomic multi-step flows.** Batch reduces round trips and keeps pass/fail checks in one call. Use separate commands when you need intermediate output (e.g., snapshot to discover refs). 6. **Use `view` when the user wants to watch or direct the browser.** The live viewer is an authenticated local workbench with Control, Annotate, Record, and Sensitive modes. Keep CLI commands on the same named session. ## Core Workflow Every browser automation follows this pattern: 1. **Navigate**: `gsd-browser navigate <url>` 2. **Snapshot**: `gsd-browser snapshot` (get versioned refs like `@v1:e1`, `@v1:e2`) 3. **Interact**: Use refs to click, fill, hover 4. **Re-snapshot**: After navigation or DOM changes, get fresh refs ```bash gsd-browser navigate https://example.com/form gsd-browser snapshot # Output: @v1:e1 [input type="email"], @v1:e2 [input type="password"], @v1:e3 [button] "Submit" gsd-browser fill-ref @v1:e1 "user@example.com" gsd-browser fill-ref @v1:e2 "password123" gsd-browser click-ref @v1:e3 gsd-browser wait-for --condition network_idle gsd-browser snapshot # REQUIRED — old refs are now stale ``` ## Command Chaining Commands can be chained with `&&` in a single shell invocation. Browser state also persists across separate invocations through the background daemon when you stay on the same session. ```bash # Chain navigate + wait + snapshot gsd-browser navigate https://example.com && gsd-browser wait-for --condition network_idle && gsd-browser snapshot # Chain multiple interactions gsd-browser fill-ref @v1:e1 "user@example.com" && gsd-browser fill-ref @v1:e2 "password123" && gsd-browser click-ref @v1:e3 ``` **When to chain:** Use `&&` when you don't need intermediate output. Run commands separately when you need to parse output first (e.g., snapshot to discover refs, then interact). --- ## Command Reference Argument syntax: `<arg>` = required positional, `[arg]` = optional positional, `--flag` = named option. Do NOT add `--` prefix to positional args. ### Navigation ```bash gsd-browser navigate <url> # Navigate to a URL gsd-browser back # Go back in browser history gsd-browser forward # Go forward in browser history gsd-browser reload # Reload the current page ``` ### Interaction All selectors are **positional** — do NOT use `--selector`. ```bash # Click gsd-browser click <selector> # Click by CSS selector gsd-browser click --x 100 --y 200 # Click by coordinates (no selector) # Type — positional: <selector> <text> gsd-browser type <selector> <text> # Atomic fill (replaces content) gsd-browser type <selector> <text> --slowly # Character-by-character gsd-browser type <selector> <text> --clear-first # Clear field before typing gsd-browser type <selector> <text> --submit # Press Enter after typing # Other interaction — all positional selectors gsd-browser press <key> # Press key: Enter, Escape, Tab, Meta+A gsd-browser hover <selector> # Hover over element gsd-browser scroll --direction down # Scroll down 300px (default) gsd-browser scroll --direction up --amount 500 # Scroll up 500px gsd-browser select-option <selector> <option> # Select dropdown by label or value gsd-browser set-checked <selector> --checked # Check checkbox/radio (omit --checked to uncheck) gsd-browser drag <source-selector> <target-selector> # Drag-and-drop gsd-browser upload-file <selector> <file>... # Set files on <input type="file"> gsd-browser set-viewport --preset mobile # Preset: mobile, tablet, desktop, wide gsd-browser set-viewport --width 1920 --height 1080 # Custom dimensions ``` ### Snapshot & Refs Refs are versioned (`@v1:e1`, `@v2:e3`). The version increments each snapshot. **Old refs become stale after page changes — always re-snapshot.** ```bash # Take snapshot gsd-browser snapshot # Snapshot interactive elements, assign refs gsd-browser snapshot --selector "form" # Scope to a CSS selector gsd-browser snapshot --mode <mode> # Semantic mode (see below) gsd-browser snapshot --limit 80 # Increase element limit (default: 40) # Use refs — all positional gsd-browser get-ref <ref> # Get metadata for a ref gsd-browser click-ref <ref> # Click element by ref gsd-browser hover-ref <ref> # Hover element by ref gsd-browser fill-ref <ref> <text> # Type into element by ref ``` **Snapshot modes** (`--mode`): | Mode | What it captures | |------|-----------------| | `interactive` | Buttons, inputs, links, selects (default) | | `form` | Form fields with labels and current values | | `dialog` | Elements inside open dialogs/modals | | `navigation` | Links and nav elements | | `errors` | Error messages, validation warnings | | `headings` | Heading elements (h1-h6) for page structure | | `visible_only` | All visible elements regardless of interactivity | ### Inspection ```bash gsd-browser accessibility-tree # Full accessibility tree (roles, names, states) gsd-browser find --text "Sign In" # Find elements by text (case-insensitive) gsd-browser find --role button # Find by ARIA role gsd-browser find --selector ".my-class" # Find by CSS selector gsd-browser find --role link --limit 50 # Increase limit (default: 20) gsd-browser page-source # Get raw HTML of page gsd-browser page-source --selector "main" # Scoped HTML source gsd-browser eval '<js-expression>' # Evaluate JavaScript in page context ``` ### Assertions Run explicit pass/fail checks against the current page state. Prefer this over inferring success from output. ```bash gsd-browser assert --checks '[ {"kind": "url_contains", "text": "/dashboard"}, {"kind": "text_visible", "text": "Welcome"}, {"kind": "selector_visible", "selector": "#user-menu"}, {"kind": "value_equals", "selector": "input[name=email]", "value": "user@test.com"}, {"kind": "no_console_errors"}, {"kind": "no_failed_requests"} ]' ``` **Assertion kinds (17):** `url_contains`, `text_visible`, `text_hidden`, `selector_visible`, `selector_hidden`, `value_equals`, `checked`, `no_console_errors`, `no_failed_requests`, `request_url_seen`, `response_status`, `console_message_matches`, `network_count`, `console_count`, `element_count`, `no_console_errors_since`, `no_failed_requests_since`. ### Batch Execution Execute multiple steps in one call to reduce round trips. Stops on first failure by default. ```bash gsd-browser batch --steps '[ {"action": "navigate", "url": "https://example.com"}, {"action": "wait_for", "condition": "network_idle"}, {"action": "click", "selector": "#login-btn"}, {"action": "type", "selector": "input[name=email]", "text": "user@test.com"}, {"action": "type", "selector": "input[name=password]", "text": "secret", "submit": true}, {"action": "assert", "checks": [{"kind": "url_contains", "text": "/dashboard"}]} ]' # With --summary-only to reduce output gsd-browser batch --steps '[...]' --summary-only ``` **Batch actions:** `navigate`, `click`, `type`, `key_press`, `wait_for`, `assert`, `click_ref`, `fill_ref`. ### Wait Conditions ```bash gsd-browser wait-for --condition selector_visible --value "#content" gsd-browser wait-for --condition selector_hidden --value ".spinner" gsd-browser wait-for --condition url_contains --value "/dashboard" gsd-browser wait-for --condition network_idle gsd-browser wait-for --condition delay --value 2000 gsd-browser wait-for --condition text_visible --value "Success" gsd-browser wait-for --condition text_hidden --value "Loading" gsd-browser wait-for --condition request_completed --value "/api/data" gsd-browser wait-for --condition console_message --value "ready" gsd-browser wait-for --condition element_count --value ".item" --threshold ">=5" gsd-browser wait-for --condition region_stable --value "#content" # Custom timeout (default: 10000ms) gsd-browser wait-for --condition selector_visible --value "#slow" --timeout 30000 ``` ### Forms (Smart Fill) Analyze forms and fill them by field label, name, placeholder, or aria-label — no selectors needed. ```bash # Analyze form structure gsd-browser analyze-form gsd-browser analyze-form --selector "#signup-form" # Fill by field identifiers (resolved: label -> name -> placeholder -> aria-label) gsd-browser fill-form --values '{"Email": "a@b.com", "Password": "secret", "Country": "US"}' gsd-browser fill-form --values '{"Email": "a@b.com"}' --submit # Fill and click submit gsd-browser fill-form --values '{"Email": "a@b.com"}' --selector "#login-form" ``` ### Intent-Based Interaction Find and act on elements by semantic intent — no selectors or refs needed. Intents are predefined categories, not free-form text. ```bash # Find top candidates for an intent (returns scored matches with selectors) gsd-browser find-best --intent submit_form gsd-browser find-best --intent accept_cookies gsd-browser find-best --intent primary_cta --scope "#modal" # Act: find best match and click/focus it in one call gsd-browser act --intent submit_form gsd-browser act --intent accept_cookies gsd-browser act --intent fill_email ``` **Intents (15):** | Intent | Action | Description | |--------|--------|-------------| | `submit_form` | click | Submit buttons, form actions | | `close_dialog` | click | Modal/dialog close buttons | | `primary_cta` | click | Primary call-to-action elements | | `search_field` | focus | Search inputs and searchboxes | | `next_step` | click | Next/continue/proceed buttons | | `dismiss` | click | Dismiss overlays, banners, toasts | | `auth_action` | click | Login/signup/register buttons | | `back_navigation` | click | Back/previous navigation links | | `fill_email` | focus | Email input fields | | `fill_password` | focus | Password input fields | | `fill_username` | focus | Username/login input fields | | `accept_cookies` | click | Cookie consent accept buttons | | `main_content` | click | Main content area (`<main>`, `<article>`, semantic markup required) | | `pagination_next` | click | Next page in pagination | | `pagination_prev` | click | Previous page in pagination | ### Pages & Frames Page and frame IDs are **positional** — do NOT use `--id`. ```bash gsd-browser list-pages # List all open tabs gsd-browser switch-page <id> # Switch to tab by ID (positional) gsd-browser close-page <id> # Close a tab by ID (positional) gsd-browser list-frames # List all frames (main + iframes) gsd-browser select-frame --name "iframe-name" # Select frame by name gsd-browser select-frame --url-pattern "embed" # Select frame by URL substring gsd-browser select-frame --index 0 # Select by index gsd-browser select-frame --name main # Return to main frame ``` ### Diagnostics ```bash gsd-browser console # Get console log entries (clears buffer) gsd-browser console --no-clear # Read without clearing gsd-browser network # Get network log entries gsd-browser dialog # Get dialog events (alert, confirm, prompt) gsd-browser timeline # Query the action timeline gsd-browser session-summary # Diagnostic summary of current session gsd-browser debug-bundle # Full debug bundle: screenshot + logs + timeline + a11y tree ``` ### Live Viewer & Workbench The live viewer is a localhost screen-sharing and control surface for the active browser session. It prints or opens a tokenized URL bound to the session, viewer id, local origin, expiry, and capabilities. The viewer displays live frames, narrated action history, ref overlays, target rings, click ripples, failure markers, and page-following across navigation or tab changes. ```bash gsd-browser view # Open the live viewer gsd-browser view --print-only # Print tokenized URL only gsd-browser view --interactive # Interactive local workbench gsd-browser view --history # Open history-focused viewer