web-ai

# Web AI Use this skill when an agent needs to drive AI provider websites through the standalone `agbrowse` Chrome/CDP runtime. ## Preconditions - Use the existing headed Chrome profile when provider login is required. - Provider commands auto-start headed Chrome when CDP is not running. Set `AGBROWSE_WEB_AI_AUTO_START=0` to fail closed instead. - Do not start a separate Chrome/profile unless the user explicitly asks for an isolated run. - Prefer `agbrowse status` before mutation. - If the user explicitly says to use `agbrowse` or standalone agbrowse, run `agbrowse --help` first, and run `agbrowse web-ai --help` before choosing web-ai flags. Treat the current help output as the command truth and adapt these instructions to that surface. - For live provider smoke tests, pass `--url` so the runtime verifies the provider host before sending. ## Commands ```bash agbrowse web-ai render agbrowse web-ai status agbrowse web-ai send agbrowse web-ai poll agbrowse web-ai query agbrowse web-ai code agbrowse web-ai code-extract agbrowse web-ai stop agbrowse web-ai watch agbrowse web-ai snapshot agbrowse web-ai sessions agbrowse web-ai doctor agbrowse web-ai project-sources agbrowse web-ai context-dry-run agbrowse web-ai context-render agbrowse web-ai claim-audit ``` Direct script form is equivalent: ```bash node skills/browser/browser.mjs web-ai query ... ``` ## Runtime Capabilities Use `agbrowse web-ai status --vendor <v> --json` before any mutation. The JSON response contains `capabilities[]` rows with `capabilityId`, `state` (`ok`/`warn`/`fail`/`unknown`), `evidence`, and `next` (retry hint). Scope a single probe with `--probe <capabilityId>`. Capability IDs per vendor (hyphenated, aligned with cli-jaw registry shape but implementation details may differ per vendor): | Capability | ChatGPT | Gemini | Grok | | --- | :---: | :---: | :---: | | `*-active-tab-verification` | ✓ | ✓ | ✓ | | `*-composer-visible` | ✓ | ✓ | ✓ | | `*-model-alias-selectable` | ✓ | ✓ | ✓ | | `*-upload-surface-visible` | ✓ | ✓ | ✓ | | `*-copy-button-present` | ✓ | ✓ | ✓ | | `*-response-streaming` | ✓ | ✓ | ✓ | Unsupported vendors or unsupported model aliases must fail before browser mutation. ### Grok: do not use context packaging Do not package files for Grok. The runtime still allows it for parity, but agents must prefer inline prompts and per-call `--file <path>` uploads instead of `--context-from-files` / `--context-file` / `--context-transport upload` when `--vendor grok`. Why: Grok's web composer attachment surface is less predictable than ChatGPT and Gemini, the package upload path frequently degrades response quality, and Grok already accepts long inline prompts well. Use ChatGPT or Gemini when the prompt genuinely needs a packaged context bundle. Grok context packages fail closed by default. If `web-ai send/query --vendor grok` is invoked with `--context-from-files` / `--context-file` / `--context-transport upload` and `--allow-grok-context-pack` is not passed, the runtime throws with `stage: 'grok-context-pack-not-allowed'`. Pass `--allow-grok-context-pack` to override deliberately for raw context only; the runtime still emits the `grok-context-pack-not-recommended` warning when the override is used. `web-ai render/send/query/code --vendor grok` explicitly rejects Repomix. The provider-neutral `context-render` and `context-dry-run` utilities only build or inspect the package and do not imply live Grok support. ## Polling Timeouts `web-ai poll`, `web-ai query`, and `web-ai watch` accept `--timeout <seconds>`. Timeout resolution is `explicit timeout → stored session deadline remainder → tier default → vendor fallback`. A resumed poll therefore keeps the deadline created by the original submit unless the caller explicitly overrides it. | Long-running tier | Default `--timeout` | Roughly | | --- | ---: | --- | | `chatgpt-pro` | 5400 | 90 minutes | | `grok-heavy` | 3600 | 60 minutes | | `deep-research` | 3600 | 60 minutes | | Vendor fallback when the tier is unknown | Default `--timeout` | Roughly | | --- | ---: | --- | | ChatGPT | 1200 | 20 minutes | | Gemini | 1200 | 20 minutes | | Grok | 600 | 10 minutes | Do not equate ChatGPT's UI-side reasoning budget with the agbrowse poll deadline. The roughly 40-minute Pro budget is a user report and was not present in the 2026-07-10 DOM. The 5400-second (90-minute) `chatgpt-pro` default is agbrowse polling headroom; `grok-heavy` and `deep-research` remain independent 3600-second tiers. The provider tab and agbrowse Chrome process remain open when polling times out. MCP clients should preserve the `sessionId` returned by `web_ai_submit_prompt` or `web_ai_work_send`. `web_ai_wait_response` and `web_ai_session_resume` use the same stored deadline and recovery path and may return a recoverable timeout with `retryHint: poll-or-resume`. A host-level MCP request timeout, such as `-32001 Request timed out`, is a client/runtime boundary; after it, retry with the same `sessionId` or use CLI `web-ai poll --session <SID>` instead of sending a new prompt immediately. ## ChatGPT Work Use the dedicated Work entrypoint; Chat `send/query/poll/watch` and `web_ai_submit_prompt` reject an active Work surface. ```bash agbrowse web-ai work send --prompt "Analyze this repository" --power 4 ``` `--power` is an integer from 1 through 6. The runtime follows the WP1-probed Power-to-Model/Effort mapping; do not infer Terra/Luna or Chat family aliases. For MCP clients, use `web_ai_work_send` with `prompt` and `power`. Do not add a `surface` field to `web_ai_submit_prompt`. ## Long-Running / Background Sessions For responses that may take many minutes (ChatGPT Pro/Heavy, Gemini Deep Think, Deep Research), do not block your agent turn on `query`. Split into `send` + background `watch`: ```bash SID=$(agbrowse web-ai send --vendor chatgpt --model pro --inline-only \ --prompt "..." --json | jq -r .sessionId) # separate/background process: agbrowse web-ai watch --session "$SID" --json --navigate ``` Key facts (verified 2026-06-11, details in `devlog/_fin/260611_background_runtime_hook/02_agbrowse_sufficiency.md`): - Sessions persist in `~/.browser-agent/web-ai-sessions.json` and survive process/machine restarts. `sessions show <SID> --json` is safe from any process. - ⚠️ `sessions show` is **read-only** — it never advances state. Only `watch`/`poll` drive the browser DOM and move a session to a terminal status (`complete`/`timeout`/`error`). A monitor that only re-reads the store will wait forever. - `watch` emits line-delimited JSON events (`watch.start`, `watch.tick` every 15s by default, then `watch.complete`/`watch.timeout`/`watch.error`) and exits on terminal status. Parse the last terminal line, then fetch the full answer with `sessions show <SID> --json`. - A per-session watcher lock makes concurrent/duplicate `watch` calls fail closed and auto-recovers stale locks from dead processes — re-running `watch` after a crash is safe. - There is no `--on-complete` callback flag. Chain with the shell instead: `agbrowse web-ai watch --session "$SID" --json; <notify command>`. Per-runtime pattern for the background `watch` process: | Agent runtime | Recommended pattern | | --- | --- | | Claude Code | Run `watch` via `Bash run_in_background: true` — process exit injects a completion notification that re-activates the agent (no polling). | | Cursor | Run `watch` as a background shell, then use the `Await` tool to wait for a `watch.complete` sentinel line. | | Codex | Background terminal polling works (15s ticks avoid the 5-min empty-poll window), but for very long runs prefer a fully external watcher that calls `codex exec resume` on completion. | | cli-jaw | Boss turns are disposable — do not background `watch` inside a turn. Register a server-owned task instead: `cli-jaw bgtask add --preset web-ai --session "$SID"` (native session probe), then end the turn — the jaw server re-invokes the boss with a `[bgtask:*]` prompt on completion (restart-durable). | ## Multi-Tab Behavior (Phase 9.1+) By default, `send` and `query` create a **new browser tab** for each session. This isolates conversations and prevents context contamination. ### Tab Reuse Reuse the existing active tab (legacy single-tab behavior): ```bash agbrowse web-ai send --vendor chatgpt --reuse-tab --inline-only --prompt "hello" # or globally: export AGBROWSE_REUSE_TAB=1 ``` ### Session-to-Tab Binding Each session record stores `targetId`, `tabId`, and `tabState`. When `poll` or `stop` is invoked with `--session <id>`, the runtime switches to that session's bound tab automatically. If the tab was closed, it auto-recovers by creating a new tab and navigating to the saved `conversationUrl`. ### Shared CDP Session Ambiguity All provider commands may share one Chrome CDP port, such as `9222`. For session-less `poll` or `stop`: `0` active sessions preserve legacy current-tab behavior, `1` active provider session auto-binds with a warning, and `2+` fail closed with `session.target-ambiguous` plus candidate `sessionId`/`targetId` evidence. Rerun with `--session <id>`; for tab drift or missing target recovery, add `--navigate`. ### Tab Pooling (Phase 9.2) Completed session tabs are kept in a vendor-specific pool for reuse. The next `send` for the same vendor will reuse a pooled tab instead of creating a new one, reducing tab creation overhead in batch scenarios. | Pool setting | Default | Env Var | | --- | --- | --- | | TTL per pooled tab | 30 min | `AGBROWSE_PROVIDER_POOL_TTL` | | Max warm tabs per `(owner,vendor,sessionType,origin,profile)` | 3 | `AGBROWSE_PROVIDER_POOL_MAX_PER_KEY` | | Global cap on warm provider tabs | 8 | `AGBROWSE_PROVIDER_POOL_GLOBAL_MAX` | Use `--new-tab` (or its alias `--parallel`) on `send` / `query` to bypass pool reuse for a single call — needed when you want a Pro query to run alongside another in-flight Pro query without lease contention. ### Tab Lifecycle | Setting | Default | Env Var | | --- | --- | --- | | Max tabs | 20 | `AGBROWSE_MAX_TABS` | | Idle timeout | 30 min | `AGBROWSE_TAB_IDLE` | Idle tabs (inactive longer than the timeout) are auto-closed unless pinned or bound to an active session. Before creating a new web-ai tab, agbrowse runs lifecycle cleanup so long-lived agent sessions do not keep growing Chrome tab count. For explicit cleanup: ```bash agbrowse tabs --json agbrowse tab-cleanup --json agbrowse tab-cleanup --include-untracked --idle-after 10m ``` Use `--include-untracked` only when you intentionally want to close older tabs that predate the activity metadata file. ## Render First Use render to inspect the exact structured prompt shape: ```bash agbrowse web-ai render \ --vendor chatgpt \ --project "project name" \ --goal "what the provider should do" \ --prompt "question" ``` Envelope shape: ```text [SYSTEM] ... [USER] ## Project ... ## Goal ... ## Question ... ``` ## Live Query Examples ChatGPT Pro: ```bash agbrowse web-ai query \ --vendor chatgpt \ --url https://chatgpt.com/ \ --model pro \ --inline-only \ --allow-copy-markdown-fallback \ --prompt "Reply exactly CHATGPT_OK" ``` Gemini: ```bash agbrowse web-ai query \ --vendor gemini \ --url https://gemini.google.com/app \ --model fast \ --inline-only \ --prompt "Reply exactly GEMINI_OK" ``` Grok: ```bash agbrowse web-ai query \ --vendor grok \ --url https://grok.com/ \ --model expert \ --inline-only \ --prompt "Reply exactly GROK_OK" ``` ## Where instructions go — `--system` vs `--context` vs `--file` Attaching instructions *and* a file already works today; pick the right channel: - **Operating instructions / skill guidance** ("how to behave", "extract X from the attached file") → `--system "..."`. Rendered in the trusted `[SYSTEM]` section and honored. `--goal` / `--constraints` (USER fields) are also trusted. - **Untrusted reference *data*** (scraped page text, provider output) → `--context "..."`. Rendered as `[UNTRUSTED_CONTEXT]`; **instructions placed here are ignored by design** (prompt-injection boundary). Never put operating instructions in `--context`. - **A file for the model to read/analyze** → `--file <path>` (repeatable; native, dialog-free attachment). The model sees it directly — you do **not** need to also describe it in `--context`. Common mistake: putting instructions in `--context` and finding they are "ignored" → move them to `--system`. Example of the working combo: ```bash agbrowse web-ai query --vendor chatgpt --model thinking --effort high --family gpt-5.6-sol \ --system "Extract every breaking change from the attached PDF and group by module." \ --file ./spec.pdf \ --prompt "Summarize the breaking changes." ``` If you are unsure whether/how a file is attached, just **ask the user** rather than relying on envelope guesswork. ## File Upload ```bash agbrowse web-ai query \ --vendor gemini \ --url https://gemini.google.com/app \ --model fast \ --file /path/to/user-requested-file.pdf \ --file /path/to/user-requested-image.png \ --prompt "Summarize the attached file." ``` `--file` is repeatable and may mix file types such as images, archives, PDFs, and text files in one prompt. Use it only when the user explicitly wants those specific files uploaded. For source/project context, use context packaging instead of creating temporary `.txt`/`.md` files. Upload must verify visible attachment evidence and sent-turn evidence where the provider exposes it. Input-only success is not enough. Large files: ChatGPT uploads at or above ~45MB are injected through raw CDP `DOM.setFileInputFiles` (local path, zero-byte transfer), bypassing the Playwright connectOverCDP ~50MB transfer limit; smaller files fall back to CDP automatically when Playwright rejects the transfer. Upload timeouts scale with total attachment bytes (acceptance up to 15 minutes). Overrides: `--attachment-upload-timeout-ms <ms>` (or `AGBROWSE_ATTACHMENT_UPLOAD_TIMEOUT_MS`) for the browser handoff, `AGBROWSE_ATTACHMENT_ACCEPT_TIMEOUT_MS` as an acceptance-wait floor. Missing sent-turn attachment evidence now FAILS the send (`provider.sent-attachment-missing`) instead of returning a false `sent`;