browser-agent-bridge

name	browser-agent-bridge
version	1.1.0
description	Use the Chrome Native Messaging browser-control bridge extension (Browser Agent Bridge) from an agent. Trigger when a user asks Codex/agent to inspect or control Chrome through the locally built Browser Agent Bridge, call browser tools exposed at http://127.0.0.1:8765/rpc or ws://127.0.0.1:8765/ws, verify the bridge/native host connection, troubleshoot the extension connection, or automate Chrome tabs/pages without embedding an LLM client in the extension.

Browser Agent Bridge

Use the Browser Agent Bridge extension as a browser tool runtime. The extension contains no agent; an agent calls the native host HTTP or WebSocket endpoint, which forwards JSON-RPC over Native Messaging to Chrome.

Agent -> 127.0.0.1:8765 /rpc or /ws -> native/host.py -> Chrome extension -> Chrome APIs/CDP

Quick Start

1. Ask the user to open the extension side panel and click Start Bridge. The local HTTP/WebSocket bridge is unavailable while the side panel shows Stopped.

2. Check bridge health:

curl -sS http://127.0.0.1:8765/health

3. If healthy and extensionReady is true, call JSON-RPC:

curl -sS -H 'content-type: application/json' \
  -X POST http://127.0.0.1:8765/rpc \
  --data '{"jsonrpc":"2.0","id":"1","method":"session.start","params":{"url":"https://example.com"}}'

4. If /health reports authRequired: true, include Authorization: Bearer $BROWSER_AGENT_BRIDGE_TOKEN. The bundled scripts/rpc.sh and scripts/ws-rpc.js helpers do this automatically when the environment variable is set.

5. Use POST /rpc for one-shot calls. Use ws://127.0.0.1:8765/ws for a long-lived agent session or when you need streamed extension notifications.

WebSocket helper:

scripts/ws-rpc.js '{"jsonrpc":"2.0","id":"ws-1","method":"extension.info","params":{}}'

Notification stream:

scripts/ws-rpc.js --listen

Python client:

scripts/browser_bridge_client.py rpc session.start '{"url":"https://example.com"}'

6. Prefer session.start first for a new isolated workspace, then use the returned session groupId or session.get to discover managed tabId values. tabs.list must be scoped to an Agent-managed group:

{"query":{"groupId":1}}

7. For detailed method parameters, read references/protocol.md.

Platform Setup

Agent-run installation scripts live under scripts/ and should be executed from the repository root. Generated Native Messaging launchers live under native/ because the browser manifest points to them at runtime.

The installation process registers the Python host as a Chrome Native Messaging Host. This creates a JSON manifest file that tells Chromium browsers how to launch the host program using stdin/stdout.

First, you can diagnose the current state using:

python3 scripts/doctor.py --skip-live

First-Time Agent Installation Checklist

When this skill is used in a fresh workspace, the Agent SHOULD perform this checklist before trying browser automation:

1. Confirm the repository layout from the repository root:

   • extension/manifest.json
   • extension/service-worker.js
   • extension/approval.html
   • extension/approval.js
   • native/host.py
   • native/com.local.browser_agent_bridge.json
   • scripts/doctor.py
   • scripts/sync-skill-scripts.sh
   • the platform installer under scripts/
   • this skill at skills/browser-agent-bridge/SKILL.md
   • the bundled runtime clients under skills/browser-agent-bridge/scripts/ (generated by scripts/sync-skill-scripts.sh; gitignored)

2. Derive the stable extension ID from extension/manifest.json. The manifest contains a fixed key; compute the ID from that key instead of asking the user to copy it from chrome://extensions.

   python3 - <<'PY'
   

import base64 import hashlib import json from pathlib import Path manifest = json.loads(Path("extension/manifest.json").read_text()) pub_key_der = base64.b64decode(manifest["key"]) sha = hashlib.sha256(pub_key_der).hexdigest() extension_id = "".join(chr(int(char, 16) + 97) for char in sha[:32]) print(extension_id) PY

Expected stable ID: `lpemchcojepfkbgjgoehfknibdjjppig`.

3. Ask the user to load the unpacked extension manually if it is not already loaded. This step is only to register the extension in Chrome and confirm the extension ID; do not ask the user to click Start Bridge yet.
- Open `chrome://extensions`.
- Enable Developer mode.
- Load this repository's `extension/` directory.
- Reload the extension after any local extension source change.

4. Run the offline setup doctor:

```bash
python3 scripts/doctor.py --skip-live

Inspect these checks:

• repo.files: required project files exist, including approval popup files.
• repo.install_layout: agent-run installers are in scripts/; generated launchers are in native/, and no installer scripts leak into skills/.
• extension.manifest: MV3 manifest is valid.
• extension.permissions: unwanted permissions such as history are absent.
• native.manifest.installed: browser Native Messaging manifest exists.
• native.manifest.path: manifest path points to a runnable wrapper.
• native.manifest.origins: includes chrome-extension://lpemchcojepfkbgjgoehfknibdjjppig/.
• native.wrapper: wrapper loads token env file, pins extension ID, and launches native/host.py.
• auth.env_file: token file exists and is private on macOS/Linux.
• auth.env_token: token is available to the Agent or client.
• package.freshness: release extension directory is up to date when preparing distribution.

5. The skill bundles only the portable runtime RPC clients (rpc.sh, ws-rpc.js, browser_bridge_client.py) under skills/browser-agent-bridge/scripts/, so a skill copied to an agent's skills directory can still call the bridge. That directory is generated (gitignored); regenerate it before copying the skill out:

   scripts/sync-skill-scripts.sh
   

   Setup and diagnostic scripts (doctor.py, the install-native-host-* installers) require this repository and are always run from the repository's top-level scripts/, never bundled into the skill. When the repository is available, run all scripts from the repository root.

6. Before the bridge can be started, the Native Host manifest MUST be installed for the stable extension ID. If native manifest, wrapper, or token checks fail, run the platform installer with the stable extension ID. These commands write outside the repository, so a sandboxed Agent MUST request elevated permission before running them.

   macOS / Linux:

   ./scripts/install-native-host-unix.sh lpemchcojepfkbgjgoehfknibdjjppig
   

   Use --browser chromium|brave|edge|all when the user is not using Google Chrome.

   Windows:

   powershell -ExecutionPolicy Bypass -File .\scripts\install-native-host-win.ps1 lpemchcojepfkbgjgoehfknibdjjppig
   

7. After the Native Host manifest is installed or repaired, ask the user to reload the unpacked extension in chrome://extensions, open the side panel, accept the initial disclaimer/Chrome permission prompts, click Start Bridge, and keep runtime approval enabled unless the user explicitly disables it.

   Do not proceed to Start Bridge before installing the Native Host manifest. Chrome can only launch native/host.py through a registered Native Messaging manifest, and the manifest must allow the current extension origin, for example chrome-extension://lpemchcojepfkbgjgoehfknibdjjppig/.

8. Verify the local bridge:

   scripts/browser_bridge_client.py health
   scripts/browser_bridge_client.py rpc native.status
   scripts/browser_bridge_client.py rpc extension.info
   

   Check that /health is reachable, authRequired is handled with the generated token, and extensionReady is true. If health is unreachable and the side panel shows Stopped, ask the user to click Start Bridge.

9. Trigger one isolated session RPC to verify browser control behavior:

   scripts/browser_bridge_client.py rpc session.start '{"url":"https://example.com"}'
   

   The created tab should appear in an Agent-managed tab group.

10. If any check fails, do not proceed with page automation until the specific failing item is fixed. Prefer re-running python3 scripts/doctor.py --skip-live after each repair.

Install or Repair the Native Host

1. macOS / Linux (Unix) Setup

Run the unified installer script from the repository root:

# macOS/Linux, Google Chrome
./scripts/install-native-host-unix.sh <extension-id>

# macOS/Linux, specify a different Chromium browser or all of them
./scripts/install-native-host-unix.sh --browser chromium <extension-id>
./scripts/install-native-host-unix.sh --browser brave <extension-id>
./scripts/install-native-host-unix.sh --browser edge <extension-id>
./scripts/install-native-host-unix.sh --browser all <extension-id>

What this script does under the hood:

• Detects OS & Python: Identifies whether it is running on macOS (Darwin) or Linux, and finds python3 or python on the system $PATH.
• Generates Unix Launcher: On macOS, copies the native host runtime to ~/Library/Application Support/Browser Agent Bridge/ and writes host-wrapper.sh there so Chrome does not need to execute native host files from ~/Downloads. On Linux, writes native/host-wrapper.sh in the project directory. This launcher sources the token environment file, sets BROWSER_AGENT_BRIDGE_EXTENSION_ID to pin the extension, and spawns native/host.py with forwarded arguments.
• Secures Auth Token: If ~/.browser-agent-bridge.env does not exist, the script generates a 16-byte random token encoded as 32 hex characters, writes it, and runs chmod 600.
• Installs Manifest JSON: Writes the manifest JSON file into the user-specific Chrome/Chromium native messaging directory:
  • macOS Google Chrome: ~/Library/Application Support/Google/Chrome/NativeMessagingHosts/com.local.browser_agent_bridge.json
  • Linux Google Chrome: ~/.config/google-chrome/NativeMessagingHosts/com.local.browser_agent_bridge.json
  • The manifest defines the "path" pointing to the generated wrapper and "allowed_origins" pointing to chrome-extension://<extension-id>/.

Note: scripts/install-native-host-macos.sh is kept as a compatibility wrapper that forwards directly to the Unix script.

2. Windows Setup

Run the PowerShell installer script from the repository root:

powershell -ExecutionPolicy Bypass -File .\scripts\install-native-host-win.ps1 <extension-id>

What this script does under the hood:

• Locates Python: Searches the system PATH for the python or py executable.
• Generates Windows Launcher: Copies the native host runtime to %LOCALAPPDATA%\Browser Agent Bridge\ and creates host-wrapper.win.bat there. The launcher reads %USERPROFILE%\.browser-agent-bridge.env, sets the target extension ID, and runs Python with argument forwarding (%*).
• Secures Auth Token: Generates a 16-byte random token and saves it in %USERPROFILE%\.browser-agent-bridge.env.
• Installs Manifest JSON: Writes the manifest JSON to %LOCALAPPDATA%\Google\Chrome\NativeMessagingHosts\com.local.browser_agent_bridge.json, with "path" pointing to %LOCALAPPDATA%\Browser Agent Bridge\host-wrapper.win.bat.
• Registers via Registry: Adds a registry value under HKCU:\Software\Google\Chrome\NativeMessagingHosts\com.local.browser_agent_bridge pointing to the manifest file path. Installing in HKCU ensures that administrator privileges are not required.

Generated Token Files

• macOS/Linux: ~/.browser-agent-bridge.env
• Windows: %USERPROFILE%\.browser-agent-bridge.env

Automated Self-Setup / 自动安装与自愈机制

When initialized in a new user workspace, the Agent SHOULD proactively check the connection status. If a connection failure or diagnostic warning occurs, the Agent is authorized to automatically perform self-registration:

1. Run python3 scripts/doctor.py --skip-live to detect host configuration health.
2. If native.manifest.path or native wrapper diagnostics fail:
   • Identify the user's OS.
   • For macOS/Linux: check if directory has quarantine flags. If so, execute xattr -rd com.apple.quarantine <workspace-dir>. Then, run ./scripts/install-native-host-unix.sh lpemchcojepfkbgjgoehfknibdjjppig to register the native messaging host. Use --browser chromium|brave|edge|all when the user is not using Google Chrome.
   • For Windows: run powershell -ExecutionPolicy Bypass -File .\scripts\install-native-host-win.ps1 lpemchcojepfkbgjgoehfknibdjjppig.
3. Automatically load the generated token from ~/.browser-agent-bridge.env on macOS/Linux or %USERPROFILE%\.browser-agent-bridge.env on Windows into the environment or credentials and proceed with client operations.
4. Reload the unpacked extension in chrome://extensions, then open the extension side panel and ask the user to click Start Bridge so Chrome starts the native host.

Troubleshooting

If /health fails, the native host is not running. Native Messaging starts only after Chrome loads the extension and the user clicks Start Bridge in the side panel. If the user clicked Stop Bridge, auto-reconnect is paused until Start Bridge is clicked again.

If /health works and authRequired is true, make sure the agent process has the same BROWSER_AGENT_BRIDGE_TOKEN as the native host.

On macOS/Linux, the native wrapper loads ~/.browser-agent-bridge.env. On Windows, the batch wrapper loads %USERPROFILE%\.browser-agent-bridge.env. If token auth is enabled, the user's shell or agent process should load the same token before using scripts/rpc.sh or scripts/ws-rpc.js. The Python client auto-loads the default token file.

macOS Gatekeeper / Quarantine Block (macOS 隔离区限制)

If the project folder or native host zip was downloaded from a browser, macOS automatically tags it with the com.apple.quarantine attribute. When Chrome tries to launch the native host wrapper, macOS Gatekeeper blocks execution silently, and the bridge cannot connect.

• Fix: Run xattr -rd com.apple.quarantine <path-to-bridge-folder> to remove quarantine attributes from the entire folder.
• Reinstall: After clearing quarantine attributes, re-run ./scripts/install-native-host-unix.sh <extension-id> to ensure the path is registered correctly.

If /health works but extensionReady is false:

• Tell the user to load or reload the unpacked extension from the repo's extension/ directory.
• Tell the user that the extension ID is stabilized by extension/manifest.json; use lpemchcojepfkbgjgoehfknibdjjppig unless the manifest key changed.
• Tell the user to run the platform installer with the stable extension ID: scripts/install-native-host-unix.sh lpemchcojepfkbgjgoehfknibdjjppig on macOS/Linux or scripts/install-native-host-win.ps1 lpemchcojepfkbgjgoehfknibdjjppig on Windows.
• Tell the user to reload the extension again after installing the native manifest.

For a full setup check, run:

scripts/doctor.py

Doctor package.freshness warnings mean the release extension directory is older than the extension source files. This does not block local unpacked-extension development, but rebuild the release package before distributing.

If doctor reports new tools as "not exposed", reload the unpacked extension in chrome://extensions.

If extension.reload is already exposed, you can reload the extension through RPC:

scripts/browser_bridge_client.py rpc extension.reload

The extension ID is stabilized via a hardcoded key in manifest.json. The stable Extension ID is: lpemchcojepfkbgjgoehfknibdjjppig.

Operating Rules

• Do not use this skill for public web research; this bridge controls the user's local Chrome.
• Mandatory site-pattern preflight: Before operating on any website or domain with this bridge, first check runtime/site-patterns/ for prior experience. Experience filenames MUST match the site domain exactly, using lowercase ASCII/Punycode hostnames without protocol, path, query, port, or trailing dot, for example example.com.md, sub.example.com.md, or xn--fsqu00a.xn--0zwm56d.md. This keeps filenames portable across macOS, Windows, and Linux; avoid Unicode hostnames and mixed-case variants. Look for the most specific {domain}.md and then parent-domain files that match the target URL. If a matching file exists, read it and apply its selectors, waits, navigation notes, CSP guidance, and pitfalls before taking browser actions. If no matching file exists, explicitly proceed with fresh inspection and create/update the domain file later if reusable knowledge is learned.
• Page CSP (Content Security Policy) response headers are not stripped globally. Temporary CSP bypass defaults to enabled for new installs, but only adds a short-lived dynamic rule for the target origin when tabs.create, session.start, page.navigate, or page.executeJavaScript needs it. Check the current state with extension.getCspBypass; the toggle can only be changed by the user in the sidepanel UI. Pass bypassCSP:false to opt out for a call.
• When CSP bypass is enabled, prefer the bypass-capable path for analyzing and extracting web page content: navigate/create the target tab with the default bypass behavior, then use page.readText, page.accessibilityTree, or page.executeJavaScript as needed. This is the preferred path for pages whose scripts or injected analysis helpers may be blocked by CSP.
• Browser history and bookmark search are intentionally not supported; ask the user for a URL or use currently open tabs instead.
• Agent Tab Isolation: Browser tab reads and controls are limited to Agent-managed tab groups. Calls that target tabs or groups outside that boundary fail immediately instead of asking for approval. Create isolated work with tabs.create or session.start; both require tabs and tabGroups optional permissions and place new tabs in an Agent-managed group.
• Runtime Permission Approval: By default, sensitive non-tab operations (recording data access, downloads records, and policy.set) and sensitive tab operations inside the Agent boundary (screenshots/DOM snapshots, console/network logs, and similar methods) are intercepted by the extension and prompt the user in the sidepanel UI for approval (Allow once, Always for session, or Deny). Operations scoped to Agent-managed tab groups are allowed without an extra runtime approval prompt — except cookies.get, which reads httpOnly session tokens and always prompts even inside the Agent boundary.
  • If the sidepanel is closed when making a sensitive call, the extension sends a Chrome notification and opens an extension approval popup window. The user can approve or deny from that popup.
  • If the user denies the request, the call will fail. Respect this choice and do not retry repeatedly; instead, explain the limitation to the user or ask for the information directly.
• Domain Experience Accumulation: At the end of every site-specific analysis, extraction, or automation task, actively decide whether the run revealed reusable site knowledge. Before writing a summary, check runtime/site-patterns/ for existing experience for the same site domain. If it exists, merge the new knowledge into that file instead of creating a duplicate. If it does not exist, create runtime/site-patterns/{domain}.md, where {domain} exactly matches the lowercase ASCII/Punycode site hostname without protocol, path, query, port, or trailing dot.
  • Save only reusable operational knowledge: stable selectors, reliable wait conditions, iframe/shadow DOM notes, CSP-bypass needs, login walls, pop-up handling, pagination/list/detail patterns, extraction scripts, known failure modes, and preferred bridge methods.
  • Do not save private user data, page contents copied from a session, credentials, personal account details, or one-off observations that are unlikely to help future runs.
  • Use concise sections such as Selectors, Wait Conditions, Extraction, Navigation, CSP, and Pitfalls; include the date when behavior may be time-sensitive.
  • Record each flow separately: when a site supports distinct flows or functions (e.g. login, search, checkout, export), give each one its own section. Do not blend steps from different flows into a single procedure.
  • For each flow, record only the durable shape of the procedure — the ordered steps, and for each step:
    • the content to obtain, or the interactive component to use;
    • where that component sits on the page as a region/orientation, not pixel coordinates — e.g. "top-right of the header", "first row of the results list", "inside the cookie-consent dialog", "footer";
    • what operation was performed on it (click / fill / select / press / hover);
    • how to verify the step succeeded — an observable post-condition such as a URL change, an element/text that appears or disappears, or a whatChanged signal.
  • Do not record volatile, run-specific values: exact coordinates, the specific text typed, query strings, ids/tokens, timestamps, or anything that changes between runs. Capture the stable selector / role / label that identifies the component and the kind of input it expects, not the input itself.
• Do not execute high-risk actions such as purchases, sending messages, deleting data, changing account settings, or submitting sensitive forms unless the user explicitly asked for that exact action.
• Prefer read-only methods first: tabs.list, page.readText, page.accessibilityTree (prefer "format": "compact" to get a token-efficient text snapshot with f{frameId}:{ref} tags), page.screenshot.
• Prefer session.start for multi-step tasks that should stay isolated in a Chrome tab group. Use session.createTab for new tabs inside the session and session.closeTab to close one managed tab while keeping session metadata clean. session.addTab only accepts tabs that are already in an Agent-managed group.
• Check policy.get before operating on sensitive domains or using high-risk methods; use policy.set only when the user asks to change local allow/block rules.
• Use page.executeJavaScript only when read-only methods are insufficient or when the user explicitly wants page scripting.
• Act by Ref: When calling page.accessibilityTree with "format": "compact", the returned nodes are labeled with a frame-aware identifier: f{frameId}:{ref} (e.g. f0:ref_1 or f7:ref_2). You can act on these elements directly by passing the prefix-tagged ref to the ref-based actions: locator.clickRef, locator.fillRef (focus + replace text), locator.pressRef (focus + press key), locator.hoverRef, and locator.selectOptionRef (select items in <select> elements). The bridge automatically parses the frame prefix and resolves the correct frame target without requiring coordinate translations.
• Prefer ref-based actions over DOM selector methods (dom.click, dom.type) or coordinate-based methods (computer.click), as refs are faster and immune to layout shifts.
• If ref-based actions are not applicable, prefer dom.query, dom.click, dom.type, dom.select, dom.hover, and dom.scroll for ordinary page controls before falling back to viewport coordinates.
• Use frameSelector with dom.* and page wait methods for same-origin iframes. Cross-origin iframes are not accessible through DOM methods.
• Use page.waitForLoad, page.waitForSelector, or page.waitForText after navigation or UI actions instead of sleeping blindly.
• Use computer.* methods for visible UI automation. Coordinates are CSS viewport coordinates. computer.click, computer.drag, and computer.hover do not display the page dot/label unless showIndicator:true is passed. Use computer.hover for cursor movements, and computer.key with combinations (e.g. "Control+a", "Meta+c") for keyboard shortcuts.
• If a call returns a restricted-page/debugger error, explain that Chrome blocks extension automation on pages such as chrome://, Chrome Web Store, or pages controlled by another debugger.

Common Workflows

Inspect Current Page

1. Use session.start for a new isolated Agent tab group, or session.get for an existing Agent session.
2. Determine the page domain and check runtime/site-patterns/ for a matching prior-experience file. Read and follow it before inspecting further.
3. Call extension.getCspBypass; if enabled, prefer the default CSP-bypass path while analyzing or extracting page content.
4. page.readText for visible text.
5. Call page.accessibilityTree with "format": "compact" to obtain a concise, token-saving text snapshot of interactive elements with inlined f{frameId}:{ref} tags.
6. Use page.executeJavaScript for structured extraction when read-only text/tree methods are insufficient.
7. page.screenshot when visual confirmation matters.
8. After finishing site-specific work, update runtime/site-patterns/{domain}.md if you learned reusable selectors, waits, extraction logic, navigation patterns, CSP needs, or pitfalls.

Interact (Click, Type, Hover, Select)

1. Determine the site domain and check runtime/site-patterns/ for a matching prior-experience file. Read and use it before interacting.
2. Read the compact accessibility tree (page.accessibilityTree with format: "compact") or screenshot first to identify target ref tags.
3. Perform the action directly by passing the prefix-tagged ref (e.g. f0:ref_1) to the appropriate ref-based action:
   • locator.clickRef to click.
   • locator.fillRef to focus and replace input values.
   • locator.pressRef to focus and press keyboard keys.
   • locator.hoverRef to move the mouse cursor over the element.
   • locator.selectOptionRef to choose options within a <select> element.
4. If ref-based actions are not viable, try dom.query to find stable selectors and call dom.click, dom.type, dom.select, dom.hover, or dom.scroll.
5. Call page.waitForSelector or page.waitForText when the action should change page state.
6. Fall back to computer.click, computer.type, computer.key (supporting combination shortcuts like "Control+a"), computer.scroll, or computer.hover when selector targeting is not enough.
7. Read the page again to verify the result.

Debug A Page

1. Call console.read to attach CDP and collect console events.
2. Call network.read to attach CDP and collect network events.
3. Reproduce or navigate as needed.
4. Call the same read methods again.

Work In An Isolated Session

1. Call session.start with a descriptive name and optional URL.
2. Use the returned mainTabId for page and computer methods.
3. Call session.get when you need current managed tabs.
4. Call session.stop when done; use closeTabs: true only if the user expects tabs to close.

Record A Workflow

Recordings persist in Chrome local extension storage with privacy defaults: 24-hour retention, 500 actions per recording, screenshots off by default, and typed text/value fields redacted unless includeText:true is passed.

1. Call recording.start with a tabId or groupId; keep captureScreenshots false unless visual replay matters and use includeText:true only when the user explicitly needs typed text captured.
2. Perform page.navigate, dom.*, and computer.* actions normally.
3. Call recording.stop.
4. Call recording.export; use download: true when the user wants a JSON artifact saved through Chrome.
5. Call recording.clear after export for large or screenshot-heavy recordings.

Repository Paths

From this project's root:

• Extension: extension/
• Native host: native/host.py
• Native manifest template: native/com.local.browser_agent_bridge.json
• Generated Unix launcher: native/host-wrapper.sh
• Generated Windows launcher: native/host-wrapper.win.bat
• macOS/Linux installer: scripts/install-native-host-unix.sh
• Legacy macOS installer entrypoint: scripts/install-native-host-macos.sh
• Windows installer: scripts/install-native-host-win.ps1
• RPC helper: scripts/rpc.sh
• WebSocket helper: scripts/ws-rpc.js
• Python client: scripts/browser_bridge_client.py
• Doctor: scripts/doctor.py
• Release builder: scripts/build-release.sh
• Protocol reference: skills/browser-agent-bridge/references/protocol.md