background-computer-use

name	background-computer-use
description	Launch and use the local BackgroundComputerUse macOS runtime through its self-documenting loopback API. Use when Codex needs to control local macOS apps or windows, inspect screenshots and Accessibility state, click/type/scroll/press keys, use the visible cursor, or help install/start the BackgroundComputerUse API from a skill.

Background Computer Use

Use this skill to start or connect to the local macOS BackgroundComputerUse runtime, then rely on the runtime's own documentation endpoints instead of memorizing route schemas.

Workflow

1. Start or discover the runtime:

   bash "$SKILL_DIR/scripts/ensure-runtime.sh"
   

   If $SKILL_DIR is not already set by the host, derive it from this skill folder path before running scripts.

2. Read the runtime manifest from:

   $TMPDIR/background-computer-use/runtime-manifest.json
   

   Never assume a fixed port.

3. Call GET /v1/bootstrap.

   • If instructions.ready is false, report the returned permission steps to the user.
   • Do not continue with actions until Accessibility and Screen Recording are ready.

4. Call GET /v1/routes.

   • Treat this as the source of truth for available routes, request fields, response fields, examples, and error codes.
   • Do not assume browser routes exist; use them only when /v1/routes advertises them.

5. For visual tasks, request screenshots with imageMode: "path" and inspect the returned image files when useful.

6. For action routes, reuse stateToken from the state you inspected. Reuse the same cursor.id when the user wants one continuous visible cursor.

Helpers

• scripts/ensure-runtime.sh: find, install, launch, and bootstrap the runtime.
• scripts/install-runtime.sh: install BackgroundComputerUse.app from an app zip or release URL.
• scripts/bcu-request.py: call runtime endpoints from the manifest base URL.

Examples:

python3 "$SKILL_DIR/scripts/bcu-request.py" GET /v1/bootstrap
python3 "$SKILL_DIR/scripts/bcu-request.py" GET /v1/routes
python3 "$SKILL_DIR/scripts/bcu-request.py" POST /v1/list_apps '{}'

Local Development

When working from a source checkout instead of an installed release:

BCU_SOURCE_DIR=/path/to/background-computer-use bash "$SKILL_DIR/scripts/ensure-runtime.sh"

That runs the repo's script/start.sh, which builds, signs, installs, launches, and bootstraps the app.

More Detail

Read references/runtime.md only when you need install modes, release packaging notes, or the permission/debugging checklist.