Walk the user through turning a Mac into a computer-use node. macOS-only; needs a display and user-granted permissions you can't grant for them. Produce copy-pasteable steps; substitute <host>, <port>, <secret>.

Install cliclick (brew install cliclick) — used for input synthesis (click/type/key/move).
Ensure a DISPLAY exists. A headless Mac has no framebuffer and capture fails with "could not create image from display." Options: attach an HDMI dummy plug, OR create a virtual display (e.g. BetterDisplay) — which needs a one-time GUI session (Screen Sharing).
Grant permissions to the runtime's node binary in System Settings → Privacy & Security:
- Screen Recording (for screenshots)
- Accessibility (for mouse/keyboard via cliclick) These require a GUI session to approve; on a headless box, do it once over Screen Sharing.
Run openagi computer-server --token <secret> --port <port>. Persist via launchd, and put /opt/homebrew/bin on the job's PATH so cliclick is found.
On the MAIN: set OPENAGI_COMPUTER_USE=1, OPENAGI_COMPUTER_NODE=http://<host>:<port>, OPENAGI_COMPUTER_NODE_TOKEN=<secret>, then restart. The computer_* tools now execute on the node.

Notes to pass along:

Screenshots are auto-downscaled to ~OPENAGI_COMPUTER_SCALE_WIDTH (default 1280) and click coordinates are mapped to the display's LOGICAL points (Retina-correct) — the model works in the returned image's space.
scroll is not supported (cliclick has no scroll primitive); it returns an honest error.
Verify via the node's /screenshot HTTP endpoint (it runs in the GUI session) — NOT a plain SSH shell, which has no WindowServer access and will report "could not create image from display" even when a display exists.
The screen must be UNLOCKED and awake — a locked/asleep display captures black. Disable auto-lock / display sleep on a dedicated node.

User asked: {{input}}

Más de este repositorio

mismo repositorio

setup-proactive-outreach

Spshulem/openAGI

Guide configuring proactive outreach so the main reaches out (digest + live decisions) and the Mac app surfaces it with notifications, an overlay list, and inline buttons.

2026-06-172

deploy-update-node

Spshulem/openAGI

Roll out new openAGI code to the main and to Mac nodes (iMessage / computer-use), and restart them safely.

2026-06-152

diagnose-agent-tool-errors

Spshulem/openAGI

Troubleshoot opaque model/tool failures and runaway budget — and their usual root causes.

2026-06-152

diagnose-no-imessage-replies

Spshulem/openAGI

Troubleshoot why the agent stopped replying to iMessages, in priority order.

2026-06-152

connect-mcp

Spshulem/openAGI

Connect an MCP server (catalog or custom URL/stdio) and make its tools usable — including completing OAuth on a headless host.

2026-06-152

drive-computer-use

Spshulem/openAGI

Operate a connected computer-use node to accomplish an on-screen task — screenshot, reason, act in a loop.

2026-06-152

name	setup-computer-use-node
description	Guide setting up a Mac as a computer-use node so the agent can see the screen and control mouse/keyboard.

Install cliclick (brew install cliclick) — used for input synthesis (click/type/key/move).
Ensure a DISPLAY exists. A headless Mac has no framebuffer and capture fails with "could not create image from display." Options: attach an HDMI dummy plug, OR create a virtual display (e.g. BetterDisplay) — which needs a one-time GUI session (Screen Sharing).
Grant permissions to the runtime's node binary in System Settings → Privacy & Security:
- Screen Recording (for screenshots)
- Accessibility (for mouse/keyboard via cliclick) These require a GUI session to approve; on a headless box, do it once over Screen Sharing.
Run openagi computer-server --token <secret> --port <port>. Persist via launchd, and put /opt/homebrew/bin on the job's PATH so cliclick is found.
On the MAIN: set OPENAGI_COMPUTER_USE=1, OPENAGI_COMPUTER_NODE=http://<host>:<port>, OPENAGI_COMPUTER_NODE_TOKEN=<secret>, then restart. The computer_* tools now execute on the node.

Notes to pass along:

Screenshots are auto-downscaled to ~OPENAGI_COMPUTER_SCALE_WIDTH (default 1280) and click coordinates are mapped to the display's LOGICAL points (Retina-correct) — the model works in the returned image's space.
scroll is not supported (cliclick has no scroll primitive); it returns an honest error.
Verify via the node's /screenshot HTTP endpoint (it runs in the GUI session) — NOT a plain SSH shell, which has no WindowServer access and will report "could not create image from display" even when a display exists.
The screen must be UNLOCKED and awake — a locked/asleep display captures black. Disable auto-lock / display sleep on a dedicated node.

User asked: {{input}}