name	desktop-act
description	Computer-use primitives for the X11 desktop on this dev box. Use when a task needs to drive a GUI app, browser, or interact with anything visual that the parent Claude session needs to see and act on.

desktop-act

This plugin exposes computer-use primitives via the mcp__desktop-act__* tools plus a one-shot act(goal) that runs an autonomous loop via OAuth (no API key).

Architecture

Parent Claude session is the "brain" — calls primitives, Reads screenshots, decides next action, calls next primitive. Natural streaming reasoning.
MCP server provides primitives: screenshot, click, type_text, keypress, scroll, list_windows, launch_app.
High-level act(goal) spawns a sub-session via claude-agent-sdk for hands-off goals — but the streaming-friendly path is calling primitives directly.

Watching the desktop

Connect via noVNC at: http://:6081 That bridges to Xvnc :1. Whatever the primitives do is visible there in real time.

When to use

Task	Approach
Browser dashboard with auth (Stripe/Vercel/AWS/etc)	Prefer `mcp__kapture__*` (Sam's real Chrome)
Local app you need to click/type into	`desktop-act` primitives
Generic public site automation	`mcp__playwright__*`
Perf/console/Lighthouse	`mcp__chrome-devtools__*`
"Just do it" hands-off	`mcp__desktop-act__act(goal)`

See ~/.claude/skills-library/browser-automation.md for the full decision tree.

name	desktop-act
description	Computer-use primitives for the X11 desktop on this dev box. Use when a task needs to drive a GUI app, browser, or interact with anything visual that the parent Claude session needs to see and act on.