// Mouse, keyboard, and scroll primitives for cua_click / cua_type / cua_key / cua_scroll. Reference for action shapes, modifier keys, key chord syntax, and screenshot-driven targeting workflows.
| name | cua-control |
| description | Mouse, keyboard, and scroll primitives for cua_click / cua_type / cua_key / cua_scroll. Reference for action shapes, modifier keys, key chord syntax, and screenshot-driven targeting workflows. |
These tools target the active Cua sandbox (or localhost when in that mode).
Always take a cua_screenshot first when you need coordinates; never
guess pixel positions blindly.
Coordinates are in pixels from the top-left of the target display. The default sandbox display is 1024x768 (XGA) which matches Anthropic's recommended computer-use resolution. Cua handles Retina/HiDPI scaling internally on macOS.
cua_clickcua_click({ x: 320, y: 180 }) // left click
cua_click({ x: 320, y: 180, button: "right" }) // right click
cua_click({ x: 320, y: 180, clicks: 2 }) // double click
Supported buttons: left, right, middle.
cua_typecua_type({ text: "hello world" })
Types one character at a time on the underlying surface. Newlines in the text are typed as Return.
cua_keyPress a single chord or a sequence of chords:
cua_key({ keys: "Return" })
cua_key({ keys: "ctrl+s" })
cua_key({ keys: ["cmd+space", "Return"] }) // Spotlight + Enter
Common chord names: Return, Escape, Tab, BackSpace, Delete,
Up, Down, Left, Right, Page_Up, Page_Down, Home, End,
F1 .. F12, ctrl+a, ctrl+c, ctrl+v, cmd+s (macOS), super+l
(Linux).
cua_scrollcua_scroll({ x: 400, y: 300, dy: -5 }) // scroll down
cua_scroll({ x: 400, y: 300, dy: 5 }) // scroll up
cua_scroll({ x: 400, y: 300, dx: 3 }) // scroll right
Use dx / dy wheel deltas. Positive dx means right; negative dy
means down. scrollX / scrollY are still accepted aliases.
The canonical pattern:
1. cua_screenshot() // see the current state
2. <find target visually> // identify x, y of the element
3. cua_click({ x, y }) / cua_type
4. cua_screenshot() // verify the change
5. repeat
Take a fresh screenshot after every state-changing action. The model's internal memory of the screen is unreliable after typing or clicking.
Drive the user's host machine directly through Cua, with no sandbox. Maps to Cua's Localhost API. Higher risk - the agent can click, type, and run shell commands on the real desktop. Use only when the user explicitly opts in and read the safety section first. Triggers - "control my computer", "click here", "use my real Chrome", "no sandbox", "내 컴퓨터 자동화".
pi-cua-integration is a Pi/Senpi extension that bridges Cua (trycua/cua) for computer-use sandboxes and host control. MUST USE when the user asks to drive a desktop UI, take screenshots, run a "computer use" task, control a sandbox, or invoke any cua_* tool. Triggers - "cua", "computer use", "sandbox", "screenshot the desktop", "click the screen", "take over the browser", "샌드박스 켜줘", "스크린샷 찍어", "내 컴퓨터에서 자동화".
Use Cua's cloud sandboxes via cua.ai. Requires CUA_API_KEY. Cloud sandboxes are isolated VMs run by trycua with billing per usage. Use when the user wants no local Docker/QEMU, wants Windows/macOS guests without Apple Silicon, or explicitly asks for cloud.
Run Cua sandboxes locally - no API key required. Covers Docker (XFCE/Kasm) for Linux containers, QEMU for full VMs, Lume for macOS guests on Apple Silicon, and Tart for Apple VZ VMs. Use when the user wants a local sandbox, wants to keep things off the cloud, or asks "spin up an Ubuntu desktop", "open a sandbox VM", "let me try this safely first".