| name | vnc |
| description | Control a remote desktop via VNC. Use when user asks to interact with a remote computer screen — click, type, screenshot, scroll, or connect/disconnect VNC sessions. |
| allowed-tools | Bash Read |
VNC Computer-Use Tool
You have access to a VNC computer-use CLI tool (vnc) that lets you control a remote desktop. Use it to interact with GUIs — click buttons, type text, take screenshots, scroll, and more.
Installation
If the vnc command is not available, install it first:
pip install vnc-computer-use
Quick Reference
Connect / Disconnect
vnc connect <host>::<port>
vnc connect <host>::<port> --password <pw>
vnc connect <host>::<port> --username <user> --password <pw>
vnc connect <host>::<port> --password <pw> --legacy-auth
vnc disconnect
Screenshots
vnc get_screenshot -o /tmp/screen.png
vnc get_screenshot
Screenshot response includes image_width and image_height:
{"ok": true, "path": "/tmp/screen.png", "image_width": 1568, "image_height": 882}
Mouse Actions
vnc left_click <x> <y>
vnc right_click <x> <y>
vnc double_click <x> <y>
vnc middle_click <x> <y>
vnc mouse_move <x> <y>
vnc left_click_drag <x> <y>
vnc scroll <direction> <x> <y>
Keyboard Actions
vnc key <key>
vnc type "<text>"
Info
vnc get_screen_size
vnc get_cursor_position
vnc status
Multi-Session
vnc connect host1::5900 --session work
vnc connect host2::5900 --session home
vnc -s work get_screenshot -o /tmp/work.png
vnc -s home left_click 100 200
vnc disconnect --session work
Coordinate System
IMPORTANT: All coordinates are in image space, not raw screen space.
- Screenshots are automatically downscaled to fit Claude's vision limits (max 1568px long edge).
- The
image_width and image_height in the screenshot response tell you the coordinate space.
- When you click/move using coordinates, the tool automatically maps them back to the real screen.
get_cursor_position and get_screen_size also return values in image space.
This means: use the coordinates as you see them in the screenshot image directly. No manual scaling needed.
Cursor Crosshair
Screenshots include a red crosshair marking the current cursor position. Use it to:
- Verify your last click landed where you intended.
- If the crosshair is far from your target, adjust coordinates proportionally to the distance — use large adjustments first, then refine.
- Consider the display dimensions when estimating positions (e.g. if something is 90% to the bottom, the y-coordinate should be ~90% of
image_height).
Usage Guidelines
- For macOS mini-machine legacy VNC passwords, use
--legacy-auth when connecting; otherwise vncdotool may choose Apple/DH auth and hang or fail.
- Always take a screenshot first before clicking — you need to see where elements are on screen.
- Prefer keyboard shortcuts over clicking when possible (faster and more reliable).
- Click center of elements — don't click edges of buttons or icons.
- Wait and re-screenshot if an action doesn't seem to take effect. GUI apps may need time to respond (a 1-second delay is built into screenshots).
- All output is JSON — parse it to check for
"ok": true or "error" fields.
- Check the crosshair after clicking to verify accuracy. If it missed, adjust.
- Coordinates outside screen bounds will return an error.
Workflow Example
A typical interaction looks like:
vnc connect <host>::<port> — establish connectionvnc get_screenshot -o /tmp/screen.png — see what's on screen (also gives you image dimensions)- Read the screenshot to identify UI elements and their coordinates
vnc left_click <x> <y> — click on the target elementvnc get_screenshot -o /tmp/screen.png — verify the result, check crosshair position- If crosshair missed the target, adjust coordinates and retry
- Repeat steps 3-6 as needed
vnc disconnect — when done
