Menu
Control desktop GUI applications via screenshots and automated actions.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
| name | gui-agent |
| description | Control desktop GUI applications via screenshots and automated actions. |
| homepage | https://github.com/bytedance/UI-TARS |
| metadata | {"syll":{"emoji":"🖥️","requires":{"bins":[],"env":["SYLL__TOOLS__GUI__ENABLED"]}}} |
You can interact with desktop applications using the gui_action tool.
When the user asks you to perform a GUI task:
gui_action with a clear instructionUser: "Open Chrome and search for 'Syll'"
You should call:
gui_action(instruction="Open Chrome browser, click on the address bar, type 'Syll' and press Enter")
Never perform destructive actions without user confirmation:
Always verify the current screen state before acting
Stop immediately if the screen shows unexpected content (login pages with sensitive data, etc.)
Report clearly what actions were taken and their results
| Action | Description | Example |
|---|---|---|
click | Left click at position | click(point='<point>500 300</point>') |
right_click | Right click | right_click(point='<point>500 300</point>') |
double_click | Double click | double_click(point='<point>500 300</point>') |
drag | Drag from A to B | drag(start='<point>100 100</point>', end='<point>200 200</point>') |
type | Type text | type(content='hello world') |
hotkey | Key combination | hotkey(key='ctrl+c') |
scroll | Scroll up/down | scroll(point='<point>500 300</point>', direction='down', amount=3) |
wait | Pause | wait(seconds=2) |
finished | Task complete | finished(content='Done') |
Enable in config.json:
{
"tools": {
"gui": {
"enabled": true,
"ui_tars": {
"api_base": "http://localhost:8000/v1",
"api_key": "your-key",
"model": "ui-tars"
},
"max_steps": 15,
"confirm_destructive": true
}
}
}
Clean up a voice recording — remove hiss, hum, background noise, and harsh sibilance — by driving the real Adobe Audition app on a macOS host. Triggers: clean up audio, clean up the voice, remove hiss, remove hum, remove background noise, denoise, noise reduction, de-ess, reduce sibilance, repair voice, fix the recording. 中文触发词：降噪、去底噪、去杂音、去噪音、人声清理、清理录音、修复人声、去齿音、去咝声。
Remove the background from an image and produce a transparent PNG by driving the real Adobe Photoshop app on a macOS host. Triggers: remove background, cut out, knock out the subject, isolate the subject, make a transparent PNG, transparent background, delete background. 中文触发词：抠图、扣图、去背景、去掉背景、扣出主体、透明背景、做透明 PNG。
Schedule reminders and recurring tasks.
Find a file on the local filesystem, show previews, and send it to the user after confirmation. Use when the user asks to find, locate, retrieve, or send a file.
Interact with GitHub using the `gh` CLI. Use `gh issue`, `gh pr`, `gh run`, and `gh api` for issues, PRs, CI runs, and advanced queries.
Simulate an overnight local mail summary when there is no real local mailbox integration or recorded GUI workflow available.