| name | native_window_control |
| description | When the user needs to operate an already-open native desktop app window (not a browser): list windows, get control tree, click buttons/tabs, or type text into controls. Windows only; requires Python and pywinauto (pip install pywinauto). | 操作已打开的原生桌面窗口（非浏览器）：列举窗口、控件树、点击或输入；仅 Windows，需 pywinauto。 |
| metadata | {"builtin_skill_version":"1.0","copaw":{"emoji":"🪟","requires":{"platform":"win32"}}} |
Native Window Control (Windows UI Automation)
For already-open native desktop app windows: list windows, snapshot controls, click buttons/tabs, type into edit controls. Unlike browser tooling, this operates native desktop windows (exe, desktop apps), not web pages.
When to use
- User says: "Click the Start button in XX app", "Click the second tab in XX window", "Type 123 into the input in XX window"
- User wants the agent to operate an already-open desktop app UI (exe window with tabs/buttons)
- Combine with screenshot tool: capture the window first, then use this skill to click or type
Prerequisites
- OS: Windows only.
- Dependencies:
pip install pywinauto. Run the script from the skill's scripts/ directory with Python.
How to run (script in this skill)
This skill provides a Python script scripts/native_window.py. Run it from the skill directory (or with the script path pointing at this skill's scripts folder). Working directory should be the skill root or the folder containing native_window.py.
Usage:
python scripts/native_window.py <action> [window_title] [ref] [text]
-
list_windows — List visible windows (title, PID). No extra args.
Example: python scripts/native_window.py list_windows
-
snapshot — Get control tree for a window (substring match on title).
Example: python scripts/native_window.py snapshot "Notepad"
Returns JSON with controls array; each has ref, control_type, name. Use ref for click/type_text.
-
click — Click a control by window title substring and ref.
Example: python scripts/native_window.py click "Notepad" "2"
-
type_text — Type into a control.
Example: python scripts/native_window.py type_text "Notepad" "1" "hello"
Recommended flow
- list_windows → identify the target window title.
- snapshot with that title substring → get controls and refs.
- click or type_text with the same title and the chosen ref (and text for type_text).
Notes
- Window title matching is substring, first match. Use a more specific substring or list_windows if multiple windows match.
- Some legacy apps do not support UI Automation; those windows cannot be snapshotted or controlled.
- You can take a screenshot first so the user can confirm the target window is open and in front.
