| name | word-web-visual |
| description | Visual-first interaction patterns for Word for the web (office.com / Word Online) when the editor surface is iframe/canvas and DOM/A11y refs are unreliable. Use to focus the document and type/paste using only user-like inputs (mouse move/click, scroll, keyboard shortcuts) and screenshots for verification. |
| allowed-tools | Bash(agent-browser:*) |
Word Web Visual
Goal
Interact with Word for the web like a human: click to focus the page, type/paste, scroll, and recover from mis-focus. Avoid relying on snapshot refs for the editor surface.
Non-negotiables
- Always include
--cdp 9222 on every agent-browser command.
- Prefer low-level mouse/keyboard (
agent-browser mouse ..., agent-browser press ...) over element refs.
- Verify with screenshots; treat “visible change in the doc” as the only success criterion.
Quick recipe: focus + type
- Set a known viewport (reduces coordinate drift):
agent-browser --cdp 9222 set viewport 1440 900
- Bring the Word tab to the front (reuse if already open):
agent-browser --cdp 9222 tab
agent-browser --cdp 9222 tab 2
- Focus the document canvas by clicking a few “safe” points.
Try in order until typing works:
- Center-ish of the page: (720, 450)
- Slightly lower (often inside page body): (720, 520)
- Left margin inside page: (420, 520)
agent-browser --cdp 9222 mouse move 720 450
agent-browser --cdp 9222 mouse down left
agent-browser --cdp 9222 mouse up left
- Type (works even when there is no DOM textbox):
agent-browser --cdp 9222 press H
agent-browser --cdp 9222 press e
agent-browser --cdp 9222 press l
agent-browser --cdp 9222 press l
agent-browser --cdp 9222 press o
- Screenshot to verify text appeared:
agent-browser --cdp 9222 screenshot
Prefer paste for longer text
For long text, do NOT send one character at a time. Use clipboard + paste.
On Windows PowerShell:
Set-Clipboard -Value @"
Your multi-line text here.
"@
agent-browser --cdp 9222 press Control+v
Then verify with a screenshot.
Recovery / troubleshooting loop
If typing doesn’t show up:
- Dismiss popovers:
agent-browser --cdp 9222 press Escape
-
Click again (use a different point) and type a short marker like zzz.
-
If you typed into the wrong surface, try undo:
agent-browser --cdp 9222 press Control+z
- Nudge scroll and retry focus:
agent-browser --cdp 9222 mouse wheel 300
agent-browser --cdp 9222 mouse wheel -150
- If Word is embedded in an iframe and clicks seem ignored, do NOT fight frames. Keep using viewport-coordinate clicks + screenshots.
Drag / selection (basic)
To select a region, use click-drag:
agent-browser --cdp 9222 mouse move 450 520
agent-browser --cdp 9222 mouse down left
agent-browser --cdp 9222 mouse move 850 520
agent-browser --cdp 9222 mouse up left
Then apply formatting shortcuts:
agent-browser --cdp 9222 press Control+b
agent-browser --cdp 9222 press Control+i
Notes
- The Word editing surface is frequently virtualized and/or nested in iframes;
snapshot -i may not expose a usable textbox. That’s expected.
- Keep iterating: click, type/paste, screenshot, retry.
