word-web-visual

name	word-web-visual
description	Visual-first interaction patterns for Word for the web (office.com / Word Online) when the editor surface is iframe/canvas and DOM/A11y refs are unreliable. Use to focus the document and type/paste using only user-like inputs (mouse move/click, scroll, keyboard shortcuts) and screenshots for verification.
allowed-tools	Bash(agent-browser:*)

Word Web Visual

Goal

Interact with Word for the web like a human: click to focus the page, type/paste, scroll, and recover from mis-focus. Avoid relying on snapshot refs for the editor surface.

Non-negotiables

• Always include --cdp 9222 on every agent-browser command.
• Prefer low-level mouse/keyboard (agent-browser mouse ..., agent-browser press ...) over element refs.
• Verify with screenshots; treat “visible change in the doc” as the only success criterion.

Quick recipe: focus + type

1. Set a known viewport (reduces coordinate drift):

agent-browser --cdp 9222 set viewport 1440 900

2. Bring the Word tab to the front (reuse if already open):

agent-browser --cdp 9222 tab
agent-browser --cdp 9222 tab 2

3. Focus the document canvas by clicking a few “safe” points.

Try in order until typing works:

• Center-ish of the page: (720, 450)
• Slightly lower (often inside page body): (720, 520)
• Left margin inside page: (420, 520)

agent-browser --cdp 9222 mouse move 720 450
agent-browser --cdp 9222 mouse down left
agent-browser --cdp 9222 mouse up left

4. Type (works even when there is no DOM textbox):

agent-browser --cdp 9222 press H
agent-browser --cdp 9222 press e
agent-browser --cdp 9222 press l
agent-browser --cdp 9222 press l
agent-browser --cdp 9222 press o

5. Screenshot to verify text appeared:

agent-browser --cdp 9222 screenshot

Prefer paste for longer text

For long text, do NOT send one character at a time. Use clipboard + paste.

On Windows PowerShell:

Set-Clipboard -Value @"
Your multi-line text here.
"@

agent-browser --cdp 9222 press Control+v

Then verify with a screenshot.

Recovery / troubleshooting loop

If typing doesn’t show up:

1. Dismiss popovers:

agent-browser --cdp 9222 press Escape

2. Click again (use a different point) and type a short marker like zzz.

3. If you typed into the wrong surface, try undo:

agent-browser --cdp 9222 press Control+z

4. Nudge scroll and retry focus:

agent-browser --cdp 9222 mouse wheel 300
agent-browser --cdp 9222 mouse wheel -150

5. If Word is embedded in an iframe and clicks seem ignored, do NOT fight frames. Keep using viewport-coordinate clicks + screenshots.

Drag / selection (basic)

To select a region, use click-drag:

agent-browser --cdp 9222 mouse move 450 520
agent-browser --cdp 9222 mouse down left
agent-browser --cdp 9222 mouse move 850 520
agent-browser --cdp 9222 mouse up left

Then apply formatting shortcuts:

agent-browser --cdp 9222 press Control+b
agent-browser --cdp 9222 press Control+i

Notes

• The Word editing surface is frequently virtualized and/or nested in iframes; snapshot -i may not expose a usable textbox. That’s expected.
• Keep iterating: click, type/paste, screenshot, retry.