windows-chat-verification

# Windows Chat Verification Use this skill when the task is to verify a real Windows chat application through DesktopManager, especially IntelligenceX Chat and similar WebView2 or Chromium-style shells. ## Golden Path 1. Make sure the DesktopManager MCP server is available through this plugin. - The plugin starts `DesktopManager` through `./scripts/start-desktopmanager-mcp.ps1`. - Build `DesktopManager.Cli` first if the repo has not produced a CLI host yet. - Keep the default server read-only unless the verification needs clicks, typing, or layout changes. 2. Enable only the minimum mutation scope you need. - Set `DESKTOPMANAGER_MCP_ALLOW_MUTATIONS=1` before starting the MCP client session when the workflow needs live interaction. - Use `DESKTOPMANAGER_MCP_ALLOW_PROCESS=IntelligenceX.Chat.App;DesktopManager.TestApp` or another narrow process list when you can. - Set `DESKTOPMANAGER_MCP_ALLOW_FOREGROUND_INPUT=1` only when a zero-handle UIA editor really needs focused fallback text or key input. 3. Start with inspection, not clicks. - Read `desktop://windows/visible`, `desktop://windows/active`, and `desktop://monitors`. - Use `list_windows`, `get_active_window`, and `window_exists` before assuming the target app is ready. - Capture `screenshot_window` or `screenshot_desktop` when visual proof matters. 4. Launch or attach to the target chat app. - For IntelligenceX Chat in a sibling repo, use `./scripts/start-intelligencex-chat.ps1` if available. - Otherwise use `launch_process` or `launch_and_wait_for_window`. - Prefer `wait_for_window` over blind retries. 5. Inspect controls before typing. - Use `list_window_controls` first. - For WebView2 or Chromium-style surfaces, use `diagnose_window_controls` when expected edit controls are missing or unstable. - Prefer saved control targets when the same prompt area or send button will be reused. 6. Interact with the smallest safe target. - Prefer `click_control`, `set_control_text`, and `send_control_keys` when the app exposes usable controls. - If the chat editor is a zero-handle UIA surface, use explicit foreground fallback only when the server was started with `--allow-foreground-input`. - If structure still fails, switch to screenshot-assisted targeting with `get_window_geometry`, `save_window_target`, and `click_window_point`. - For `IntelligenceX.Chat`, prefer the app-owned automation contract through `./scripts/invoke-intelligencex-chat-automation.ps1` before spending time on coordinate fallbacks. 7. Verify the result from more than one angle. - Capture an after screenshot. - Re-read the relevant controls or window text when possible. - Confirm the visible transcript or status change instead of trusting the input acknowledgement alone. - When `IntelligenceX.Chat` runs in automation mode, confirm `wait_for_idle`, `get_transcript_tail`, or `list_conversations` from the app-owned pipe. ## IntelligenceX Notes - IntelligenceX Chat currently uses a WebView2 shell with a composer textarea and send button in the web UI. - In the sibling repo, the current shell template exposes `textarea#prompt` and `button#btnSend`. - Treat that as helpful context, not a guarantee that UI Automation will expose the same shape on every run. - `IntelligenceX.Chat` now also exposes an opt-in app-owned automation pipe in automation mode. That is the preferred verification path when the visible WebView surface is opaque to UIA. - For IntelligenceX verification, prefer this flow: 1. launch the app in automation mode or attach to an existing automation-mode instance 2. capture the real window with DesktopManager so you still have visual evidence 3. call `./scripts/invoke-intelligencex-chat-automation.ps1 -Command new_conversation -LaunchIfNeeded` 4. call `./scripts/invoke-intelligencex-chat-automation.ps1 -Command send_prompt -Text "Reply with only the single word PONG."` 5. call `./scripts/invoke-intelligencex-chat-automation.ps1 -Command wait_for_idle` 6. call `./scripts/invoke-intelligencex-chat-automation.ps1 -Command get_transcript_tail -Count 2` 7. confirm the latest assistant item matches the expected reply 8. keep DesktopManager screenshots and window/control diagnostics alongside the app-owned verification ## Decision Rules - Prefer inspection-first MCP workflows over ad-hoc shell automation. - Prefer `launch_and_wait_for_window` when the very next step depends on the real app window. - Prefer UIA or control-aware actions before coordinate clicks. - Prefer `diagnose_window_controls` before escalating to foreground fallback on modern apps. - Prefer screenshot-assisted window targets over raw pixel guesses when structure is weak. - Prefer `captureAfter` for mutating actions that matter. - Prefer app-owned contracts over brittle coordinate replay when the target app explicitly provides one. - If the server blocks mutating tools, explain that the MCP session is still in read-only mode instead of pretending the selector failed. ## Useful Environment Variables - `DESKTOPMANAGER_REPO_ROOT` - `DESKTOPMANAGER_MCP_EXE` - `DESKTOPMANAGER_MCP_DLL` - `DESKTOPMANAGER_MCP_ALLOW_MUTATIONS` - `DESKTOPMANAGER_MCP_ALLOW_PROCESS` - `DESKTOPMANAGER_MCP_DENY_PROCESS` - `DESKTOPMANAGER_MCP_ALLOW_FOREGROUND_INPUT` - `DESKTOPMANAGER_MCP_DRY_RUN` - `INTELLIGENCEX_REPO_ROOT` - `IXCHAT_AUTOMATION_MODE` ## Reference Files - `Docs/DesktopManager.Mcp.md` - `.agents/skills/desktopmanager-operator/SKILL.md` - `plugins/desktopmanager-windows-verification/scripts/start-desktopmanager-mcp.ps1` - `plugins/desktopmanager-windows-verification/scripts/start-intelligencex-chat.ps1` - `plugins/desktopmanager-windows-verification/scripts/invoke-intelligencex-chat-automation.ps1`