powertoys-verification

## When to use this skill Use this skill whenever you need to **verify PowerToys behavior with the winapp CLI** and emit a structured PASS / FAIL / BLOCKED verdict with evidence (UIA enumeration, log line, settings.json diff, screenshot, etc.). It runs in **two scenarios** that share the same drive techniques, helpers, taxonomy and report format — and differ only on what the checklist is and what bits you test: | Scenario | Trigger | Checklist source | Bits under test | |---|---|---|---| | **A — Module checklist** | "verify all `<Module>` items", "sign off Color Picker" | Supplied file (`references/release-checklist/<module>.md`) | Installed shipped artifact (read-only) | | **B — PR validation** | "validate PR #N" (open or merged), "build it and test the fix", "verify the PRs in this release/hotfix", "sign off 0.X.Y" | **Derived** from each PR's description + diff | **Installed** shipped artifact if the code is already in the build; **build + sideload** if it isn't (unmerged / not-yet-released) | > **Step 0 for every run — pick the scenario.** Read **`references/scenarios/index.md`** (the > router + the "bits under test" contract + the verdict-vocabulary mapping), then read the one > matching scenario doc (`references/scenarios/{module-checklist,pr-validation}.md`). For **B**, also > resolve the bits sub-decision ("is the PR's code in the build under test?") before driving. > This `SKILL.md` is the *shared engine* (drive techniques, helpers, taxonomy, pitfalls) common to both. Each item produces a PASS / FAIL / BLOCKED verdict with evidence. For **A** the checklist is supplied; for **B** you derive it from the PR(s). The skill is the *how* — independent of any specific checklist. ## Required reads (in order) 1. **`references/scenarios/index.md`** — **read FIRST**: the scenario router (A/B), the **"bits under test" contract** (installed-and-immutable, vs build-and-sideload for a PR whose code isn't in the build under test), and the verdict-vocabulary mapping. Then read the **one** matching scenario doc: `references/scenarios/module-checklist.md` (A) · `pr-validation.md` (B). 2. **`references/winapp-ui-testing.md`** — the **prerequisite** UIA mechanics doc (winapp ui verbs, scripted batch testing, file pickers, accessibility audits, screenshots, click-vs-invoke, PostMessage, SendInput cb=40, stunted-UIA recovery, settings-mutation safety contract). **Read this first** — this skill assumes you know its content and only adds PT-specific extensions. 3. **This `SKILL.md`** — the shared engine: the 3-bucket drive-technique selector (Step 2), classification taxonomy, critical pitfalls, helper-script catalog. 4. **`references/modules/<module>.md` IF IT EXISTS** — per-module entry-paths, item-by-item recipes, common BLOCKED traps, fixture lists, source citations. **Always check `references/modules/` first.** If no profile exists, fall back to this SKILL.md and create one after you finish (template in `references/modules/README.md`). 5. **`references/explorer-context-menu-flow.md` IF your module registers an Explorer right-click entry** (PowerRename, File Locksmith, Image Resizer, New+, Preview Pane, RegistryPreview) — shared synthetic-right-click + UIA-invoke + multi-file-selection flow + module-caption table. Helper: `scripts/pt-explorer-contextmenu.ps1`. 6. **`references/pre-flight.md`** — pre-flight checks, bootstrap snippet, state-hygiene cleanup, final wrap-up, hard rules. 7. **`references/reporting-format.md`** — per-item table template, top-of-report summary, step-table rules, anti-patterns, worked example. 8. **`references/environment-setup.md`** — RDP/sleep/screensaver/session-attachment gotchas. Cite in BLK-ENV verdicts. 9. **`references/release-checklist/<module>.md` — SCENARIO A ONLY** — the supplied checklist for the module under test (one file per module; see `references/release-checklist/index.md`). Each item carries `[ADMIN: …]` + `[CLARITY: …]` metadata. **This file IS the set of items to verify.** For B the checklist is derived from the PR(s) instead (see the scenario doc). ## Helper scripts shipped with this skill | File | Purpose | |---|---| | `scripts/pt-shared-events.ps1` | `Invoke-PtSharedEvent`, `Test-PtSharedEvent`, `Get-PtSharedEventCatalog` — 56-entry friendly-name map for PT Named Events (CmdPal.Show, AOT.Pin, PowerLauncher.Invoke, LightSwitch.Toggle, ZoomIt.Draw, ...). The deterministic, foreground-free, UIPI-immune way to trigger a module. | | `scripts/pt-sendinput-chord.ps1` | `Send-PtChord`, `Wait-PtHotkeyAccepted` — last-resort SendInput hotkey injection with the cb=40 fix. Use only when the module has no Named Event and the hotkey itself is the test subject. | | `scripts/pt-foreground-guard.ps1` | `Test-PtForeground`, `Force-PtForeground`, `Assert-PtForegroundOrAbort` — guard helpers to ensure target window IS foreground before SendInput, so keys don't leak to caller's terminal. | | `scripts/pt-cmdpal-recycle.ps1` | `Reset-CmdPalAppX`, `Reset-CmdPalToHome`, `Test-CmdPalDegraded`, `Invoke-CmdPalQuery` — CmdPal-specific lifecycle (handles TextChanged-broken state, BackButton navigation, AppX recycle). | | `scripts/pt-admin-probe.ps1` | `Test-PtAdmin`, `Test-ProcessElevated`, `Test-PtRunnerAdmin` — TokenElevation probes to verify your session and the PT runner have the right elevation for the test. | | `scripts/pt-clipboard-diff.ps1` | `Get-PtClipboardFormats`, `Compare-PtClipboardFormatDiff`, `Set-PtClipboardRich` — multi-format clipboard inspection for Advanced Paste tests. | | `scripts/pt-explorer-com.ps1` | `Get-PtExplorerWindows`, `Open-PtExplorerAtPath`, `Select-PtExplorerFiles`, `Invoke-PtPeekWithExplorerSelection`, `Test-PtInteractiveDesktop` — drive Explorer via Shell COM to set up multi-file selections, then trigger Peek/FZ/PowerRename/Image Resizer/Workspaces via their hotkeys. **Use this for Peek L706-L709, L719-L720 and any test that needs an Explorer file selection.** | | `scripts/pt-explorer-contextmenu.ps1` | `Test-PtDesktopInteractive`, `Open-PtExplorerContextMenu`, `Open-PtBackgroundContextMenu`, `Invoke-PtContextMenuItem`, `Get-PtContextMenuItems` — open Win11's real context menu **coordinate-free** (COM-select the file → focus it → Shift+F10; coordinate right-click only as fallback), then UIA-invoke a menu item by name. **Canonical user-flow path for File Locksmith / Image Resizer / PowerRename / New+ menu-presence + launch tests** — prefer selecting a file (`Open-PtExplorerContextMenu -FileName`) over the background menu; open one menu per fresh window; assert menu TYPE + focus, not module presence. Needs an unlocked interactive desktop. See `references/explorer-context-menu-flow.md` for the full write-up, stability notes, and per-module captions. | | `scripts/pt-shell-verbs.ps1` | `Get-PtShellVerbs`, `Invoke-PtShellVerb`, `Reset-PtShellComCache` — enumerate + invoke CLASSIC HKCR shell verbs via Shell.Application COM. **NOT for PT context-menu modules on Win11** (PT registers via IExplorerCommand, not classic — use `pt-explorer-contextmenu.ps1` for those). Useful for non-PT verbs (Open/Edit/Send-to/third-party) and as a negative check that PT verbs are NOT classic-shadowed. | | `scripts/pt-state.ps1` | `Get-PtSettings`, `Get-PtModuleSettings`, `Get-CmdPalSettings`, `Get-PtRunnerLogTail`, `Test-PtModuleEnabled`, `Test-PtModuleProcess`, `Restart-PtRunner`, `Backup-PtModuleSettings`, `Restore-PtModuleSettings` — common state checks. | | `scripts/pt-nonelevated.ps1` | `Start-PtNonElevated`, `Invoke-PtNonElevatedCapture` — launch an exe at **Medium IL (non-elevated)** from an elevated agent shell via a one-shot `RunLevel Limited` scheduled task. Required for elevation-visibility tests (a non-elevated module must NOT see higher-integrity processes; e.g. File Locksmith L649/L650). Verify the result with `Test-ProcessElevated`. | Dot-source them **all** at once in your bootstrap (the `Get-ChildItem` loop loads every helper — see **Step 1 — Bootstrap**): ```powershell $skill = '<this skill folder>' # the folder containing SKILL.md, e.g. <PT-repo>\.github\skills\powertoys-verification Get-ChildItem "$skill\scripts" -Filter '*.ps1' | ForEach-Object { . $_.FullName } ``` ## Step 1 — Bootstrap ```powershell $module = 'AdvancedPaste' # or 'CmdPal', 'FZ', 'Peek', ... # Work out of %TEMP% during the run (keeps screenshots/scratch off OneDrive); move to the # sign-off archive at the very end (see Step 7). $workspace = "$env:TEMP\verify-$module-$(Get-Date -Format yyyyMMdd-HHmmss)" New-Item -ItemType Directory -Path $workspace, "$workspace\artifacts" -Force | Out-Null $report = "$workspace\verify-$module.md" # Dot-source helpers $skill = '<this skill folder>' # set once at top of your script (the folder containing SKILL.md) Get-ChildItem "$skill\scripts" -Filter '*.ps1' | ForEach-Object { . $_.FullName } # Verify environment "=== Environment ===" | Tee-Object $report -Append "IsAdmin: $(Test-PtAdmin)" | Tee-Object $report -Append $rn = Test-PtRunnerAdmin "PT runner: PID=$($rn.Pid) Elevated=$($rn.Elevated)" | Tee-Object $report -Append # The checklist source depends on the scenario (see references/scenarios/): # A - read the supplied references/release-checklist/<module>.md # B - derive 1-3 items from each PR's `gh pr view/diff`; then drive the installed bits, OR # build+sideload if the code isn't in the build under test (pr-validation.md) # Then iterate the items (see Step 6 - Verifier loop). ``` ## Step 2 — Drive techniques Every checklist item boils down to ONE of three intents. **Pick the bucket from the verb in the item, then use the best technique inside it.** Stop at the first technique that works. | Intent | Verb-cues in the checklist item | Bucket | |---|---|---| | Change a setting | "default is X", "setting persists", "is enabled/disabled by default", "value Y is accepted" | §2.A | | Interact with a UI element | "click X", "toggle X", "type into Y", "X button is visible", "selecting Z does W" | §2.B | | Trigger a module action | "pressing hotkey X opens Y", "module launches", "Z happens when invoked" | §2.C | ### §2.A — Change a setting (single technique) Edit the JSON file the module reads, wait for the file-watcher debounce, assert, then restore from backup. Zero external tools. ```powershell $bk = Backup-PtModuleSettings -ModuleDir AdvancedPaste try { $j = Get-PtModuleSettings -ModuleDir AdvancedPaste $j.properties.IsAdvancedAIEnabled.value = $false $j | ConvertTo-Json -Depth 12 | Set-Content "$env:LOCALAPPDATA\Microsoft\PowerToys\AdvancedPaste\settings.json" Start-Sleep -Seconds 4 # debounce — runner re-reads via file-watcher # ... assertion ... } finally { Restore-PtModuleSettings -ModuleDir AdvancedPaste -BackupPath $bk } ``` > For shell-extension modules (PowerRename, File Locksmith, Image Resizer, New+) edit the **module-owned** file under `%LOCALAPPDATA%\Microsoft\PowerToys\<Module>\`, then `Restart-PtRunner` (and on stubborn handlers, restart Explorer). See pitfall #12 below. > > If you need to flip the *enabled* bit for a whole module, debounce isn't enough — call `Restart-PtRunner` after the write. ### §2.B — Interact with a UI element (2 techniques, most-reliable first) #### B1. UIA invoke / set-value — **always try first** ```powershell winapp ui invoke 'SubmitButton' -a PowerToys.Settings winapp ui set-value 'QueryTextBox' '=2+3*4' -a PowerToys.PowerLauncher Start-Sleep -Milliseconds 600 winapp ui inspect -a PowerToys.PowerLauncher --depth 7 -i 2>$null ``` Invoke goes through UIA InvokePattern COM IPC — no foreground steal, no UIPI. See references/winapp-ui-testing.md §CRITICAL — invoke vs click. #### B2. PostMessage WM_KEYDOWN/CHAR — when UIA can't reach the target For elevated targets, AppX windows with stunted UIA trees, or keystrokes that UIA `set-value` can't dispatch (arrow-key ListView nav, Enter to commit). See references/winapp-ui-testing.md §CRITICAL — Keystroke input that bypasses UIPI (PostMessage). Esc is often filtered by WinUI 3 raw-input hook — use BackButton invoke instead. ### §2.C — Trigger a module action (2 techniques, most-reliable first) | | C1 Named Event | C2 SendInput chord | |---|---|---| | **Proves** | The action fires (the path *downstream* of the hotkey). **Not** that the chord is bound. | The full path: real keys → runner hook → action. The **only** method that proves the chord binding itself. | | **Robustness** | Highest — no foreground, no input desktop, UIPI-immune; works headless / RDP-minimized. | Lowest — needs an attached input desktop (else `BLK-ENV`), steals foreground, can't inject OS-reserved chords (Win+L / Win+Tab). | | **Precondition** | Owning module process is running (the event only exists while it is). | Attached input desktop + foreground. | **Pick by what the item asserts:** for "does action Y happen" use C1; for "pressing chord X triggers Y" or "the rebind takes effect", C1 is insufficient (it bypasses the chord) — use C2, or C1 *plus* a runner-log line proving the chord was accepted. #### C1. Named Event signal — preferred ```powershell Invoke-PtSharedEvent -Name 'CmdPal.Show' # opens CmdPal without keyboard Invoke-PtSharedEvent -Name 'AOT.Pin' # pins foreground window via AOT Invoke-PtSharedEvent -Name 'PowerLauncher.Invoke' # opens PT Run Invoke-PtSharedEvent -Name 'LightSwitch.Toggle' # toggles theme Get-PtSharedEventCatalog | Format-Table # full list ``` No synthetic input — it's a `SetEvent` on the kernel event the module waits on, the same downstream path the runner's hotkey handler signals. Verify the side effect via UIA (`winapp ui list-windows -a <module>`), a log line (`Get-PtRunnerLogTail`), or settings.json diff (`Get-PtModuleSettings`). The event only exists while the owning process runs, so `Test-PtSharedEvent` doubles as an "is the module alive" check. #### C2. SendInput chord — last resort / chord-binding verification Real synthetic keys. Loud (steals foreground) and fragile, but the only way to prove the activation chord is actually bound. The runner's global keyboard hook catches the chord regardless of focus, so the precondition is just an **attached input desktop** (pitfall #7; on a detached desktop `SendInput` returns `ACCESS_DENIED` and the keys vanish → mark `BLK-ENV`). ```powershell # Precondition: input desktop attached? 0 = detached → don't bother sending, mark BLK-ENV (pitfall #7) if ([PtFg]::GetForegroundWindow() -eq [IntPtr]::Zero) { throw 'No input desktop — BLK-ENV (pitfall #7)' } Send-PtChord -Mods 0x5B,0x10 -Key 0x43 # Win+Shift+C → Color Picker (cb=40 fix is inside the helper) $line = Wait-PtHotkeyAccepted -ModuleHint 'Color' -TimeoutSec 3 if (-not $line) { throw 'Runner did not log hotkey invocation' } ``` > **Rare fallback — a module that uses its own `RegisterHotKey` and exposes no Named Event.** Post `WM_HOTKEY` (`0x312`) straight to its message window (find the HWND via `EnumWindows`+`GetClassName` through `Add-Type` — same P/Invoke pattern as `pt-foreground-guard.ps1`). **No current PT module needs this:** ZoomIt — the obvious candidate — also waits on Named Events (`ZoomIt.Zoom`, `ZoomIt.Draw`, …; source: `Zoomit.cpp` `CreateEventW(ZOOMIT_ZOOM_EVENT)`), so drive it with C1. > **Different case — sending keys *into* a specific focused window** (e.g. a CmdPal alias like `=` / `<` / `>` that `winapp ui set-value` can't trigger because it bypasses TextChanged; see `references/modules/command-palette.md`). Here the keystrokes go to whatever currently has focus, so you must bring the target window foreground first: > ```powershell > Assert-PtForegroundOrAbort -AppId Microsoft.CmdPal.UI # -AppId = the window you're typing INTO