verify-on-device

# Verify on Device Build, install, and visually verify the app on an Android emulator or physical device. The skill prefers the Android CLI for agents (`android`) when installed and falls back to mobile-mcp + adb otherwise. **Prerequisites:** Node.js v22+, Android SDK with platform-tools, and an emulator or device to target (step 0 can provision one when the Android CLI is installed). **Optional:** Google's Android CLI for agents (`android`). When present, the skill uses `android run` for combined install+launch, `android layout --diff` for low-token screen-transition polling, and `android docs` for platform-API lookups. If not installed, every step falls back to the mobile-mcp / adb path with no behavior change. ## Detect the Android CLI Once Per Session Run this probe once at the start of a verification task and cache the result. Every CLI-based block in this skill is gated on `USE_ANDROID_CLI=1`. The probe validates that `android` on PATH is Google's agent CLI (prints a semver like `0.7.15232955`) and not the deprecated Android SDK `android` tool from `tools/`, which shadows it whenever the legacy SDK tools dir is on PATH. The probe only detects — it does not mutate PATH or create symlinks, since an in-script `export PATH` would not survive subsequent shell invocations in this skill. If `android` is missing, shadowed, or broken, it sets `USE_ANDROID_CLI=0` and prints a hint pointing at the canonical install location (`~/.android/bin/android-cli`) when present. Exact PATH/symlink fix depends on the user's environment — the goal is to make `android --version` resolve to the agent CLI; the user applies the fix once in their shell rc so it persists. The probe also writes `USE_ANDROID_CLI` to `/tmp/.verify_on_device.env`. Each subsequent `Bash` tool call starts a fresh shell, so a plain shell variable would be lost between invocations — every gated block in this skill begins by sourcing that file to restore the cached result. ```bash _android_is_agent_cli() { android --version 2>&1 | grep -qE '^[0-9]+\.[0-9]+\.' } if command -v android >/dev/null 2>&1 && _android_is_agent_cli; then USE_ANDROID_CLI=1 android --version # log for reproducibility else USE_ANDROID_CLI=0 if [ -x "$HOME/.android/bin/android-cli" ]; then echo 'NOTE: Agent CLI binary is at ~/.android/bin/android-cli but `android` does not resolve to it (likely missing symlink or shadowed by the legacy Android SDK tool).' echo 'Fix: update your PATH/symlinks until `android --version` prints a semver (e.g. symlink android-cli to a name `android` on a directory earlier in PATH than the legacy Android SDK tools), then restart the session.' fi fi # Persist for subsequent Bash invocations (shell state does not survive # across separate Bash tool calls). Subsequent gated blocks `source` this. echo "USE_ANDROID_CLI=$USE_ANDROID_CLI" > /tmp/.verify_on_device.env ``` If `USE_ANDROID_CLI=0`, follow the fallback blocks (labelled "Fallback") throughout this skill. ## Verification Status of Agent-CLI Blocks The `USE_ANDROID_CLI=0` fallback paths in this skill (mobile-mcp + adb) have been exercised end-to-end against this repo. The `USE_ANDROID_CLI=1` blocks were all re-run against agent CLI 0.7.15232955 on 2026-04-21 against this repo. Results: | CLI block | Status | |---|---| | `android run` (step 7 install + launch) | Verified — installs and launches in one call. | | `android layout --diff` (screen-transition polling) | Verified — captured the `ordersList` transition on a tab switch; diff JSON is a small fraction of a full layout dump. *Note: `--device=<device_id>` was added to the documented invocation post-verification (multi-device fix); the flag is documented by the CLI but the new combination has not been re-run end-to-end.* | | `android screen capture --annotate` + `screen resolve` (Option B tap) | Verified — both short (`-a`/`-o`) and long (`--annotate`/`--output=…`) flag forms work. | | `android docs search` / `docs fetch` | Verified — first invocation auto-downloads a knowledge-base zip (~one-time, a few seconds). | | `android emulator list` (step 0 lifecycle) | Partial — `list` runs end-to-end. `create`/`start`/`stop` shape confirmed via `--help` only; no AVD was created during verification, so step 0 is flagged **Experimental** in its heading. | | `android describe` | **Rejected.** Output is multi-line plain text (not JSON, not paths-to-JSON). Requires `ANDROID_HOME` set; produces listings only after a build. Replaced with `find` in step 7. | If a CLI block fails in practice, **do not assume the docs are right**. Fall back to the `USE_ANDROID_CLI=0` path for that step, file the discrepancy as a skill issue, and fix it in the skill before the next run. ## Critical Rule: Default to Main App (Store Management) Unless the task explicitly mentions **POS**, **Point of Sale**, or **WooPos**, always operate in the **main app** (store management) context — `MainActivity` with bottom navigation tabs. This applies to all workflows: creating orders, viewing products, collecting payments, etc. The main app is the default; POS is only used when specifically requested. ## Critical Rule: Always Restart the App Do NOT attempt to recover from the current screen state when you start a task. Always force-stop the app and relaunch it to start from a known state (the dashboard or POS). This avoids wasted time navigating out of unknown screens. Restarting is not resetting. Preserve and reuse a coherent authenticated session whenever one exists. Never uninstall the app or clear its data merely to begin verification, and install APK updates without clearing app data. ```bash adb -s <device_id> shell am force-stop com.woocommerce.android.dev adb -s <device_id> shell am start -n com.woocommerce.android.dev/com.woocommerce.android.ui.main.MainActivity ``` For POS tasks (only when explicitly requested): ```bash adb -s <device_id> shell am force-stop com.woocommerce.android.dev adb -s <device_id> shell am start -n com.woocommerce.android.dev/com.woocommerce.android.ui.woopos.root.WooPosActivity ``` ## Critical Rule: Never Estimate Tap Coordinates From Raw Screenshots **NEVER estimate tap coordinates directly from a raw (un-annotated) screenshot.** Screenshots are scaled down from the actual device resolution (e.g., a 1080x2400 device produces a ~480x1065 screenshot). Coordinates derived from raw screenshots will be systematically wrong. Use one of the two workflows below. Both translate a human-readable target (an accessibility label, a visual element) into exact device-pixel coordinates — neither relies on pixel-measuring a screenshot. ### Option A — Accessibility-tree workflow (default) 1. Call `mobile_list_elements_on_screen` to get elements with their **device-pixel coordinates** 2. Compute tap target as the **center** of the element's bounding rect: `tap_x = x + width/2`, `tap_y = y + height/2` 3. Call `mobile_click_on_screen_at_coordinates` with those computed coordinates 4. Call `mobile_take_screenshot` AFTER tapping to visually confirm the result Only use `mobile_take_screenshot` for **visual verification** — never for deriving coordinates. ### Option B — Visual-label workflow (`USE_ANDROID_CLI=1`) Useful when an element lacks an accessibility label or test tag, or when you already have an annotated screenshot in context. `android screen capture --annotate` overlays numeric labels (#1, #2, ...) on every interactive element; `android screen resolve` substitutes `#N` placeholders in a template string with the element's device-pixel `x y` coordinates. The `android screen ...` commands do not support `--device`, so use this workflow only when the CLI's default device is the same device you intend to tap. ```bash # Capture an annotated screenshot — each interactive element gets a number. android screen capture --annotate --output=/tmp/ui.png # Idiomatic: let resolve produce a complete `input tap X Y` command and pipe # it straight to `adb shell`. The CLI replaces `#5` with the resolved coords. android screen resolve --screenshot=/tmp/ui.png --string="input tap #5" \ | adb -s <device_id> shell # Alternative: capture just the coordinates and feed mobile-mcp's tap tool. COORDS=$(android screen resolve --screenshot=/tmp/ui.png --string="#5") # $COORDS is now "<x> <y>"; call mobile_click_on_screen_at_coordinates with those. ``` Option A remains the default — accessibility-tree coordinates are stable and don't require visual inspection. Reach for Option B when Option A does not surface the element you need. ## Waiting for Screen Transitions After every navigation action (tap, BACK press, app launch, swipe), the screen may be animating or loading data. ALWAYS follow one of the two stabilization protocols below. ### Preferred: Diff-Based Polling (`USE_ANDROID_CLI=1`) `android layout --diff` returns only the elements that changed since the last snapshot, instead of re-reading the entire accessibility tree (50-200+ elements per call). This is the single biggest token-consumption win over repeated `mobile_list_elements_on_screen` calls — measure on your own flow to confirm the magnitude. Always pass `--device=<device_id>` to keep these calls pinned to the same device chosen in step 1; without it, `android layout` may target a different connected device than the one the app was launched on. ```bash # `[[:space:]]*` around the colon makes the pattern tolerant of both # compact and pretty-printed JSON, so a stray --pretty in the chain # doesn't silently break the grep. TARGET='"resource-id"[[:space:]]*:[[:space:]]*"com.woocommerce.android.dev:id/ordersList"' # Baseline snapshot immediately after the action — also greppable: if the # transition was instantaneous, the target is already on screen and every # subsequent --diff would return empty (diffs are delta-only). android layout --device=<device_id> --pretty --output=/tmp/layout_t0.json if ! grep -qE "$TARGET" /tmp/layout_t0.json; then # Poll diffs until the expected target appears (1 second between polls). for i in 1 2 3 4 5; do android layout --device=<device_id> --diff --output=/tmp/layout_diff.json grep -qE "$TARGET" /tmp/layout_diff.json && break sleep 1 done # Safety net — one full-layout read in case the target arrived between # two diffs but didn't change after that (so no later diff mentions it). android layout --device=<device_id> --pretty --output=/tmp/layout_final.json grep -qE "$TARGET" /tmp/layout_final.json || echo "Target not found after polling." fi ``` Replace the `TARGET` pattern with the `resource-id`, Compose test tag, or `content-description` of the screen you expect to land on (see the WooCommerce Navigation Reference). Compose test tags surface as `resource-id` because `testTagsAsResourceId` is on, so the example pattern works for both — but `content-description` lives under a different JSON key (typically `content-desc`), so swap the key, not just the value. ### Fallback: Repeated Layout Reads (no `android` CLI) 1. Call `mobile_list_elements_on_screen` after the action. 2. If the expected target element is NOT present, call `mobile_list_elements_on_screen` again. Each tool round-trip takes ~1-2 seconds, which provides sufficient implicit delay. 3. Repeat up to 5 times. 4. If after 5 attempts the expected element is still missing, take a screenshot for diagnosis and report the issue. ### Guidance That Applies to Both Paths **Loading indicators to watch for:** - Skeleton/shimmer views (animated placeholder content) — the screen is loading data, keep waiting. - `CircularProgressIndicator` or `ProgressBar` elements — an operation is in progress, keep waiting. - Empty state views with text like "No orders yet" — the screen IS loaded, just empty. Do NOT keep waiting. **When NOT to retry:** If the layout (full or diff) returns the same result 3 times in a row with no change, the screen is stable. The element you want is genuinely not present — consider scrolling or navigating differently. ### Timing Guidelines | Action | Expected Wait | Max Attempts | |--------|--------------|--------------| | App launch to dashboard | 3-8 seconds | 5 | | Tab navigation (bottom bar) | <1 second | 3 | | Opening a detail screen | 1-3 seconds | 4 | | Network data load (pull to refresh) | 2-10 seconds | 8 | | Dialog appearance after button tap | <1 second | 3 | | Keyboard appearing after field tap | <1 second | 2 | ## Text Input Workflow Typing text into a field requires a specific sequence: 1. **Find the input field** using `mobile_list_elements_on_screen`. Look for elements with type `EditText`, `TextField`, or hint text like "Search". 2. **Tap the field** using `mobile_click_on_screen_at_coordinates` at its center to give it focus. The soft keyboard will appear. 3. **Confirm focus** — call `mobile_list_elements_on_screen` to verify the field is focused. 4. **Type the text** using `mobile_type_keys`. Set `submit: false` unless you want to press Enter after typing. 5. **Dismiss the keyboard** if needed: call `mobile_press_button` with `BACK`. On Android, the first BACK press while the keyboard is visible dismisses the keyboard only — it does NOT navigate back. A second BACK press would navigate back. **Common pitfall:** Calling `mobile_type_keys` without first tapping the input field types into whatever element last had focus (or nothing). **Search fields:** The orders and products lists use a toolbar search icon. Tap the magnifying glass icon first, wait for the search field to expand, then type into the expanded field. ## Handling Unexpected Dialogs WooCommerce may show dialogs automatically on launch or during navigation. Detect and dismiss these before proceeding. After launching the app or navigating to a new screen, call `mobile_list_elements_on_screen` and check for: | Dialog Type | How to Detect | How to Dismiss | |-------------|---------------|----------------| | **Privacy Banner** | Elements with text "Privacy Settings" or "Save" button on a bottom sheet. This is NOT cancellable — tapping outside won't work. | Tap the "Save" button. | | **What's New / Feature Announcement** | Element with identifier containing `closeFeatureAnnouncementButton` or text "Close". | Tap the close button. | | **App Rating Dialog** | AlertDialog with text containing "rate" or "enjoy". | Tap "No Thanks" or "Remind Me Later". | | **Android Permission Dialog** | Elements from `com.android.permissioncontroller`, or text containing "Allow" / "Don't allow". | Tap "Allow" for testing purposes. | | **Snackbar** | Element with identifier containing `snackbar_text` near the bottom of the screen. | Do NOT dismiss — auto-dismisses after a few seconds. May temporarily cover bottom nav tabs; if a bottom tab tap fails, wait 3-4 seconds and retry. | | **Store Name Dialog** | Text "Name your store" (id: `nameYourStoreDialogFragment`). | Tap "Save" or dismiss. | | **Create Test Order Dialog** | Text related to test order creation. | Tap "Dismiss" or "Create". | **General dialog dismissal strategy:** Look for a dismiss/close/cancel button and tap it. If none visible, try `mobile_press_button` with `BACK`. If BACK doesn't work (non-cancellable dialogs), look for any actionable button ("OK", "Save", "Got it") and tap it. After dismissing, call `mobile_list_elements_on_screen` to confirm the dialog is gone. ## Finding Elements That Require Scrolling When `mobile_list_elements_on_screen` does not return the element you expect, it may be off-screen: 1. Call `mobile_list_elements_on_screen` and check for the target element. 2. If not found, call `mobile_swipe_on_screen` with direction `up` (swipe up = scroll down) from the center of the screen. 3. Call `mobile_list_elements_on_screen` again. 4. Repeat up to 10 times. If the same elements keep appearing (no new content), you have reached the bottom of the list. 5. If still not found, try scrolling back up (direction `down`) or try an alternative navigation path. **Tip:** To scroll within a specific scrollable container (not the full screen), use the container's center coordinates as the swipe starting point. ## Working with Element Lists `mobile_list_elements_on_screen` can return 50-200+ elements. To find what you need: - **By resource identifier (most reliable):** Match the `identifier` field (e.g., `com.woocommerce.android.dev:id/ordersList`). Resource IDs are stable across app versions. - **By display text:** Match the element's `text` or `label` field. Useful for finding specific list items (e.g., order "#1234"). - **By position:** Elements are returned in document order (top to bottom, left to right). Toolbar/status bar elements appear first, list items in visual order. **Compose vs View elements:** View-based screens have stable `com.woocommerce.android.dev:id/*` identifiers. Compose-based screens (Dashboard cards, Settings, newer screens) may lack resource IDs — rely on `contentDescription` or display text instead. ## Authentication and Session Preservation Prefer an already coherent, authenticated app session. Upgrade/reinstall the APK in place so app data is preserved; do not call `mobile_uninstall_app` or `pm clear` unless the task explicitly requires destructive clean-state testing. After launch, inspect the screen: - If the dashboard for the expected target appears, preserve and reuse that session. - If login is required, the target is not the expected store, or the session cannot be confirmed, invoke `tools/agent-login/agent-login.sh` with the target flavor, selected serial, and Android user. It uses the deterministic `default` profile. Add `--profile <name>` only when the user or task explicitly supplies that nonsecret override. Do not list, open, read, or print profile files. Report only the script's sanitized outcome. - On `PROFILE_ERROR`, ask the user to complete the one-time profile setup documented in `references/agent-auto-login.md`, or to log in manually on the device. Never ask for credentials in chat. `ALREADY_ACTIVE` means the selected session matches the requested target and connection. A different selected store or connection returns `CONFLICT`; auto-login never replaces it. The dev tool assumes the configured target is connected,