Designs Android testing with Espresso, UI Automator, and Compose. Use when planning device matrices, screenshot tests, CI flows, or flake-control workflows.
Instrucciones de origen · Vista previa de solo lectura
name
qa-testing-android
description
Designs Android testing with Espresso, UI Automator, and Compose. Use when planning device matrices, screenshot tests, CI flows, or flake-control workflows.
compatibility
Portable core. Works on Claude Code and Codex.
version
1.1
last_validated
2026-07-11T00:00:00.000Z
QA Testing (Android)
Android testing automation with Espresso, UI Automator, Compose Testing, screenshot tests, and adaptive UI validation.
17 (API 37), shipped 2026-06-16; Android 16 (API 36) is the prior release
See "Google Play Target API Policy" below for what to actually target — the newest OS version is not automatically the target-API requirement
AGP
9.2.x
Requires Gradle 8.11+; breaking DSL changes from 8.x
Robolectric
4.16.x
Supports up to SDK 36 (Baklava); SDK 36 requires JDK 21. API 37 support is not yet released as of 2026-07-11 — an open upstream issue (robolectric/robolectric#11239, filed 2026-06-14) tracks it. Do not assume Robolectric can simulate Android 17 behavior yet; verify before relying on it for API-37-specific logic
UI Automator
2.4.0-rc01
Modern uiAutomator {} DSL; release-candidate, not yet fully stable — verify current status before pinning in a template
Compose Preview Screenshot Testing
0.0.1-alpha15
Still alpha; requires AGP 8.5+, Kotlin 2.2.10+ (raised from 1.9.20 — verify against the current release notes before relying on an older Kotlin floor), JDK 17+
ATD images
API 30 only
Use standard google/aosp images for API 35/36/37
Maestro
API 35/36 added 2026 Q2
API 37 support unverified as of 2026-07-11 — check release notes before targeting Android 17 devices in Maestro Cloud
Google Play Target API Policy (verify before every release cutover)
As of the 2026-08-31 deadline: new apps and app updates submitted to Google Play must target API level 36 (Android 16) or higher — submissions targeting lower are rejected in Play Console. Apps not updated at all must still target at least API level 35 (Android 15) or they become invisible/uninstallable for new users on newer OS versions. A one-time extension to 2026-11-01 is available by request. This is a moving deadline — re-check https://developer.android.com/google/play/requirements/target-sdk before treating any specific API number as "the" requirement, since Google raises it roughly once a year.
Quick Start
Prefer build-managed devices (GMD) for CI, using ATD images where the suite and supported API levels allow it; use connectedAndroidTest for local ad-hoc runs.
ATD images only exist for API 30. For API 35/36 suites, use google or aosp images.
Enable Android Test Orchestrator and clearPackageData for instrumented suites that need strong isolation.
Disable animations in Gradle, not with ad-hoc ADB steps inside the test flow.
Treat screenshot tests as a separate fast regression layer: Compose Preview Screenshot Testing for @Preview coverage, Paparazzi or Roborazzi for JVM rendering, device-based snapshots only when hardware fidelity matters.
Cover adaptive layouts explicitly: compact and expanded layouts, tablet or foldable paths when supported, and orientation changes where the product depends on them.
Add accessibility checks to existing UI tests early instead of treating accessibility as a manual-only phase.
Recommended Gradle defaults for stable instrumented tests:
Use test-failure snapshots or screenshots in CI so failures have artifacts, not just stack traces.
Accessibility And Adaptive Coverage
Run accessibility checks in the same suite as high-value Espresso or Compose journeys.
Validate compact and expanded layout states intentionally; do not assume a phone-only test matrix covers responsive layouts.
Prefer stable selectors: withId() for Views, testTag for Compose, and resource-id or content descriptions for UI Automator.
Expert Judgment (What A Checklist Misses)
Emulator vs. Physical Device — the actual decision
A checklist says "use emulators for CI, real devices for release." The judgment call is which real-device signals are worth paying for:
Emulators (ideally build-managed, ATD where hardware fidelity is not required) are correct for PR gates because they are deterministic, disposable, and free of thermal/battery/OEM noise that would otherwise get misattributed to app bugs.
Physical devices or a device farm earn their cost only for journeys where OEM or hardware behavior is the actual risk: camera capture, biometric auth, NFC/BLE, background-work reliability under real Doze/App Standby, foldable hinge/posture transitions, and payment SDKs that behave differently under real Play Integrity checks. If none of those are in the flow, a device farm run is theater, not signal.
A common anti-pattern is running the entire regression suite nightly on a device farm "for confidence." This burns budget without changing the failure mode you'd actually catch — most of what fails there also fails, faster and cheaper, on a managed emulator. Reserve device-farm minutes for the handful of hardware-coupled journeys above and gate them separately from the general regression suite.
Emulator system-image choice is itself a judgment call, not just "google vs aosp": Google Play Services-dependent flows (Play Billing, FCM, Play Integrity, Maps) need google images; anything asserting on hardware-rendered pixels needs a non-ATD image; everything else should default to ATD for speed.
Android Flake Taxonomy (diagnose by category, not by symptom)
Generic "flaky test" triage wastes time re-running instead of classifying. Android UI-test flakiness clusters into a small number of root-cause families — identify which one you're looking at before reaching for retries:
Synchronization gaps — an async operation Espresso/Compose does not know how to wait for. Signature: passes on a fast local machine, fails under CI load; NoMatchingViewException or an assertion firing against a stale loading state. Root cause is almost always an unregistered IdlingResource (Espresso) or a missing waitUntil/synchronized TestDispatcher (Compose/coroutines) — see references/espresso-patterns.md for the registration trap.
Animation/transition timing — window or property animations still running when Espresso samples the view tree. Fix at the Gradle level (animationsDisabled = true), never per-test with sleeps; ad-hoc adb shell settings put global *_scale 0 inside a test body is a workaround that silently stops working if the runner changes.
Semantics-tree merging (Compose-specific) — assertExists()/onNodeWithText() finds nothing because Material components merge child semantics into the parent node by default. Not a timing issue at all; do not "fix" it with a wait loop. See the merged-tree trap in references/compose-testing.md.
State leakage across tests — a prior test left SharedPreferences, a DB row, or a singleton in a state the next test doesn't expect. Symptom is order-dependent failures that disappear when the failing test is run alone. Fix with Test Orchestrator + clearPackageData, not by reordering tests.
Doze / App Standby / background restrictions — tests that rely on background work (WorkManager, FCM, foreground services) can behave differently under device power-management states than in a freshly booted emulator. This shows up almost exclusively on physical devices or long-running CI machines, rarely on ephemeral managed devices — if a background-work test is flaky only in one environment, suspect power-management state before suspecting the test.
Renderer/fidelity mismatches — screenshot diffs that are real but meaningless: ATD disabling hardware rendering, GPU driver differences between CI and local, or font substitution differences across OS images. Distinguish this from an actual visual regression by re-running the same golden generation pipeline on the same image type used to record it.
Non-deterministic test data or clocks — live network calls, real System.currentTimeMillis(), or unseeded random data leaking into assertions or screenshot goldens. Always mockable; the fact that it's still happening usually means a fake wasn't reused when a new screen was added.
When a test is flaky, name which of the seven it is before touching the test — the fix for #1 (register/await) will not touch #4 (isolate/clear state), and applying #2's fix (disable animations) to a #5 issue (Doze) does nothing.
Test Pyramid Reality For Compose Apps
The classic pyramid (many unit tests, some integration, few E2E) still holds, but Compose changes where the middle layer sits:
Compose lets you unit-test a composable's rendered semantics tree via createComposeRule() without an Activity or device at all — this is functionally a unit test even though it "looks like" a UI test. Prefer it over createAndroidComposeRule<Activity>() whenever the composable doesn't need real navigation, DI graph, or activity lifecycle.
This shifts a large share of what used to be slow instrumented Espresso coverage down into fast, parallelizable JVM-adjacent tests — but only if ViewModels are tested independently (Turbine/MockK against StateFlow, not through the UI). A Compose app that only has "instrumented Compose tests that also exercise the ViewModel" has recreated the old inverted pyramid with new tools.
Screenshot tests (Compose Preview Screenshot Testing / Paparazzi / Roborazzi) form a separate, parallel layer, not a replacement for either unit or instrumented behavioral tests — they catch visual regressions that behavioral assertions cannot see (spacing, overlap, dark-mode contrast) but say nothing about correctness of state transitions.
Reserve UI Automator and Maestro-style E2E flows for the top of the pyramid: a handful of true cross-app or full-stack smoke journeys (onboarding, checkout, deep-link entry), run on nightly/release pipelines, not PR gates.
Device / API Fragmentation Strategy
Always cover the app's actual min supported API and its current target API — these two are non-negotiable regardless of market share.
Beyond that pair, prioritize by behavioral risk, not by device popularity: API levels that changed permission models, background-execution limits, or storage scoping (e.g., scoped storage, notification runtime permission, foreground-service type declarations) deserve dedicated coverage on the OS version that introduced the change, even if that OS version is not the most common one in your install base.
Treat any specific device/API market-share number as unverified unless sourced from your own analytics (Play Console's own device catalog and Android Vitals for your app) — public aggregate stats go stale within months and vary enormously by app category and geography. Do not hardcode "X% of users are on API N" into test-planning docs; pull it fresh from the app's own Play Console dashboard each planning cycle.
Foldable/tablet/expanded-layout coverage is now a fragmentation axis in its own right, independent of API level — a phone-only test matrix increasingly under-covers real usage on large-screen and foldable devices even when API coverage looks complete.
When To Invest In Screenshot Testing
Screenshot testing has real setup and maintenance cost (goldens go stale, false positives from font/renderer drift, review burden on every intentional UI change). It is worth that cost when:
The team ships UI changes frequently enough that manual visual review does not scale (multiple PRs/day touching shared components).
The app has a design system or component library where a single regression fans out across many screens — screenshot tests catch that fan-out cheaply; behavioral tests do not.
Dark mode, large font scale, or RTL layouts are supported and have historically regressed silently.
It is a poor early investment for a small team still iterating rapidly on visual design — churn in intentional goldens will dominate signal from real regressions. Start with host-side tools (Compose Preview Screenshot Testing, Paparazzi, Roborazzi) for cheap iteration; only add device-based snapshot testing (Shot, or ATD-excluded device runs) once a specific rendering-sensitive surface (WebView, Maps, camera preview, custom Canvas/GPU work) has already caused a shipped visual bug that host-side rendering could not have caught.
Writing Tests
Espresso (Views): open references/espresso-patterns.md
Compose: open references/compose-testing.md
UI Automator (system or cross-app): open references/uiautomator.md
Screenshot tests: open references/screenshot-testing.md
Adaptive screen-size testing: open references/adaptive-screen-testing.md
Accessibility checks: open references/accessibility-checks.md
Workflow
Add a New UI Test (Instrumented)
Pick the narrowest framework that can observe the behavior: Espresso or Compose first, UI Automator at the system boundary.
Add stable selectors: View id, Compose Modifier.testTag, system resource-id or content description.
Control externals: fake network, deterministic data, fixed time and locale, and stable image sources.
Add waits through framework synchronization or explicit conditions; avoid Thread.sleep().
Run locally with connectedAndroidTest or a single managed-device task before widening the matrix.
Add Screenshot Coverage
Use Compose Preview Screenshot Testing for @Preview-driven Compose UI states.
Use Paparazzi or Roborazzi when you need JVM-fast coverage across Compose and legacy Views.
Keep goldens small, reviewable, and tied to stable UI states instead of whole-screen snapshots of volatile content.
Capture HTML or diff artifacts in CI on every failure.
Diagnose a Flaky Instrumented Test
Reproduce locally or on one managed device before widening the matrix.
Replace sleeps with idling, waitUntil, onElement, or watcher-based synchronization.
Capture logcat, screenshot, screen recording, and any managed-device failure snapshot.
If still flaky, isolate app state further with Orchestrator or runner args and bisect the interaction steps.
Add A CI Gate (Preferred: Build-Managed Devices)
Configure build-managed devices plus ATD images when hardware rendering is not required and the API level is supported by the image.
Add the GitHub Actions GPU flag when your runner lacks hardware rendering support.
Keep PR gates small and deterministic; expand device groups or sharding on nightly and release pipelines.
Upload reports from build/reports/androidTests/, screenshot reports, diff images, logcat, and managed-device outputs.
ADB Commands (Triage)
# Screenshot
adb exec-out screencap -p > screenshot.png
# Screen recording
adb shell screenrecord /sdcard/demo.mp4
# Pull managed-device test artifacts after a local failure
adb pull /sdcard/Android/media ./device-artifacts
CI Integration
Preferred: build-managed devices. See references/gradle-managed-devices.md and references/android-ci-optimization.md.
# .github/workflows/android.ymlname:AndroidCIon: [push, pull_request]
jobs:test:runs-on:ubuntu-lateststeps:-uses:actions/checkout@v6# or later — verify at actions/checkout releases-uses:actions/setup-java@v5# or laterwith:java-version:'17'# or laterdistribution:'temurin'-uses:gradle/actions/setup-gradle@v5# or later-run:./gradlewtestDebugUnitTest<device><api>DebugAndroidTest-Pandroid.testoptions.manageddevices.emulator.gpu=swiftshader_indirect
ASCII Flow
Android testing request
-> Classify layer: unit, Robolectric, Compose, Espresso, UI Automator, screenshot
-> Choose device/API matrix from risk, analytics, and adaptive UI needs
-> Stabilize state, idling, selectors, permissions, and test data
-> Run local targeted tests before managed-device or connected-device gates
-> Capture artifacts: logs, screenshots, videos, and test reports
-> Deflake root cause before expanding CI matrix or retries
Navigation
The reference guides are intentionally large; search within them instead of loading everything:
Prefer build-managed devices plus ATD images for stable CI
Use Android Test Orchestrator only where isolation solves a real state-leak or crash problem
Add screenshot and accessibility checks to high-value UI flows
Run a small matrix on PRs and widen via groups or shards on nightly or release pipelines
Use a MainDispatcherRule + TestDispatcher (runTest { }, advanceUntilIdle(), kotlinx-coroutines-test) for every ViewModel and StateFlow test — Dispatchers.setMain(UnconfinedTestDispatcher()) hides off-main crashes that only surface in production
Run a release-variant smoke test in CI that exercises at least one @Serializable endpoint per data class, so R8 full-mode stripping of kotlinx-serialization $serializer classes is caught before release (see software-android-runtime-debugging/references/proguard-r8-triage.md)
For Compose tests, assert on semantic state — onNodeWithTag(...).assertTextEquals(...) — not on reference equality of UI state objects, because Strong Skipping Mode means the UI may or may not receive the same instance across emissions
Avoid
Thread.sleep() for synchronization
Tests that depend on live backends or time-sensitive external content
Flaky selectors such as localized text or positional-only nodes
Device-wide screenshot assertions on ATD when hardware-rendered fidelity is required
Asserting StateFlow<UiState> reference equality across emissions (assertThat(state).isSameInstanceAs(previous)) — new data class instances from copy() have different references but structurally equal content; test semantic equality, not identity
Running serialization-dependent tests only on debug builds — R8 is off in debug, so debug-only suites cannot catch serialization keep-rule failures
Native Android visual design and Material 3 review
Fact-Checking
Known bugs, regressions, framework/compiler/runtime footguns, and version-specific crash or workaround guidance must be verified against current primary web sources before being treated as current fact.
Use web search or web fetch to verify current external facts, versions, pricing, deadlines, regulations, or platform behavior before final answers.
Prefer primary sources; report source links and dates for volatile information.
If web access is unavailable, state the limitation and mark guidance as unverified.
Learnings Loop
Before applying this skill on a non-trivial task, read learnings.consolidated.md in this directory (and learnings.md if present).
After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.
