running-instrumented-tests-via-adb

name	running-instrumented-tests-via-adb
description	Use this skill to run instrumented Android tests directly through `adb shell am instrument -w -r` without going through Gradle. Covers the required `-w` (wait — REQUIRED for meaningful exit codes) and `-r` (raw output) flags, the `-e` argument table (`class`, `class#method`, `package`, `size`, `numShards`/`shardIndex`, `debug`, `annotation` / `notAnnotation`, `listener`, `clearPackageData`, `targetInstrumentation`), the canonical runners `AndroidJUnitRunner` and `AndroidTestOrchestrator`, the orchestrator wrapping pattern (target = orchestrator, `-e targetInstrumentation <pkg>/<runner>`), and the output framing (`INSTRUMENTATION_STATUS_CODE` 1=start, 0=ok, -1=error, -2=failure, -3=ignored, -4=assumption-failure; `INSTRUMENTATION_RESULT`; `INSTRUMENTATION_CODE`). Use when the user mentions `am instrument`, `AndroidJUnitRunner`, "run tests from CI without Gradle", "Orchestrator", `clearPackageData`, `targetInstrumentation`, exit codes from `am instrument`, or `INSTRUMENTATION_STATUS_CODE`.
license	Apache-2.0. See LICENSE for complete terms.
metadata	{"author":"Jaewoong Eum (skydoves)","keywords":["am-instrument","AndroidJUnitRunner","AndroidTestOrchestrator","clearPackageData","targetInstrumentation","INSTRUMENTATION_STATUS_CODE","test-sharding","numShards","shardIndex","run-listener"]}

Running Instrumented Tests via ADB — am instrument without Gradle

adb shell am instrument is the underlying command Gradle invokes; running it directly is the right tool for CI scripts that already manage their own APKs, for sharding fan-out across many devices, and for tight-loop debugging of a single test method. The pitfall most CI scripts fall into: omitting -w, which makes the exit code meaningless. The second-most-common pitfall: flipping the orchestrator-target relationship.

When to use this skill

• The user wants a single test method to run from a script: adb shell am instrument -w -r -e class com.example.MyTest#myMethod ....
• The user wants to shard a test suite across N devices using numShards / shardIndex.
• The user is wiring AndroidX Test Orchestrator with clearPackageData true and gets the target/targetInstrumentation order confused.
• The user's CI script reports green when tests actually failed because $? is 0 even though INSTRUMENTATION_STATUS_CODE: -2 shows a failure.
• The user wants the on-device runner to wait for a debugger attach before running tests.

When NOT to use this skill

• The user wants to install or reset the app under test — use ../../apps/installing-and-managing-apps/SKILL.md.
• The user wants to choose a device, wait for boot, or set up Wi-Fi debugging — use ../../devices/connecting-to-devices/SKILL.md and ../../devices/connecting-over-wifi/SKILL.md.
• The user is writing the JUnit4 / AndroidJUnit4 test class itself — use ../../../instrumentation/runner/running-instrumented-tests-with-androidjunit4/SKILL.md.
• The user wants Gradle to do the run for them (./gradlew connectedDebugAndroidTest) — that path also reads instrumentation runner args from testInstrumentationRunnerArguments.

Prerequisites

• App APK installed (com.example.app) and test APK installed (com.example.app.test) — see ../../apps/installing-and-managing-apps/SKILL.md.
• The test APK was built with -t allowed (android:testOnly="true").
• For Test Orchestrator: androidx.test.orchestrator APK installed (typically via androidTestUtil("androidx.test:orchestrator:1.6.1") and adb install -r androidx.test.orchestrator.apk).
• Device in device state and animations disabled for stable runs (see docs/CORPUS.md §I.6 hermetic test setup).

Workflow

• 1. Compose the canonical command. Verbatim from developer.android.com/studio/test/command-line and am help:

  adb shell am instrument -w -r \
    [-e <key> <value>] ... \
    <pkg>/<runner>
  

  • -w "Wait for instrumentation to finish before returning. Required for test runners." Without it, the shell returns immediately and $? is meaningless.
  • -r "Print raw results (otherwise decode report_key_streamresult)." Pair with -w for CI-friendly output.

  Common runners:

  • androidx.test.runner.AndroidJUnitRunner — the default for AGP androidTest.
  • androidx.test.orchestrator.AndroidTestOrchestrator — the wrapper that runs each test in its own instrumentation invocation.

  Bare invocation against the AndroidJUnitRunner:

  adb shell am instrument -w -r \
    com.example.app.test/androidx.test.runner.AndroidJUnitRunner
  

• 2. Use the -e argument table for selection, sharding, and runner behavior. The full table understood by AndroidJUnitRunner / AndroidX Test:

  Key	Value	Meaning
  package	<java_package>	"Fully qualified Java package name. Any test class using this package executes." Takes precedence over class. Comma-separate for multiple.
  class	<fqcn>	"Only this test case class executes."
  class	<fqcn>#<method>	"Only this method executes." Hash separator.
  class	<fqcn1>,<fqcn2>#m,...	Comma-separate to combine multiple selectors.
  size	/ /

Patterns

Pattern: WRONG vs RIGHT — exit code without -w

# WRONG
adb shell am instrument -r \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner
echo $?
# 0
# WRONG because: -w is omitted, so the adb shell returns immediately. $?
# reflects the launch, not the test outcome. Failures in the test run are
# silently lost.

# RIGHT — gate on stdout, NOT on $?
output=$(adb shell am instrument -w -r \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner)
if echo "$output" | grep -qE 'INSTRUMENTATION_STATUS_CODE: -[12]$'; then
  echo "FAILED"; exit 1
fi
# am instrument exits 0 even when tests fail (System.exit(0) in AOSP Instrument.java),
# so the only reliable signal is parsing INSTRUMENTATION_STATUS_CODE: -2 (failure) or -1 (error).

Pattern: WRONG vs RIGHT — orchestrator target/targetInstrumentation flip

# WRONG
adb shell am instrument -w -r \
  -e clearPackageData true \
  -e targetInstrumentation androidx.test.orchestrator/.AndroidTestOrchestrator \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner
# WRONG because: the orchestrator is the runner, not the target. AndroidJUnitRunner
# does not know how to invoke an "orchestrator target", and clearPackageData is
# not a recognised arg of AndroidJUnitRunner — only of the orchestrator.

# RIGHT
adb shell am instrument -w -r \
  -e clearPackageData true \
  -e targetInstrumentation com.example.app.test/androidx.test.runner.AndroidJUnitRunner \
  androidx.test.orchestrator/.AndroidTestOrchestrator
# Orchestrator is the runner (last positional arg).
# AndroidJUnitRunner is the wrapped target (passed via -e targetInstrumentation).

Pattern: WRONG vs RIGHT — sharding across two devices

# WRONG
adb -s emulator-5554 shell am instrument -w -r -e numShards 2 -e shardIndex 0 \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner &
adb -s emulator-5554 shell am instrument -w -r -e numShards 2 -e shardIndex 1 \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner &
wait
# WRONG because: both shards run on the same device (-s emulator-5554). They
# serialize against each other through am, defeating the parallelism goal of
# sharding. Worse, they may collide on the same app under test.

# RIGHT
adb -s emulator-5554 shell am instrument -w -r -e numShards 2 -e shardIndex 0 \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner &
adb -s emulator-5556 shell am instrument -w -r -e numShards 2 -e shardIndex 1 \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner &
wait
# One shard per device. Each device runs roughly half the suite.

Pattern: WRONG vs RIGHT — running a single method

# WRONG
adb shell am instrument -w -r -e class com.example.app.LoginTests:login_succeeds \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner
# WRONG because: the separator between class and method is `#`, not `:`.
# AndroidJUnitRunner treats the whole string as a class name and finds nothing.

# RIGHT
adb shell am instrument -w -r -e class com.example.app.LoginTests#login_succeeds \
  com.example.app.test/androidx.test.runner.AndroidJUnitRunner

Mandatory rules

• MUST pass -w to am instrument for any CI/script invocation so the shell waits for the runner to complete (necessary for stdout parsing). MUST NOT gate on $? — am instrument calls System.exit(0) regardless of test outcome.
• MUST pair -w with -r for CI consumption — raw output is parseable; the decoded report_key_streamresult form is not.
• MUST use <class>#<method> (hash separator) for single-method selection. Colons or dots fail silently.
• MUST put the orchestrator as the runner (positional last arg) and the AndroidJUnitRunner as the target via -e targetInstrumentation — not the other way around.
• MUST NOT rely on am instrument exit codes when -w is omitted.
• MUST NOT assume clearPackageData true works without Orchestrator — it is an Orchestrator-only argument.
• PREFERRED: use Test Orchestrator + clearPackageData true for hermetic test isolation in CI; this maps to Gradle's testOptions.execution = "ANDROIDX_TEST_ORCHESTRATOR".
• PREFERRED: disable animations (settings put global *_animation_scale 0) before the run, or rely on Gradle's testOptions.animationsDisabled = true.

Verification

• No script gates on $? from am instrument — it is always 0 (System.exit(0) in AOSP Instrument.java; finish(RESULT_OK, …) in AndroidJUnitRunner). Pass/fail is detected by grepping stdout for INSTRUMENTATION_STATUS_CODE: -1 (error) / -2 (failure).
• Output stream contains exactly one INSTRUMENTATION_CODE: <int> line at the end.
• Per-test frames carry matching INSTRUMENTATION_STATUS_CODE: 1 (start) followed by 0 (ok) or -1/-2/-3/-4.
• When using Orchestrator, the runner positional arg is androidx.test.orchestrator/.AndroidTestOrchestrator and -e targetInstrumentation carries the AndroidJUnitRunner FQN.
• When sharding, -e numShards N -e shardIndex i runs on distinct devices (-s differs per shard).
• -e class <fqcn>#<method> runs exactly one test method (cross-check with INSTRUMENTATION_STATUS: numtests=1).
• Animation scales are 0.0 on the device (adb shell settings get global window_animation_scale).

References

• Run tests from the command line (am instrument): https://developer.android.com/studio/test/command-line
• ADB user guide (am instrument flag table): https://developer.android.com/tools/adb#am
• AndroidX Test Orchestrator: https://developer.android.com/training/testing/instrumented-tests/androidx-test-libraries/runner
• Instrumentation reference (report keys, status codes): https://developer.android.com/reference/android/app/Instrumentation
• Advanced test setup (Gradle Managed Devices, sharding): https://developer.android.com/studio/test/advanced-test-setup
• tasks/research/A2-adb-shell-commands.md — full -e table, status-code values including AndroidJUnitRunner's -3/-4, orchestrator wiring, exit-code-only-with--w rule.
• docs/CORPUS.md §I.5 (am instrument invocation), §I.6 (hermetic test setup), §I.10 (-w exit-code rule).
• Sibling skills:
  • High-level architecture: ../../architecture/understanding-adb-architecture/SKILL.md
  • Connect a device / wait for boot: ../../devices/connecting-to-devices/SKILL.md
  • Wireless ADB: ../../devices/connecting-over-wifi/SKILL.md
  • Install / clear / list apps: ../../apps/installing-and-managing-apps/SKILL.md
• Cross-set neighbours:
  • Run instrumented tests with AndroidJUnit4: ../../../instrumentation/runner/running-instrumented-tests-with-androidjunit4/SKILL.md
  • Configure JUnit4 on Android: ../../../jvm-tests/runner/configuring-junit4-on-android/SKILL.md
  • Source-set strategy: ../../../fundamentals/strategies/organizing-test-source-sets/SKILL.md