android-harness

name	android-harness
description	Drive an Android phone over ADB — tap, type, read.
when_to_use	Use whenever the user wants something done ON their phone — open an app on my phone, what's on my phone screen, check my phone, tap that, type that on the phone, swipe, take a phone screenshot, navigate to somewhere in Maps, read my notifications or an SMS, or any task that only exists in a mobile-only app. Needs a USB-connected Android with USB debugging authorised.
version	0.1.0
license	MIT
platforms	["windows"]
allowed-tools	Bash(android-harness *)
metadata	{"hermes":{"tags":["android","adb","phone","device-control","automation"],"requires_toolsets":["terminal"]}}

android-harness

Drive a real Android phone from this PC over ADB. Nothing is installed on the phone.

When to Use

• The user asks for something to happen ON their phone: open an app, read the screen, tap, type, swipe, navigate a flow.
• The task only exists in a mobile app (no web or desktop equivalent).
• The user wants to read a notification, an SMS, or an app's on-screen state.

Don't use for: anything achievable on the desktop or the web — those are faster, cheaper and need no device. This is for phone-only work.

Usage

Pipe a Python program in; helpers are already in scope. One turn = one script = one screenshot, so do several steps per invocation rather than one call per tap:

android-harness <<'PY'
launch("com.android.settings")
wait_stable()
obs = observe(rung=2)
print(obs.compact)
tap_text("Network & internet")
PY

Check the connection first if anything looks wrong: android-harness --doctor

The loop

1. observe(rung=1) — pixels + state. Cheap (~800 tokens). This is the default.
2. Act: tap_text(...), tap(x, y, frame), type_text(...), launch(pkg, uri).
3. wait_stable() — never sleep().
4. observe() again to verify. The capture is the ground truth.

Escalate with observe(rung=2) for numbered tappable elements, rung=3 for the full tree, rung=4 for OCR. Every observation reports its cost in obs.receipt — check it, and do not escalate without a reason.

Rules

• Prefer launch(pkg, uri="...") deep links over tapping through menus: they skip whole sequences and are immune to layout drift.
• Use tap_text rather than raw coordinates when a label exists. It resolves the label to its nearest clickable ancestor, which is usually a different node than the one holding the text.
• Irreversible actions (pay, send, delete) are gated and will prompt the human. Do not try to route around the gate.
• A black screen means the app set FLAG_SECURE. That is respected, not bypassed: check obs.capture_protected, then use the tree or ask the human.
• type_text is ASCII-only on this version. Non-ASCII (Georgian, emoji) raises a clear error — ask the human to type it. Reading non-ASCII works fine.