# android-cli-ui-automation-skill
An [Agent Skill](https://agentskills.io/) that lets an AI agent automate UI interactions with an Android app on a connected device or emulator, using the Android CLI.
The agent follows a **launch → observe → act → re-observe** loop:
1. Launch — `android run --apks=<path>` for APKs, or `adb shell monkey` / `am start` for installed apps / URLs.
2. Observe — `android layout` to pull the UI tree as JSON. Match nodes by `text`, `content-desc`, or `resource-id`, and use each node's `center` as tap coordinates. Fall back to `android screen capture --annotate` + `android screen resolve` when `layout` is ambiguous.
3. Act — `adb shell input tap|text|swipe|keyevent …`.
4. Re-observe after every action and repeat until the goal is reached.
If multiple devices are connected, pass `--device=<serial>` to `android …` and `-s <serial>` to `adb …` (get the serial with `adb devices -l`).
## Install
Use the [skills CLI](https://github.com/vercel-labs/skills):
```bash
npx skills add takahirom/android-cli-ui-automation-skill
```
Pick your agents when prompted; the CLI copies the skill into each selected agent's skills directory.
> [!NOTE]
> In the "Additional agents" list, Claude Code and others are **not selected by default** — press **Space** to toggle (`○` → `●`) before pressing Enter. Hitting Enter while the row is only highlighted (`❯ ○`) confirms without selecting, and the skill ends up in `~/.agents/skills/` only, which Claude Code does not read.
## Requirements
- [Android CLI](https://developer.android.com/tools/agents/android-cli) with `screen capture --annotate` / `screen resolve` support
- `adb` on `PATH`
- A connected device or running emulator
## Try it
In an agent chat, ask something like:
> Launch my debug APK and tap the "Sign in" button.
The agent should activate the skill and drive the UI for you.
README の元ファイルを読む