Run any Skill in Manus with one click

android-control

Stars4

Forks0

UpdatedJune 18, 2026 at 09:41

Drive an Android emulator or USB-connected device reliably from the command line without guessing pixel coordinates. Reads the UI hierarchy (uiautomator) to tap elements by resource-id, text, content-desc, or index — bounds, screenshots, and `input tap` share one coordinate space, so taps land on target. Screenshots are used only to understand the screen; pixel taps are a documented fallback for screens with no accessibility tree. Use when automating, testing, or operating an Android app over adb, when a tap "lands in the wrong place", or when the user mentions adb, an emulator, a connected Android phone, uiautomator, or UI automation. Bundles a deterministic CLI (scripts/android_ctl.py) for shot/dump/tap/text/key/swipe.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

Hayao0819

Hayao0819/dotfiles

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Software DevelopersComputer and Mathematical Occupations·SOC 15-1252

File Explorer

4 files

SKILL.md

readonly

name	android-control
description	Drive an Android emulator or USB-connected device reliably from the command line without guessing pixel coordinates. Reads the UI hierarchy (uiautomator) to tap elements by resource-id, text, content-desc, or index — bounds, screenshots, and `input tap` share one coordinate space, so taps land on target. Screenshots are used only to understand the screen; pixel taps are a documented fallback for screens with no accessibility tree. Use when automating, testing, or operating an Android app over adb, when a tap "lands in the wrong place", or when the user mentions adb, an emulator, a connected Android phone, uiautomator, or UI automation. Bundles a deterministic CLI (scripts/android_ctl.py) for shot/dump/tap/text/key/swipe.
allowed-tools	Bash, Read

android-control

Reliable, repeatable Android automation over adb. The hard problem this skill solves: when a model looks at a screenshot and taps the pixel it sees, the tap lands in the wrong place — the screenshot the model sees is downscaled, so its pixel coordinates do not match the device's. The fix is to never tap by guessed pixels. Instead read the UI hierarchy, pick an element, and let the tool compute the exact centre of that element's bounds. bounds, screencap, and input tap all use the same logical-pixel coordinate space, so this always hits the target.

The one rule

Screenshots are for understanding the screen ("what is on screen, what should I do next"). Taps come from dump (structured UI data), not from reading pixels off a screenshot. Only drop to pixel taps when a screen genuinely has no accessibility tree (see Fallback).

Setup check

python3 scripts/android_ctl.py devices   # confirm a device is online
python3 scripts/android_ctl.py info      # serial, screen size, density

If several devices are connected, pass -s <serial> to every command (or set ANDROID_SERIAL). A USB device needs USB debugging enabled and the host authorised; an emulator just needs to be running.

Core loop

See — capture a screenshot and Read it to decide the next action:
```
python3 scripts/android_ctl.py shot --out screen.png
```
Read screen.png to understand state. Do not read coordinates off it.
List — dump the interactable elements:
```
python3 scripts/android_ctl.py dump
```
Output is a numbered list: [3] Button "Log in" @(540, 1505) id=login_btn. The dump is cached, so tap --index refers to exactly this listing.

Act — tap the element you chose. Prefer the most stable selector:

python3 scripts/android_ctl.py tap --id login_btn      # most stable
python3 scripts/android_ctl.py tap --text "Log in"     # visible text
python3 scripts/android_ctl.py tap --desc "Search"     # icon buttons
python3 scripts/android_ctl.py tap --index 3           # from the dump above

Selector preference: --id > --desc > --text > --index. Resource-ids survive layout and language changes; text does not.

Verify — re-shot or re-dump and confirm the screen changed as expected before the next step. If nothing changed, the element may not have been tappable — re-dump and pick again. Give the UI a moment to settle between an action and the next dump.

Typing, keys, scrolling

python3 scripts/android_ctl.py text "hello world"   # ASCII only; spaces handled
python3 scripts/android_ctl.py key enter            # back/home/enter/del/tab/...
python3 scripts/android_ctl.py swipe up             # scroll content up
python3 scripts/android_ctl.py swipe down --dur 700 # slower = less overshoot

Tap the target field first so it has focus, then text. input text cannot send Japanese, emoji, or other non-ASCII — the tool refuses these and points to the ADBKeyBoard recipe in the reference.

Fallback: screens with no accessibility tree

Games, Canvas-drawn UIs, FLAG_SECURE screens, and Flutter without semantics return no usable dump. Only then tap by pixel — and account for screenshot downscaling. See reference/adb-recipes.md for the scale-correction formula, tap --xy/--scale, the full keycode table, ADBKeyBoard for non-ASCII input, and troubleshooting.

Notes

All paths use forward slashes; run commands from the skill directory or pass an absolute path to android_ctl.py.
The tool targets one device at a time. Be explicit with -s whenever more than one device is attached so actions don't hit the wrong one.
This skill operates a real device/emulator: confirm with the user before destructive in-app actions (purchases, deletions, sending messages).

More from this repository

same repository

writing-tech-article

Hayao0819/dotfiles

Drafts fact-based technical articles for hayao's outlets — Zenn (@hayao), Qiita (@Hayao0819), the personal blog (hayao0819.com) — matching his structural register without faking his personality. Use when writing, drafting, or rewriting a 技術記事 / blog post / Zenn / Qiita article in hayao's name about a technical topic. The LLM writes facts only: it does NOT manufacture his jokes, 毒舌, 自虐, casual throwaway closers, or first-person anecdotes — performing his personality is fabrication and he finds it offensive. Encodes the factual baseline of his writing (genre-driven 敬体/常体, direct openings, plain content-label headings, inline-code density, primary-source links) in a bundled reference.md, and composes with the natural-writing skill. Self-contained — does not depend on machine-local memory. Do NOT use for: non-hayao docs, pure code, commit messages, or chat replies.

2026-06-264

natural-writing

Hayao0819/dotfiles

Rewrites prose so it reads as if a human wrote it — stripping out LLM tells in both English and Japanese. Use whenever producing user-visible writing of any kind: documentation, README, markdown, blog posts, commit messages, PR descriptions, GitHub Issues, code-review comments, code comments and docstrings, replies in issue threads, professional or casual emails, Slack/Discord/chat messages, release notes, design docs, academic-style writeups and 論文. Use it even when the user did not ask for "natural" prose — any drafted text leaving Claude that a human will read should pass through this checklist first. Triggers include: any request to "write", "draft", "compose", "summarize", "explain", "describe", "review", "comment", "reply", or 「書いて」「文章作って」「まとめて」「メール作って」「返信して」「PR/Issueの説明書いて」. Do NOT use for: pure code generation with no prose, raw data transformations, or terse tool output.

2026-06-254

auditing-c-memory

Hayao0819/dotfiles

Security audit of memory-safety vulnerabilities in C/C++ (userland/systems, not kernel), with confidence-gated reporting to keep false positives low. Taint-traces untrusted input (argv, env, files, network, stdin) to memory sinks (copy length, array index, allocation size, format string) and reports only when input is attacker-controlled AND reaches the sink AND bounds aren't enforced. Covers buffer overflow, use-after-free, double-free, OOB read, integer overflow into undersized alloc, format string, the dangerous-function table, hardening flags to verify, and the ASan/UBSan/MSan/fuzzing toolchain. Covers userland/systems C and the SGX enclave/host (ECALL/OCALL) pointer boundary. Use when reviewing C/C++ for memory-corruption bugs. Kernel code → auditing-kernel.

2026-06-224

auditing-kernel

Hayao0819/dotfiles

Security audit of Linux kernel code and out-of-tree modules/drivers, with confidence-gated reporting to keep false positives low. Traces attacker input from userspace entry points (syscalls, ioctl, sysfs/proc/debugfs writes, netlink, char-dev, mmap) — and, for confidential-computing guests, from the untrusted host — to memory/refcount/locking/infoleak sinks. Covers UAF via refcount imbalance, the copy_to/from_user boundary, double-fetch TOCTOU, missing capability checks, integer-overflow allocations, ioctl handler review, and the TDX/SEV-SNP/SGX threat model. Use when reviewing kernel C, a driver/ LKM, an ioctl handler, or enclave/attestation code for vulnerabilities.

2026-06-224

auditing-web-app

Hayao0819/dotfiles

Security audit of web applications — backend implementation and authentication/ authorization flows — with confidence-gated reporting to keep false positives low. Traces attacker-controlled request input (params, body, headers, cookies, path) to sinks, distinguishing server-controlled from attacker-controlled, and reports HIGH only. Leads with broken access control / IDOR (OWASP #1), then injection (and where GORM/sqlc/React are already safe), XSS, SSRF, CSRF, auth (password storage, sessions, reset), and tokens (JWT alg confusion, OAuth/OIDC PKCE). Framework-aware for Go/Gin/GORM/sqlc and Next.js Server Actions. Use when reviewing a web backend, API, route handler, Server Action, or auth flow.

2026-06-224

writing-go

Hayao0819/dotfiles

Idiomatic, version-current Go review and authoring for an experienced engineer. Covers deliberate error wrapping (errors.Is/As/Join, AsType), structured concurrency (errgroup, context, the loopvar fix), modern stdlib that replaced old patterns (slices/maps/cmp, log/slog, min/max/clear, range-over-func iterators), table-driven and synctest testing, API design the linter can't check, and Gin/GORM/cobra backend patterns. Use when writing, reviewing, or refactoring Go — .go files, go.mod, goroutines, errors.Is/As, a Go backend or CLI — or before committing Go. Flags outdated pre-1.22 idioms (loopvar v:=v shadows, tools.go, hand-rolled helpers).

2026-06-224

name	android-control
description	Drive an Android emulator or USB-connected device reliably from the command line without guessing pixel coordinates. Reads the UI hierarchy (uiautomator) to tap elements by resource-id, text, content-desc, or index — bounds, screenshots, and `input tap` share one coordinate space, so taps land on target. Screenshots are used only to understand the screen; pixel taps are a documented fallback for screens with no accessibility tree. Use when automating, testing, or operating an Android app over adb, when a tap "lands in the wrong place", or when the user mentions adb, an emulator, a connected Android phone, uiautomator, or UI automation. Bundles a deterministic CLI (scripts/android_ctl.py) for shot/dump/tap/text/key/swipe.
allowed-tools	Bash, Read

android-control

The one rule

Setup check

python3 scripts/android_ctl.py devices   # confirm a device is online
python3 scripts/android_ctl.py info      # serial, screen size, density

Core loop

See — capture a screenshot and Read it to decide the next action:
```
python3 scripts/android_ctl.py shot --out screen.png
```
Read screen.png to understand state. Do not read coordinates off it.
List — dump the interactable elements:
```
python3 scripts/android_ctl.py dump
```
Output is a numbered list: [3] Button "Log in" @(540, 1505) id=login_btn. The dump is cached, so tap --index refers to exactly this listing.

Act — tap the element you chose. Prefer the most stable selector:

python3 scripts/android_ctl.py tap --id login_btn      # most stable
python3 scripts/android_ctl.py tap --text "Log in"     # visible text
python3 scripts/android_ctl.py tap --desc "Search"     # icon buttons
python3 scripts/android_ctl.py tap --index 3           # from the dump above

Selector preference: --id > --desc > --text > --index. Resource-ids survive layout and language changes; text does not.

Verify — re-shot or re-dump and confirm the screen changed as expected before the next step. If nothing changed, the element may not have been tappable — re-dump and pick again. Give the UI a moment to settle between an action and the next dump.

Typing, keys, scrolling

python3 scripts/android_ctl.py text "hello world"   # ASCII only; spaces handled
python3 scripts/android_ctl.py key enter            # back/home/enter/del/tab/...
python3 scripts/android_ctl.py swipe up             # scroll content up
python3 scripts/android_ctl.py swipe down --dur 700 # slower = less overshoot

Tap the target field first so it has focus, then text. input text cannot send Japanese, emoji, or other non-ASCII — the tool refuses these and points to the ADBKeyBoard recipe in the reference.

Fallback: screens with no accessibility tree

Notes

All paths use forward slashes; run commands from the skill directory or pass an absolute path to android_ctl.py.
The tool targets one device at a time. Be explicit with -s whenever more than one device is attached so actions don't hit the wrong one.
This skill operates a real device/emulator: confirm with the user before destructive in-app actions (purchases, deletions, sending messages).