meshtastic-e2e

# Meshtastic Device ↔ App E2E Testing Two independent planes, each with its own **stimulus** and **oracle**. A closed-loop test stimulates one plane and asserts on the *other*. | Plane | Stimulus | Oracle (observation) | |---|---|---| | **Device** (Meshtastic MCP) | `send_text`, `set_config`, `push_fake_nodedb`, `send_input_event`, `uhubctl_cycle`, traceroute | recorder `packets_window` / `telemetry_timeline` / `logs_window` / `events_window`, `device_info`, `list_nodes`, serial logs, `capture_screen`+OCR | | **App — Android** (android CLI + adb) | `adb shell input tap/text/swipe`, `android run` | `android layout --diff`, `android screen capture [--annotate]` | | **App — Android, agent-driven** (Artemis MCP) | `mobile_run_task` runs a whole journey out of context | **none — never its own oracle**; the device plane decides (see `references/journeys.md`) | | **App — Apple** (xcrun simctl + idb) | `idb ui tap/text`, `simctl install/launch` | `idb ui describe-all`, `simctl io screenshot` | | **App — Desktop** (Compose Hot Reload MCP) | `click` / `type_text` / `scroll` (by `nodeId`) | `get_semantic_tree`, `get_logs`, `get_ui_error` | ## Reference files (load the one you need) - `references/topology.md` — two-radio wiring, app-over-TCP, the uhubctl USB switch, preflight. - `references/harness.md` — the hard rules (marker tokens, bounded polling, recorder oracle, adb nav primitives, verdict format). **Read this before any loop.** - `references/mesh_e2e.py` — runnable device-plane helper (`devices`/`info`/`send`/`recv-text`/`watch-tx`/`traceroute`/`recorder`); emits grep-able `PASS …`/`FAIL …` lines. - `references/loop-inbound.md` — device → app message render - `references/loop-outbound.md` — app → device wire truth - `references/loop-node-sync.md` — node appears in app node list - `references/loop-config-writeback.md` — app setting persists on device (RAM + NVS) - `references/loop-resilience.md` — uhubctl power-cycle fault injection - `references/emulator-lab.md` — hardware-free Android: AVD app over TCP + native UDP mesh - `references/replay-app-features.md` — **drive app behavior via the replay engine**: feature injection (geofence/waypoints/alerts via `replay_inject`), sustained-load/stress (`conference-stress` preset, `duration`/`rate` pacing, self-verifying `achieved_rate`), mDNS/Bonjour auto-discovery, traceroute-log population, and tapback/bot-plane reactions; logcat/notification oracles + gotchas - `references/simulator-apple.md` — hardware-free Apple: iOS Simulator / macOS app over TCP (`127.0.0.1`) + native UDP mesh - `references/journeys.md` — **journey-driven UI (recommended over hardcoded coordinates)**: drive the app from a goal via the live a11y tree, version-resilient. Ships journey XML in `references/journeys/`. For Android, hand the journey to **Artemis** (`mobile_run_task`) rather than looping inline — same XML, runs out of context. Its verification is fail-open, so the recorder stays the verdict. - `references/vision-oracle.md` — use a screenshot + vision as the assertion when the a11y tree is empty (WebView/Canvas/animation). - `references/triage.md` — dual-plane root-cause analysis of a FAIL (pairs with the `triage_e2e_failure` MCP prompt). The device plane (`mesh_e2e.py`, native nodes, recorder) is platform-neutral and shared; only the app plane differs (Android CLI+adb, Apple `xcrun simctl`+`idb`, or the Compose Hot Reload MCP server for desktop). ## Fast app bring-up — use this before any manual UI driving Never hand-walk onboarding and the manual-IP dialog on a debug build; the app has a scripted path (and hand-walking it is how sessions waste ten minutes of taps): ```bash adb shell am start -n <pkg>/org.meshtastic.app.MainActivity \ --ez skip_onboarding true \ -a android.intent.action.VIEW -d "https://meshtastic.org/connections?address=t10.0.2.2:4403" ``` - `skip_onboarding` is MainActivity's debug-only `EXTRA_SKIP_ONBOARDING` — bypasses the whole intro flow on a fresh install (release builds ignore it). - `/connections?address=t<host>:<port>` auto-connects (transport prefixes: `t` TCP, `x` BLE MAC, `s` serial). `address=n` disconnects. Full path list: `docs/en/developer/navigation-and-deep-links.md` in the app repo. - Builds after 2.8.1 pop a trust dialog ("Connect to this device?") on link-initiated connects — one exact-match tap on **Connect**. `avd.connect_app_to_tcp()` does all of this (bypass, deep link, dialog, fallback taps) — prefer it over raw adb. - Driving Settings/UI with a sim attached? Start it with `sim_profile={"traceroute_pairs_per_hour": 0}` — the default sim injects traceroute responses that pop a modal over whatever you are testing (`replay_stop` also works once the node DB is populated; the app keeps its nodes). ## Topology (read first) Three supported topologies — pick the one that matches your hardware: | Topology | Device plane | App plane | DUT address | |---|---|---|---| | **Emulator lab** (no hardware) | `meshtasticd` native nodes via UDP multicast | Android AVD | `10.0.2.2:<port>` | | **Physical Android** | Real radios via USB serial | USB-attached Android phone | `adb reverse` → `127.0.0.1:<port>` | | **Physical Apple** | Real radios via USB serial | iOS Simulator only (see caveat) | `127.0.0.1:<port>` | | **Desktop app** | Real radios via USB serial, or native nodes over UDP | Compose `:desktopApp` on the host | `127.0.0.1:<port>` | For **physical Android**: `avd.tcp_dut_address(port, serial=<phone_serial>)` sets up the `adb reverse` tunnel automatically and returns the correct address. Pass it to `connect_app_to_tcp()`. UI observation falls back from `android layout` to `adb exec-out uiautomator dump` transparently — all helpers (`poll_for_text`, `find_text`, `_tap_text`) work on both emulators and physical phones. > **iOS physical devices — not supported.** `apple_sim.py` targets the iOS Simulator only > (`xcrun simctl`, `idb`). Physical iPhone/iPad requires code signing + provisioning profiles, > `libimobiledevice`/`usbmuxd` for device comms, and XCTest/XCUITest for UI automation > (idb is simulator-only). This is a separate, significant project — not a configuration change. Full wiring detail and the single-radio app-over-TCP workaround in `references/topology.md`. One radio cannot be both tester and DUT (the serial port lock is exclusive). ### Desktop app plane (Compose Hot Reload MCP) CMP 1.12+ is what bundles the MCP-capable Compose Hot Reload, and `Meshtastic-Android` is on 1.12.0 — so this plane is available there today. Its tracked `.mcp.json` registers the server as `compose-hot-reload` (`:desktopApp:hotMcpServer`), so a session opened in that repo drives the **live** desktop app with no rebuild between assertions. Start it with `./gradlew :desktopApp:hotRun`, then: - **Poll `status` until `connected: true`** before any other call — the server accepts requests before the app has connected to it. - **`get_semantic_tree` is the oracle:** roles, text, `selected`/`focused`, actions and bounds — everything the journey-driven approach in `references/journeys.md` needs. `click` / `type_text` / `scroll` address nodes by `nodeId` from that tree. `take_screenshot` exists but its output depends on the host renderer; prefer the tree. - **`reload`** applies source edits into the running app (`await_reload` when started with `--auto`), so a fix can be re-asserted without restarting the mesh setup. The device plane is unchanged: same radios, same recorder, same `mesh_e2e.py`. ## Prerequisites Before running any loop, verify all of these. Missing items cause silent failures, not clear errors. | Requirement | Check | How to fix | |---|---|---| | `MESHTASTIC_FIRMWARE_ROOT` set | `echo $MESHTASTIC_FIRMWARE_ROOT` | Set to your firmware checkout path | | Two radios (TESTER + DUT) | `list_devices()` → ≥2 ports with `likely_meshtastic=true` | Plug in both radios; use TCP DUT if only one physical radio | | App reachable on the plane you are testing | Android: `adb shell pm list packages \| grep meshtastic` · Apple: `xcrun simctl listapps` · Desktop: MCP `status` → `connected: true` | `android run` / `adb install`; `simctl install`; `./gradlew :desktopApp:hotRun` | | Recorder running (process-global) | `recorder_status()` → `running=true` | Auto-starts on first MCP serial call; captures every interface | | `uhubctl` available (Loop 5 only) | `uhubctl -l` returns hub info | `brew install uhubctl`; set `MESHTASTIC_UHUBCTL_LOCATION_TESTER` | | `doctor()` returns `ok=true` | Call `doctor()` | Run each `fix_commands` entry in order | ## Bootstrap ```bash export MESHTASTIC_FIRMWARE_ROOT="$HOME/meshtastic/firmware" MCP="$MESHTASTIC_FIRMWARE_ROOT/mcp-server/.venv/bin/python" S="$HOME/.agents/skills/meshtastic-e2e/references/mesh_e2e.py" $MCP "$S" devices # list tester radios TESTER=/dev/cu.usbmodem101 # pick one adb devices # confirm the phone (DUT) is attached adb shell pm list packages | grep meshtastic # confirm the app is installed ``` The MCP server (registered as user-scope `meshtastic`) exposes the 53 tools directly when running inside Claude Code; the `$MCP "$S" …` helper is the standalone/CI path. ## Hard rules (mesh is async + lossy — respect these or get flaky tests) 1. **Marker token per message.** Never assert on a bare "hello". Embed a unique token (`E2E-$(date +%s)-$RANDOM`) so a busy 80-node mesh can't false-positive your grep. 2. **Bounded polling, never `sleep N` then assert once.** Poll every 1 s up to a deadline. Use these ceilings: single-hop broadcast → 20 s; directed/PKI send → 30 s; multi-hop (≥2) → 45 s. Mesh delivery is best-effort; these cover the 99th-percentile lab case. 3. **Warm up directed/PKI sends.** Directed + encrypted sends need bilateral NodeInfo (both sides hold each other's current pubkey). Broadcast (`^all`) first to exchange, or send to a node already in both DBs. 4. **The recorder is the device-side source of truth.** It timestamps every RX packet to JSONL; align those timestamps with `android layout` snapshots. Start it before the stimulus, query the window after. 5. **One MCP call per serial port at a time** (exclusive lock): open → act → close. 6. **`layout` can fail on WebView/animation** — fall back to `screen capture --annotate` + visual/OCR inspection. ## Loop 1 — inbound message (device → app) Stimulate from the tester radio; assert the bubble renders in the app. ```bash TOKEN="E2E-$(date +%s)" # 1. device stimulus: tester radio broadcasts (or directs to the DUT node) $MCP -m meshtastic_mcp.cli ... # or drive send_text via the meshtastic API: $MCP -u -c " import meshtastic.serial_interface as si i=si.SerialInterface('$TESTER'); i.sendText('$TOKEN', wantAck=False); i.close()" # 2. app oracle: poll the UI tree for the token (bounded) for t in $(seq 1 30); do android layout 2>/dev/null | grep -q "$TOKEN" && { echo "PASS: rendered"; break; } sleep 1 done ``` Open the messages screen in the app first (`adb shell input` to navigate, or `android run` to the messages activity). Use `layout --diff` to keep only the changed bubble in context. ## Loop 2 — outbound message (app → device) Type+send in the app; assert the wire truth on the tester radio's recorder. 1. Start the recorder, then drive the app: ```bash # ensure recorder is capturing (it auto-starts when the MCP server is live; # standalone: open a SerialInterface to TESTER and call get_recorder().start()) ``` 2. App stimulus — focus the compose field, type the token, tap send: ```bash android layout --pretty | jq '.[] | select(.interactions|index("focusable"))' # find input adb shell input tap <cx> <cy> # focus compose field (must show "focused") adb shell input text "$TOKEN" adb shell input tap <send_cx> <send_cy> ``` 3. Device oracle — the tester must *receive* it; query the recorder window: ```bash $MCP -c "from meshtastic_mcp import log_query as q,json; print(json.dumps(q.packets_window(max=20)))" | grep -i "TEXT_MESSAGE_APP" ``` Decode the payload hex / matched text for `$TOKEN`. PASS when a `TEXT_MESSAGE_APP` packet from the DUT node carrying the token lands within the deadline. ## Loop 3 — node sync (device → app) `push_fake_nodedb` (or a real beacon) on the DUT radio → assert the node appears in the app's node list (`android layout` over the nodes screen, match long_name/short_name). Backstop the device truth with `list_nodes`. ## Loop 4 — config write-back (app → device) Change a setting in the app (e.g. region, device role, a channel name) → assert it persisted on the DUT radio: ```bash $MCP -c "from meshtastic_mcp import admin,json; print(json.dumps(admin.get_config('lora', port='$DUT')))" ``` Reboot the radio (`reboot` MCP tool) and re-read to prove NVS persistence, not just RAM. ## Loop 5 — resilience / fault injection Mid-conversation, power-cycle a relay or the peer with `uhubctl_cycle` (needs `uhubctl`): assert the app shows the node go **offline → online** and that a queued message recovers once the path heals. This is the app-facing mirror of the firmware suite's `test_peer_offline_recovery`. ## Loop 6 — ATAK/iTAK render (sim TAK squad → TAK client) Validates the TAK plane. Meshtastic apps (≥2.8) bridge mesh TAK traffic to a connected ATAK/iTAK client via an **in-app local TAK server** that emits CoT (the deprecated `IMeshService` plugin is gone). Two layers: - **Bridge-semantics (no emulator, in CI):** the sim emits a TAKPacketV2 squad (`replay_start(source=..., sim_profile={"tak": {"team_nodes": N, "wire": "v2"}})` — v2 rides portnum 78 `ATAK_PLUGIN_V2`; v1 rides 72). `replay/tak_server.py` `capture_to_cot_events()` reproduces the bridge's wire→TAKPacketV2→CoT path; `tests/unit/test_tak_bridge.py` asserts the CoT is well-formed, typed (`a-f-G-U-C` PLI), and carries the right callsign/position/GeoChat. Needs the `[tak]` extra (meshtastic-tak SDK). - **App-plane (opt-in, needs an emulator + ATAK-CIV), bidirectional:** `scripts/ci_atak_app_loop.py` stands up `CotTakServer` from that squad, points ATAK-CIV at `10.0.2.2:<port>` (a pushed streaming-input `.pref`), launches it, and asserts (receive) a squad callsign marker renders **and** (send) ATAK's own self-PLI streams back, is captured, and converts to a mesh TAKPacketV2.