| name | test-harness |
| description | Run, debug, and extend the MobileIoT cross-platform test harness (Python `harness/` + .NET `TestHarness` build configuration). Use when invoking pytest tiers, building TestHarness binaries, adding a new integration plugin, or interpreting harness JSON output. |
MobileIoT test harness
The harness drives end-to-end tests across phones and a Raspberry Pi from a Windows or Mac dev box. The Python framework lives under harness/; the in-app HTTP host lives in the shared .NET project under the TestHarness build configuration.
When to use
- Running any tier of harness tests (
mock, hardware, auto)
- Building the TestHarness APK / IPA / MSIX
- Adding a new integration (BLE, NFC, WiFi Direct, etc.)
- Debugging why a hardware-tier test is skipping
- Reading or producing the harness JSON event stream
Layout
harness/harness/ framework code (cli, inventory, transports, app, integrations, reporting)
harness/tests/ self-tests for the harness (mock tier)
harness/tests/integrations/ per-integration hardware-tier tests (test_<name>_hw.py)
src/MobileIoT/QiMata.MobileIoT.Shared/Services/TestHarness/
HarnessHttpHost, IScenario, scenario implementations, DI extension
devices.yaml committed inventory (no secrets)
devices.local.yaml per-dev-box overrides (gitignored)
.env.harness secrets (gitignored); .env.harness.example is the template
harness/runs/ run artifacts + build cache (gitignored)
CLI cheatsheet
Run from repo root.
python -m harness doctor
python -m harness run --tier mock|hardware|auto --app maui|uno|both [--json]
python -m harness run --integration <name> --tier hardware --app <variant>
python -m harness build --app maui|uno --platform android|ios|windows
python -m harness repl
Exit codes: 0 = green, 1 = real failure, 2 = hardware skipped (offline / wrong host).
JSON mode (--json) emits one event per line on stdout — use for agent-driven runs.
Adding a new integration
The framework is pluggable; do not edit harness/harness/ core files when adding integrations — entry-points auto-discover plugins.
Four touch points (see project_test_harness.md in memory for design rationale):
- Plugin —
harness/harness/integrations/<name>.py, subclass IntegrationPlugin from harness.integrations.base. Use _smoke.py as a template.
- Entry point — register in
harness/pyproject.toml under [project.entry-points."mobileiot_harness.integrations"].
- Scenario (only if the app must respond) —
src/MobileIoT/QiMata.MobileIoT.Shared/Services/TestHarness/Scenarios/<Name>Scenario.cs implementing IScenario; register it in HarnessServiceCollectionExtensions.
- Tests — mock-tier alongside existing mocks (
src/MobileIoT/QiMata.MobileIoT.Tests/ or src/pi/tests/); hardware-tier at harness/tests/integrations/test_<name>_hw.py using the android_phone / ios_phone / windows_local / pi fixtures with @pytest.mark.hardware(roles=[...], capabilities=[...]).
Gating: hardware tests auto-skip when inventory can't satisfy roles/capabilities; the reason surfaces in the JSON reporter. iOS tests skip cleanly on Windows hosts (reason_offline = "ios-requires-macos").
Build configuration
dotnet build -c TestHarness defines TEST_HARNESS + DEBUG on QiMata.MobileIoT, QiMata.MobileIoT.Uno, and QiMata.MobileIoT.Shared.HarnessHttpHost (127.0.0.1:47821) is compiled under #if TEST_HARNESS and started by MauiProgram.cs / App.xaml.cs only when MIOT_TEST_MODE=1 (env or Android intent extra) — Uno also accepts --miot-test-mode on the command line. Stripped from Release builds.- App binaries are built on demand by
harness/harness/app/builder.py and cached under harness/runs/.cache/ by git SHA + dirty-tree hash.
Transports
USB-only, no LAN SSH:
- Android: ADB (
adb forward for port forwarding).
- iOS: pymobiledevice3 usbmux (Mac dev box only).
- Pi: paramiko SSH tunnel over USB-gadget ether (e.g.
192.168.7.2).
- Windows: local execution of the published Uno executable.
Pointers
harness/README.md — quick-start, app selection, build pairs.devices.yaml — committed inventory shape..env.harness.example — required secret keys.- Memory:
project_test_harness.md (design decisions), MEMORY.md (project overview).
tools/run_checks.sh — baseline CI script (Python unit tests + dotnet test QiMata.MobileIoT.Tests); the harness CLI is the integration-level entry point.
