efficiency-test-authoring

name

efficiency-test-authoring

description

Author or convert Fenix Android UI tests onto the ui/efficiency framework. Use this whenever the task involves writing a new efficiency UI test, converting/porting a legacy ui/ (robot-based) test onto the efficiency framework, or building what a test needs — page objects, selectors, navigation-graph nodes/edges, or BasePage primitives. Trigger even when the user doesn't name the framework: "convert this smoke test", "add a page object for the bookmarks screen", "wire up navigation to Settings", "write an efficiency test for X" all qualify. Do NOT use for edits to legacy robot-based tests that are staying on the old framework, or for non-Android test work.

Efficiency test authoring

This skill turns "I need this UI test on the efficiency framework" into a reliable, repeatable build. Its job is to (1) figure out which of three build modes a test needs and (2) route you to the right rule-set for each missing piece. The detailed rules live in-tree — read only the one(s) you need, when you need them.

Framework root in the repo (called <eff> below): mobile/android/fenix/app/src/androidTest/java/org/mozilla/fenix/ui/efficiency/

Every docs/… path in this skill is relative to <eff>. Those docs are the source of truth — read them from the tree rather than relying on a summary here, since the monorepo moves under you.

The host-side eff* scripts this skill drives (effnext, effscaffold, effcheck, effbuild, effverify, effloop, the effwatch bridge) live in the testops-tools repo under tae-conversion/tools/. The device-side dump tools (effview, effpretty) ship in-tree at <eff>/devtools/.

Status: this is a v0 guide we are dogfooding (read this)

This skill is refined as we go — it is not a hard rule set yet. Treat it as the live guide and follow it, but when we agree on something that isn't aligned with what's written here, stop and flag the mismatch, discuss it, then update this skill (and the reference/logs) at that time. New lessons land first in the running logs and get folded up into this skill:

• <eff>/docs/gotchas.md — harness bug catalog + authoring/review checklist (A* bugs, B* checks), distilled and landed in-tree.
• testops-tools/tae-conversion/docs/HARNESS-GOTCHAS.md — the running catalog, ahead of the in-tree copy while entries are still being confirmed.
• testops-tools/tae-conversion/docs/CONVERSION-LESSONS.md — assumption→reality→rule, tagged by whether a tool enforces it. Check those as if they were part of this skill; they are the source the skill distills.

Three build modes (cheapest first)

Every test is one of these. Picking the cheapest workable mode is the point.

1. Factory-generated — a factory emits the test from metadata; you write nothing. Today only the Reachability factory is production-ready (covers "does this page open" for pages already in the graph). Prefer this for pure page-open checks.
2. Hand-composed — write the test by composing existing building blocks (BasePage moz* verbs, page objects, selectors, nav nodes). Default for real smoke tests; usually ~5–20 lines. Reuse first: a lot of capability already exists (custom-tab launch, trust-panel state verify, recently-closed screen, web-form submit + save-login prompt, settings→Home back-edge, screen-dump dev tools). Check before building.
3. Hand-composed + extend the harness — a needed block is missing, so add it first (page object / selectors / nav node / primitive), then compose. Extending is the exception — add the smallest general thing; every extension is reusable by later tests.

The feedback loop (run this per test)

Work the gates in order. At each gate you either compose with what exists or add the missing block, then continue. Navigation is the spine — resolve reachability first. Lean on the tools (below); they make each gate faster and safer than doing it by hand.

0. Pick the next test (local, no network). Run effnext --json — next candidate(s) from the local prioritized pool minus what's done, minus skips, minus anything whose method already exists in the efficiency tests package. Never call the Google Sheet to choose — it's slow and the local pool is the working queue. (The Sheet is systems-of-record for status, not the per-test picker.) If the pick isn't one to take now — too complex for whoever is picking it up, blocked on a harness gap, deliberately deferred — record that rather than stepping over it: effnext --skip Class.method --reason "…" parks it (reversible with --unskip; it never marks the test converted) and prints the new next pick. Fetch main before you pick. Both effnext's in-tree filter and effscaffold's already-converted check read your checkout, so a branch that predates someone else's landing cannot see their conversion — and the duplicate then surfaces as a rebase conflict after review and submission, which is how bug 2060292 ended up duplicating bug 2060174. And heed ⚠ already converted on <branch> (also_on_branches in JSON): the in-tree filter reads only the CHECKED-OUT branch, so a conversion you already sent for review from another branch is still offered as the next pick. It is advisory on purpose — backup/* is excluded and abandoned work lives on branches too — so confirm against SMOKE-CONVERSION-AUDIT.md or Phabricator before redoing it. If branches_unchecked is non-empty, the scan did not complete and its silence proves nothing.
1. Scaffold + extract intent. Run effscaffold <Class.method> --json first — it pulls the legacy body, TestRail id, whether an efficiency test of that name already exists (don't re-convert!), the robots + their selector lines, and which screens are already modeled. From that, write the template: entry state, target page(s), ordered steps, assertions. Keep the what; drop legacy DSL.
2. Navigation gate. Can you route to each target page? Before choosing selectors, discover the real handles an element exposes — never trust a stubbed locator. → docs/guides/discovering-selectors.md (uses effdump). If a page object, its selectors, or a nav edge is missing, build it. → docs/guides/adding-navigation.md, docs/guides/creating-a-page-object.md, docs/guides/authoring-selectors.md. Four traps that each cost a device cycle, all now in HARNESS-GOTCHAS:

Tools (fast + safe; prefer --json when driving them programmatically)

Tool	Use at	Does
effnext	gate 0	Next candidate(s): pool minus done, minus skips, minus what's already in-tree. Warns also_on_branches when a pick is already converted on another local branch (advisory; backup/* excluded), and reports branches_unchecked if that scan did not finish. --skip/--unskip/--skips. Local-only, no network. --json.
effscaffold	gate 1	Legacy body, TestRail, already-converted check, robots+selectors, existing coverage.
effdump / ScreenDump	gate 2	Dumps a screen's real handles in all 3 layers (Compose / Espresso / UIAutomator). Author from ground truth, not stubs.
effcheck	gate 5	Static pre-flight (no device) — resolution, nav, inline selectors, verbs, boilerplate.
effbuild	gate 6	Compile verdict + only the error lines. --json. Read this, not the raw build log.
effverify	gate 6	Done-gate (aggregates every run block, not just the last): ok/clean, failed_total, runs, retried, per-test status incl. retry-pass, and a capped failure_excerpt on failure. Takes METHOD names — a class name yields a bogus not-run/clean:false. --json — the agent reads THIS, never the raw report.
effpretty	(human)	Renders the Eff run log for a person inspecting a run. Not part of the agent's read path.
effwatch bridge	gate 6	Runs the build/run on the engineer's device and returns reports.
efftriage	gate 6	Maps a failed run to the gotcha that explains it, with the fix. Read-only, safe on every failure. When it says "no rule matched", add a rule once you know why rather than routing around it.

Gates 3 (interaction) and 4 (assertion) have no tool — they're code edits (add a moz* verb or a mozVerify* primitive) via docs/guides/extending-basepage.md. effwatch is a persistent bridge you start once, not a per-step tool; effscaffold/effcheck/effbuild/effverify may each run several times per test.

Reference rule-sets

When you need to…	Read
Find an element's real handles before choosing a strategy (effdump, stubs, web ids, state-invariance)	docs/guides/discovering-selectors.md
Reach/route to a screen; add graph nodes/edges; onboarding/launch-flag entries	docs/guides/adding-navigation.md
Model a new screen	docs/guides/creating-a-page-object.md
Add element locators + their groups	docs/guides/authoring-selectors.md
Compose the actual test method	docs/guides/writing-a-test.md
Add a moz* primitive or page-object helper	docs/guides/extending-basepage.md
Run and debug a test (effpretty, ScreenDump, retry-masking, SKIPPED trap)	docs/guides/debugging-tests.md

Guardrails

• Tests describe only the what; the harness owns the how (navigation, retries, selectors).
• Reuse an existing capability before adding one; add the smallest general block, not a test-specific hack.
• Selector priority: Compose testTag → resource id → content-description → text (last resort). Verify handles against the live app UI (via effdump), not from how a legacy robot matched.
• Verify every claim against the live repo; it's a syncing monorepo and state shifts between runs.
• This skill is a soft v0 — when reality and the skill disagree, flag it, discuss, then update the skill.
• You can draft skill content but can't install it from a Cowork session — install via Settings → Capabilities.