test-manual-ui

name	test-manual-ui
description	Walk a single api-wired UC scenario through the browser UI manually: for each step in the source test file, print the UI gesture (page + action), ask you to perform it and report what you observed, then independently probe the backend (REST read-back, DB rows, DataHub aspects, k8s secrets) to confirm the side effect fired. The human-driven sibling of the automated `tests/e2e/` use-case group — same scenarios, with a human at the browser. Optional argument `scope` selects the case (free-form: "UC1", "UC1 Case 2", "passive kafka", filename fragment, etc.).
disable-model-invocation	true
argument-hint	[scope, e.g. "UC1 Case 2"]
allowed-tools	Bash(*), Read, Edit, Glob, Grep, AskUserQuestion, Skill(k8s-deploy)

Purpose

A guided manual harness that walks one tests/integration/api_wired/test_*.py scenario through the browser UI instead of curl. It is the browser-facing sibling of /test-manual-api-wired and the human-driven counterpart of the automated E2E use-case group (tests/e2e/use-case/, spec/TESTING.md §End-to-End (E2E) Testing) — same UC scenarios, run with a human at the browser when you want eyes on the rendered UI rather than an unattended Playwright run.

Use when:

• verifying the reference UI renders and drives each UC end-to-end
• confirming a frontend change still produces the right backend side effects
• exploring how a UC scenario looks to a user, page by page
• catching UI/backend divergence — UI looked right but nothing persisted, or it persisted but the UI never reflected it

Do not use as a substitute for pytest tests/integration/api_wired/... (the regression gate) or /test-manual-api-wired (the curl-level harness).

Example namespace names. Where scenario recipes show in-cluster hosts such as dataspoke-01 (in kubectl -n …) and dataspoke-dummy-data-01 (in the recipe host_port) — these are illustrative. Substitute your actual namespaces from helm-charts/.env.dev (DATASPOKE_KUBE_DATASPOKE_NAMESPACE, DATASPOKE_DEV_KUBE_DUMMY_DATA_NAMESPACE) before running the commands verbatim.

Division of labour

No browser-automation tool is available, so the work is split:

• You drive the browser, perform each gesture, and report what you observed. You may paste a screenshot — this skill reads it (vision) and checks it against the expected UI state.
• This skill scripts each gesture from the source test, and after you act, independently probes the backend to confirm the mutation landed.
• A step passes only when both agree: your UI observation ✓ and the backend probe ✓. Either alone is insufficient.

Source of truth (no hardcoded scripts)

Every step is derived dynamically, never hardcoded here:

• The api-wired test file is the canonical sequence — step order, request payloads, expected statuses, and the side effects to verify. If the test changes, the walkthrough changes with it. Payloads shown to you are copied verbatim from the test, never paraphrased.
• spec/feature/FRONTEND_*.md route tables map each API route to its page path + UI gesture. These specs anchor every UI element to an API route, so the route→gesture mapping is reliable (see §UI-gesture mapping).

Reused helpers (shared with test-manual-api-wired)

This skill does not duplicate the env/token/probe machinery — it calls the sibling skill's helpers by path:

• .claude/skills/test-manual-api-wired/helpers/setup_env.sh — bootstrap env + admin JWT into /tmp/_manual_test_env.
• .claude/skills/test-manual-api-wired/helpers/refresh_token.sh — re-issue the admin JWT on 401.
• .claude/skills/test-manual-api-wired/helpers/probes.py — DB / GMS / k8s side-effect probes. Run python3 .../probes.py --list to enumerate.

Own helper:

• .claude/skills/test-manual-ui/helpers/preflight.sh — health-check, confirm the frontend is reachable, print the app URL + login credentials, optional reset-seed. Bootstraps env by calling setup_env.sh.

Workflow

1. Resolve scope (dynamic discovery — no hardcoded list)

Glob tests/integration/api_wired/test_*.py. For each file, Read the first ~80 lines and extract (filename, test_function_name, docstring_first_line).

If $ARGUMENTS (scope) is given: lowercase + tokenize, score each candidate by token-overlap against {filename, function name, docstring first line}. Single hit → confirm with one AskUserQuestion. Multiple hits → menu of matches. Zero hits → menu of all discovered scenarios.

If no scope arg → menu of all discovered scenarios. Cap at 4 options per AskUserQuestion; if more, group by UC prefix.

2. Pre-flight

bash .claude/skills/test-manual-ui/helpers/preflight.sh

This runs ./helm-charts/bin/health-check.sh, bootstraps /tmp/_manual_test_env via setup_env.sh, resolves and curl-probes the frontend URL (host pnpm dev at http://localhost:3000 when --frontend local, else the cluster http://app.<INGRESS_DOMAIN>/ — <IP>.nip.io in managed mode or the operator's host in shared mode), and prints the app URL plus the admin login ([e-mail] / dataspoke).

On any health-check FAIL: name the failed subsystem, offer to reinstall via Skill(k8s-deploy) action reinstall, do not proceed until green. If the frontend URL is unreachable: tell the user to start it (pnpm -C src/frontend dev for --frontend local, or install.sh --profile dev --components frontend for the cluster build) and stop.

Ask once: reset-seed baseline? Default Yes (per feedback_reset_before_api_wired). Yes → set -a && source helm-charts/.env.dev && set +a && uv run python -m tests.integration.util --reset-seed.

Reset-seed re-creates the admin user, invalidating the JWT preflight minted before it (the next admin call returns 401 "User no longer exists."). After a reset-seed completes, re-issue the token via refresh_token.sh before Step 1 so the walkthrough's first backend call does not 401.

For UC4 scenarios, additionally seed the LLM context after the reset-seed has finished — --uc4-seed masks the customers.eu_profiles and orders.events descriptions that --reset-seed ingests into DataHub, so the datasets must be present first. Run it in the same invocation (reset always runs first): … --reset-seed --uc4-seed. On completion run --uc4-restore to unmask. If the masking targets are absent, --uc4-seed now raises rather than writing an empty-mask no-op — re-run --reset-seed and let it finish before retrying.

For UC1 Case 1 (test_uc1_01_datahub_managed) and UC1 Case 3 (test_uc1_03_passive_kafka), guard against an out-of-band hourly sweep: these scenarios assert read-only mirroring of dataset_registry / source state, and the datahub-sync-hourly DAG mutating it mid-walkthrough would cause flaky observations. Read GET /admin/dags; if the datahub_sync group is unpaused (paused: false), ask via AskUserQuestion whether to pause it for the run (default Yes) — PATCH /admin/dags/datahub_sync {"paused": true}. Restore the prior state ({"paused": false}) on completion if you paused it.

For UC1 Case 3, after the imazon kafka topics are mapped to the PASSIVE source, add a passive-observation assertion phase: emit a fresh Operation aspect on the mapped topics (uv run python -m tests.integration.util --emit-passive-kafka-ops), re-trigger the sync sweep (POST /internal/activities/ingestion/sync), then confirm a fresh passive_observation event in the Events panel and via GET /spoke/ingestion/sources/{id}/event (one event per mapped dataset URN).

Then have the user log in at the printed URL before Step 1.

3. Granularity prompt (once)

Approval granularity? [per-step (default) | per-phase (setup/verify/cleanup)]

Per-step: one gesture at a time. Per-phase: bundle the setup gestures, the read-only verifications, and the cleanup into 3 prompts. Observe-only steps (read-back the UI already polls) never block on a gesture but still confirm.

4. Step extraction + UI-gesture mapping

Read the resolved test file. Walk the test function body in source order. Extract every await api_client.{put,post,patch,get,delete}(...), every httpx.{get,post}(...) against ${GMS}, and every internal-route call (/internal/activities/...). For each, record method + URL, body (verbatim from the test), and the expected status/assertions that immediately follow.

Then classify and map each call to a UI step:

• [UI gesture] — the call is something a user triggers in the browser (create / edit / delete a config, trigger a run, approve / reject). Map the route → page path + the concrete gesture via §UI-gesture mapping.
• [observe] — a read the UI performs on its own (list/detail/timeseries the page renders or polls). Map to "what the page should now show."
• [API-fired, no UI surface] — internal-activity triggers, GMS GraphQL seeding, direct DB setup the test does to stage state. These have no user gesture; fire them via the curl path exactly as /test-manual-api-wired would (same approval gate) so the scenario stays coherent. State plainly that this step has no UI surface and is being fired on the user's behalf.

Preserve test source order. Group consecutive [observe] reads under one heading.

5. Per-step loop

For each step:

1. Print preview (concise — under 25 lines):

   STEP N: <operation-framed action>                       [UI gesture]
     page:   <UI path>
     do:     <gesture — clicks / field values / submit>
     expect (UI):      <what you should see — toast, row, badge, redirect>
     expect (backend): <method URL → status; side effects to probe>
   

   For [API-fired, no UI surface] steps, replace page/do with a REQUEST block (full pretty-printed body, verbatim from the test) as in /test-manual-api-wired.
2. Gate (per-step mode): AskUserQuestion. Pick the verb by step type — never use "Done" for a step the user has not performed:
   • [UI gesture] → Done / Skip / Abort (the user performs the gesture, then confirms it is done).
   • [API-fired, no UI surface] → Fire it / Skip / Abort (the skill runs the call on the user's behalf; the user is authorizing, not reporting). Use authorization framing ("Fire it" / "Run it"), never "Done".
   • [observe] reads never block on a gesture (see step 3).
3. Act, then report: for [UI gesture] the user performs the gesture and describes the observed UI state (or pastes a screenshot); the skill checks it against expect (UI). For [API-fired] the skill fires the call and reports the response. [observe] steps confirm without a gesture gate.
4. Backend probe: confirm the side effect independently —
   • the REST read-back the test asserts (the same GET the page makes), via curl with the admin token from /tmp/_manual_test_env; and
   • deeper probes the UI cannot show (k8s secret, DataHub aspect, event row) via helpers/probes.py (see §Probe selection). Build the curl per §Backend probe — request conventions (three call families, three prefixes + headers). On HTTP 401, run refresh_token.sh and retry once; inside a poll loop, check the HTTP status — never treat a 401 body as "absent".
5. Result line: ✓ pass (UI ✓ + backend ✓) / ✗ fail / ⚠ warn — one-line reason. If UI and backend disagree, the step is ✗ and the disagreement is the headline (e.g. "backend emitted 2 datasets but the Datasets table is empty").

6. Cleanup phase

Walk the test's finally: block as its own phase (usually a delete). Same prompt rules. Honour "skip cleanup" — leave state for inspection; the next run reset-seeds.

7. Summary

Dual-confirmation table:

| step | action                       | UI observed        | backend probe     | result |
|------|------------------------------|--------------------|-------------------|--------|
| 1    | create active-custom source  | detail, secret ●●● | 201 + k8s secret  | ✓      |
| 2    | dry-run                      | run: success, 0 ds | emitted_count=0   | ✓      |
| 3    | real run                     | run: success       | emitted_count≥2   | ✓      |
| 4    | datasets table               | 2 rows, emitted    | GET datasets ≥2   | ✓      |
| 5    | events table                 | INGESTION.COMPLETE | event status=success | ✓   |
| 6    | reverse-lookup page          | owning source, run | source_id matches | ✓      |
| 7    | delete source                | gone from list     | GET source → 404  | ✓      |

Then a one-paragraph narrative of what the run proved about the UI↔backend wiring.

UI-gesture mapping

Map each extracted route to a page + gesture using the FRONTEND_*.md route table for that feature (read the relevant spec at runtime — do not hardcode). The stable anchors:

Feature	API route prefix	Page(s)	Gesture source
Ingestion	/spoke/ingestion/..., /spoke/common/data/{urn}/attr/ingestion	/ingestion, /ingestion/sources/new, /ingestion/sources/[id], /ingestion/data/[urn]	FRONTEND_INGESTION.md
Validation	/spoke/validation, /spoke/common/data/{urn}/attr/validation/...	/validation, /validation/data/[urn]	FRONTEND_VALIDATION.md
OntoGen	/spoke/ontogen/...	/ontogen, /ontogen/conf, /ontogen/seed	FRONTEND_ONTOGEN.md
MetaGen	/spoke/metagen/..., /spoke/common/data/{urn}/attr/metagen/...	/metagen, /metagen/data/[urn]	FRONTEND_METAGEN.md
Governance	/spoke/governance/metric/...	/governance/dashboard, /governance/metrics, /governance/metrics/new, /governance/metrics/[id]	FRONTEND_GOVERNANCE.md

Method → gesture heuristics (refine from the spec's component notes): POST .../sources → create form + Submit; POST .../method/run → run panel (toggle dry_run); POST .../method/review → Approve/Reject on the candidate card (ConfirmDialog); PUT/PATCH .../attr/conf → conf editor Save; DELETE ... → delete behind ConfirmDialog; bare GET list/detail → the page renders or polls it ([observe], no gesture).

Caveat — passive features: for validation (UC2) the canonical conf-registration path is the API (DEs/pipelines PUT .../attr/validation/conf); the UI editor is a thin convenience. Classify the "register slot" step as [API-fired, no UI surface] and drive registration via curl — do not map it to a UI create form. Reserve UI conf gestures for steps that deliberately exercise the convenience editor (e.g. retire/resurrect).

Backend probe — request conventions

The backend probes are hand-built curls against the live cluster, not the pytest api_client (which hides the base-URL split, the auth header, and URL-encoding — so reading the test alone will mislead you). /tmp/_manual_test_env (written by setup_env.sh) exports BASE, ADMIN_TOKEN, GMS, GMS_TOKEN, INTERNAL_TOKEN. Three call families take three different prefixes + auth headers — getting one wrong reads as a spurious 404/401, i.e. "side effect missing" when it isn't:

Call family	URL	Auth header
Public spoke	$BASE/api/v1/spoke/…	Authorization: Bearer $ADMIN_TOKEN
Internal activity	$BASE/internal/… — no /api/v1	X-Internal-Token: $INTERNAL_TOKEN
DataHub GMS GraphQL	$GMS/api/graphql	Authorization: Bearer $GMS_TOKEN

Why the split trips you up: the test drives both with the same api_client, whose base-URL is the host root. So internal calls appear as api_client.post("/internal/…") (router mounted with no API prefix) while public calls carry an explicit /api/v1/spoke/… in the test string. Mirror that in curl — do not prefix internal routes with /api/v1, and use the X-Internal-Token header (not Bearer) for them. GMS GraphQL always needs Bearer $GMS_TOKEN; an unauthenticated GMS call returns a bare 401 Unauthorized JSON. URL-encode dataset URNs in path params (urllib.parse.quote(urn, safe='')) — they contain ( ) , :.

Long-poll trap (JWT expiry masquerading as a timeout): the admin JWT TTL is short (~15 min). A poll loop (ES-settle reads budget ≥60–180s) can out-run it; the API then returns 401 "Token has expired.", and a loop that only inspects a body field (r.get('source_id')) sees the 401 error-body parse to a falsy value identical to "not indexed yet" — a real pass disguised as a 180s timeout. Defences: (1) call refresh_token.sh proactively right before any ES-poll step; (2) in poll loops capture the HTTP status (curl -w '%{http_code}') and treat 401 as refresh-and-retry, never as "absent".

Probe selection

Reuse the /test-manual-api-wired SKILL's "Probe selection" table verbatim — same route→probe rules, same helpers/probes.py. The UI skill adds one layer: the REST read-back the page itself performs (the GET whose JSON the page renders) is run first as the primary backend confirmation, then probes.py for what the UI can't surface (k8s secret contents, DataHub aspects, raw event rows). If no probe rule matches, report "no automatic probe — confirm from the UI only" and continue.

Worked example — UC1 Case 2 (test_uc1_02_active_custom_postgres.py)

Payloads and expectations below are copied verbatim from the test; the skill generates the same for any scenario.

STEP 1 — create ACTIVE_CUSTOM_MANAGED source [UI gesture]

page:   /ingestion/sources/new
do:     mode selector → ACTIVE_CUSTOM_MANAGED; in the YAML recipe editor enter
        the lossless NESTED YAML form of this body, then Submit. Render the recipe as
        genuinely nested mappings — never dotted-key shorthand (`schema_pattern.allow:`),
        which YAML parses as a literal flat key the extractor ignores (→ allow-all):
          mode: ACTIVE_CUSTOM_MANAGED
          name: dummy postgres example_db in catalog schema
          schedule: '0 0 * * *'
          recipe:
            source:
              type: postgres
              config:
                host_port: example-postgres.dataspoke-dummy-data-01.svc.cluster.local:5432
                database: example_db
                username: postgres
                password: ${dummy-data-pg__password}
                env: DEV
                schema_pattern:
                  allow: ['^catalog$']
expect (UI):      success → redirect to source detail; Recipe panel renders the
                  YAML with password masked as ${dummy-data-pg__password}
                  (never the plaintext); mode badge ACTIVE_CUSTOM_MANAGED.
expect (backend): POST /spoke/ingestion/sources → 201; body.mode=ACTIVE_CUSTOM_MANAGED,
                  body.schedule='0 0 * * *', NO schedule_tier on the wire,
                  recipe…password == '${dummy-data-pg__password}' verbatim,
                  plaintext password absent from the response.
probe:            GET /spoke/ingestion/sources/{id} (read-back, assert masked ref);
                  probes.py k8s_secret dataspoke-source-cred-dummy-data-pg

Precondition (self-provisioning): ensure the K8s Secret exists before Step 1 (create-if-absent; see preflight.sh):

kubectl create secret generic dataspoke-source-cred-dummy-data-pg \
  --from-literal=password=<DATASPOKE_DEV_DUMMY_DATA_POSTGRES_PASSWORD> \
  -n dataspoke-01 --dry-run=client -o yaml | kubectl apply -f -

spec: feature/SECRET_RESOLUTION.md §Reference-only model — out-of-band provisioning.

STEP 2 — dry-run [UI gesture]

page:   /ingestion/sources/[id]  → Run panel
do:     toggle dry_run ON, trigger Run
expect (UI):      run result shows success; "no datasets emitted".
expect (backend): POST .../method/run?dry_run=true → 200; detail.dry_run=true,
                  detail.emitted_urns_count == 0; run_id non-empty.

STEP 3 — real run [UI gesture]

page:   /ingestion/sources/[id]  → Run panel
do:     toggle dry_run OFF, trigger Run
expect (UI):      run result success; emitted-dataset count ≥ 2.
expect (backend): POST .../method/run → 200; detail.dry_run=false,
                  detail.emitted_urns_count >= 2.
probe:            probes.py gms_aspect <catalog.title_master urn> datasetProperties;
                  probes.py gms_aspect <…> schemaMetadata; probes.py gms_lastingested <…>

where <catalog.title_master urn> = urn:li:dataset:(urn:li:dataPlatform:postgres,example_db.catalog.title_master,DEV).

STEP 4 — datasets table [observe]

page:   /ingestion/sources/[id]  → Datasets panel
expect (UI):      ≥ 2 catalog rows (title_master, editions), origin = emitted.
expect (backend): GET .../sources/{id}/datasets → ≥ 2 urns, origins include 'emitted'.

STEP 5 — events table [observe]

page:   /ingestion/sources/[id]  → Events panel
expect (UI):      an INGESTION.COMPLETE row for this run, newest first.
expect (backend): GET .../sources/{id}/event → event with detail.run_id==run_id,
                  event_type=INGESTION.COMPLETE, status='success' (poll ≤ 30s).

STEP 6 — per-dataset reverse-lookup [UI gesture]

page:   /ingestion/data/<encoded catalog.title_master urn>
expect (UI):      Ingestion panel names the owning source (links to its detail),
                  mode ACTIVE_CUSTOM_MANAGED, latest run = success.
expect (backend): GET /spoke/common/data/{urn}/attr/ingestion → source_id==this source,
                  mode=ACTIVE_CUSTOM_MANAGED, latest_run.status='success' (poll ≤ 30s
                  for ES settle, per project_es_indexing_lag_after_reset_seed).

STEP 7 — cleanup: delete source [UI gesture] (the test's finally:)

page:   /ingestion/sources/[id]
do:     Delete (ConfirmDialog)
expect (UI):      source gone from /ingestion list.
expect (backend): GET .../sources/{id} → 404.

Operating principles

• The test file is the source of truth. Discover steps, payloads, and expectations from the test source — never hardcode them here. Payloads shown to the user are verbatim from the test, never paraphrased.
• Two independent confirmations per step. A green UI with no backend write is a fail; a backend write the UI never shows is a fail. Report the disagreement.
• Operation-framed previews (feedback_naming_operation_framed): "create active-custom source" beats "POST /spoke/ingestion/sources".
• Don't patch src/ from here. This is a manual-test session: on a real UI or backend bug, abort and surface the gap for the Plan → generator → reviewer loop (feedback_no_onthefly_fix_during_manual_test). Editing the test file or this skill to correct a step is fine.
• Reset before run unless the user opts out (feedback_reset_before_api_wired).
• JWT TTL is short. Refresh proactively via refresh_token.sh — before each ES-poll step, not just on a visible 401. A token that expires mid-poll surfaces as a misleading empty read, not an error (see §Backend probe — request conventions).
• Curl the right family. Internal routes are $BASE/internal/… (no /api/v1) with X-Internal-Token; GMS GraphQL needs Bearer $GMS_TOKEN. The pytest api_client hides this — don't copy its paths/headers verbatim into curl.
• Never truncate response bodies in the backend-probe output.