| name | report-viewer-spec |
| description | Add, run, or fix bugs exposed by end-to-end tests for the report viewer itself (it has grown into a complicated app in its own right) |
Report Viewer E2E Specs
The e2e report viewer (app/e2e-tests/report-viewer/) has grown into a
complicated application in its own right, so it needs its own end-to-end
test suite — separate from the main Meadow e2e tests in
app/e2e-tests/test-runner/tests/.
When invoked, the user wants to either add a new spec, run the
existing suite, or fix a bug a spec exposed. Use this skill to
establish shared context, then follow the user's specific ask.
Where everything lives
app/e2e-tests/report-viewer/
├── e2e/
│ ├── playwright.config.ts # two-entry webServer (express + vite)
│ ├── fixtures/
│ │ └── publish-flow-fixture.ts # ensurePublishFlowArtifact()
│ ├── pages/
│ │ ├── TickNavComponent.ts # navigate to a specific tick
│ │ └── MeadowHomeFilesComponent.ts # Files-tab assertions
│ └── tests/
│ └── *.spec.ts # one spec per file
├── tsconfig.e2e.json # separate tsconfig for e2e/
└── package.json # has "e2e" and "e2e:install" scripts
Page object pattern
Specs should be thin. DOM selectors and interaction sequences belong in
page-object components under e2e/pages/, mirroring the convention used
by the main e2e suite (see
app/e2e-tests/test-runner/src/run/pages/SiteEditorPage/components/FilterPanelComponent.ts
for the reference).
- One class per logical area of the UI (a tab, a panel, a modal).
- Constructor takes
(page: Page, expect: Expect).
- Private getters for reusable locators, public
async methods for
actions and assertions.
- Prefer stable
data-* attribute selectors over structural CSS. The
ScenarioViewer already exposes data-idx, data-tick-index,
data-file-path, and data-file-status; add more when a new spec
needs a stable hook rather than relying on class names or text content.
Existing components:
TickNavComponent.goToTick(n) — 1-indexed tick navigation via the
header dropdown. Throws if the tick isn't in the dropdown.MeadowHomeFilesComponent.activate() / fileEntry(path) /
expectFileNotModified(path) / expectFileCommitted(path) /
getFileStatus(path). Prefer expectFileCommitted when the intent
is "this file should have been committed by the app" — it covers
both uncommitted-new and uncommitted-modified. Use
expectFileNotModified when the tighter "shouldn't have local edits"
assertion is what you want.
How to run the suite
cd app/e2e-tests/report-viewer
npm install
npm run e2e:install
npm run e2e
npm run e2e -- tests/sanity.spec.ts
npm run e2e -- --headed
First-run cost. The very first run builds the cached publish-flow
fixture, which shells out to the main test-runner CLI and actually runs
publish-flow.spec.ts (Docker containers, MinIO, any extension backing
services, the works). Expect it to take a couple of minutes. Subsequent
runs reuse the cache and are fast.
Port isolation. The e2e suite runs on dedicated ports 3556 (server)
and 5275 (client) — NOT the /dev defaults of 3456/5175. This is
deliberate so the suite works regardless of what dev servers other
worktrees are running. The server and vite configs accept
REPORT_VIEWER_PORT and REPORT_VIEWER_CLIENT_PORT env vars (defaults
preserved for /dev); e2e/playwright.config.ts sets them for its
webServer subprocesses and uses reuseExistingServer: false.
The cached publish-flow fixture
All specs share a single fixture: the artifact directory produced by running
app/e2e-tests/test-runner/tests/publish-flow.spec.ts.
Per-spec pattern
import { test, expect } from "@playwright/test";
import {
ensurePublishFlowArtifact,
type PublishFlowFixture,
} from "../fixtures/publish-flow-fixture.js";
let fixture: PublishFlowFixture;
test.beforeAll(() => {
fixture = ensurePublishFlowArtifact();
});
test("...", async ({ page, request }) => {
const { runId, testSlug } = fixture;
});
One test per file. Reuse the cached fixture. Prefer a cheap API-level
sanity check before the UI assertions — it produces a clearer failure
message if the fixture is malformed.
UI assertion gotchas
Tick X/Y: counter is hidden until a tick is selected. On initial
scenario load, ScenarioViewer renders Tick: -- (dash) until the
user scrubs the video or picks a tick from the dropdown. Don't assert
on the X/Y text unless your test first selects a tick.- Check
hasTicks via the dropdown button. The tick dropdown button
is present whenever hasTicks=true, so page.getByRole("button", { name: /^Tick/ }) is a reliable way to prove the client saw ticks
in the manifest.
- Breadcrumb as a load indicator.
page.getByText(testSlug) being
visible is a simple proof that the SPA router mounted
ScenarioViewer for the target artifact.
Useful report-viewer server endpoints
(Served by app/e2e-tests/report-viewer/src/server/index.ts on port 3456,
proxied through Vite at 5175.)
GET /api/:runId/:testSlug/manifest — full manifest, includes
ticks: ProcessedTick[]GET /api/:runId/:testSlug/snapshots — MeadowHome commit snapshotsGET /api/:runId/:testSlug/minio-snapshots — S3/MinIO snapshotsGET /api/:runId/:testSlug/state-repos — extension-contributed state
repos with their rendering meta (display name, key map, etc.)GET /api/:runId/:testSlug/state-snapshots/:repoName — snapshots for
a named state repoGET /api/:runId/:testSlug/state-snapshot/:repoName/:hash — table
contents at a state-repo commit
The client route for a scenario is /:runId/:testSlug, rendered by
ScenarioViewer.tsx.
Notable facts about the artifact
For the current publish-flow.spec.ts, the test slug is
publish-flow-uploads-files-to-minio (derived as
title.replace(/[^a-zA-Z0-9]+/g, "-").toLowerCase()). If the test title
ever changes, update PUBLISH_FLOW_TEST_SLUG in
publish-flow-fixture.ts.
After adding a spec
- Run
npm run e2e in app/e2e-tests/report-viewer/ and confirm it passes.
- Run
./quickcheck from the repo root (per project AGENTS.md). If
changes are limited to app/e2e-tests/, the scoped check is faster:
cd app/e2e-tests && _module/scripts/quickcheck
Quickcheck type-checks the e2e/ directory via tsconfig.e2e.json but
does NOT run the Playwright tests — those are opt-in via npm run e2e.
Fixing bugs exposed by a spec (the fix → regenerate → verify loop)
Many specs in this suite are intentionally written to fail first —
they are tripwires pointing at real bugs in the Meadow app (e.g.
"app/app_config.yaml should not be marked modified at tick 2"). When
the user asks to fix the underlying bug, use this tight loop:
- Diagnose at the right layer. The report viewer just renders what
manifest.json and ticks.jsonl tell it. If a file is reported as
modified/new/etc., check:
- First, run real
git status against the saved state repo at
~/meadow-e2e-artifacts/current/<runId>/<testSlug>/meadowHome-state-repo
(use --git-dir=.git --work-tree=.). If git itself agrees with the
tick data, the bug is NOT in fast_git_ops or the report viewer —
it's upstream in the Meadow app (some code path writes to the file
without committing).
- If git disagrees, the bug is in
fast_git_ops (see
app/native_utils/fast_git_ops/fast_git_ops_code/src/main.rs) or in
how test-fixtures.ts consumes its output.
- Fix at the source. For "file written but not committed" bugs,
the typical fix site is
app/backend/src/index.ts startServer()
where init helpers like ensureAppConfigInitialized run. Have the
init helper report whether it patched the file, and commit when it
did — not only on first creation. See the existing
ensureAppConfigInitialized / ensureResourcesConfigInitialized
pattern for reference.
- Invalidate the cached fixture. The fixture builder only
rebuilds when the spec file's content hash changes, so after an app
fix you must manually clear the cache:
rm -rf ~/meadow-e2e-artifacts/current/rv-fixture-*
- Regenerate + verify. Run
npm run e2e in
app/e2e-tests/report-viewer/. The fixture builder will rerun
publish-flow.spec.ts (slow, ~20s) to produce a fresh artifact, then
Playwright runs every spec against it. Confirm the target spec flips
from red to green — and that no other spec regresses.
- Run
./quickcheck from the repo root to catch any type or lint
fallout from the app-side fix.
- Repeat if the spec is still red — the fix wasn't sufficient, or
the bug lives somewhere else than you thought. The cycle is cheap
because step 3 is one command and step 4 is automated.
Cache key caveat. The fixture cache key is a hash of
publish-flow.spec.ts alone — it does NOT invalidate when the Meadow
app code changes. That's why step 3 is mandatory for app-side fixes.
Adjacent-bug heuristic. When you fix one "written but not
committed" bug, look at the same startup sequence for sibling bugs in
the same family, then have the user decide whether to add sibling
specs before or after the fix. Writing the failing spec first is
valuable because it proves the tripwire works.
Flakiness policy
Per project AGENTS.md: flaky tests are never acceptable. If a spec fails
sometimes and passes sometimes, fix the root cause — don't retry it away.
An intentionally failing regression spec (tripwire for an unfixed
bug) is NOT flakiness — it's a controlled, deterministic failure that
should stay red until the underlying bug is fixed. Don't silence or
skip such specs; the failure is the signal.
