migrate

# stardust:migrate Apply the target spec authored by `direct`, the visual canon written by `prototype --prep`, and the brand-module catalog extracted during `prepare-migration` to every page in the inventory. Produces a self-contained, deployable static HTML site under `stardust/migrated/`. Per-page, incremental, idempotent. `migrate` is the final stardust phase. Output is platform- agnostic HTML — downstream conversion (AEM EDS, a CMS, a framework) is the job of a separate plugin that consumes `migrated/` plus `DESIGN.json` plus the per-page `_meta.json` sidecars. ## Inputs - `<slug>` — optional positional. Migrate just this page. Without it, migrate every page whose status is `directed`, `prototyped`, or `approved` (and not `stale`). - `--all` — migrate every page including stale ones. - `--force` — re-migrate every page even when the idempotent skip would skip them. - `--require-approved` — refuse to migrate any non-`approved` page. Default behaviour migrates `directed` pages too (using Path A′ or Path B per `reference/template-and-module-rendering.md`); this flag flips approval-gating on. - `--strict-canon` — refuse approvals that conflict with canon. Default logs the deviation and continues. Useful for projects where canon discipline matters more than per-template flexibility. - `--clean` — delete assets previously bundled but no longer referenced from `stardust/migrated/assets/`. Off by default (migrate is additive). **Implies `--force`**: every page is re-rendered so the run's `bundledAssets` Set is the complete union of currently-referenced assets — otherwise `--clean` would risk deleting assets still referenced by idempotent-skipped pages. See `reference/asset-bundling.md` § Stale asset cleanup. - `--pin-timestamp <ISO8601>` — pin the migrate-provenance timestamp so re-runs without source changes produce byte- identical HTML. Default re-uses the current wall clock, which is fine for normal use; CI deployment fingerprinting may want the pin. The mobile-adapt audit, content-sourcing scan, and placeholder refusal are all mandatory gates — there is no `--skip-*` or `--allow-*` flag to bypass them. If a gate refuses a page, the remediation is to fix the proposed file (re-prototype, edit inline, or run an impeccable command) and re-invoke migrate. ## Setup 0. **Playwright re-probe (mandatory first step).** `--no-save` playwright installs from earlier phases are pruned by any later real `npm i` (extract SKILL.md § Setup → `--no-save` installs are ephemeral). Before any rendering step, probe `node -e "import('playwright').then(()=>process.exit(0))"` from the project root and re-install (`npm i -D playwright --no-save --legacy-peer-deps`) on failure. 1. Run the master skill's setup (`skills/stardust/SKILL.md` § Setup). **Flow guard.** If `stardust/state.json` exists without `flow` and the ask is a migration (a URL plus "migrate" / "to EDS" / "re-platform"), do not run: print the two-flow table from the master skill § Two migration flows and hand back to its routing — the flow is chosen and stamped there before any sub-skill runs (`skills/stardust/reference/state-machine.md` § Flow keys): `replica` unless the ask names a redesign. (Recorded: `migrate <url>` as the first command of two same-design migrations led to a hand-built compiler tuned by eye instead of the replica gate.) 2. Verify `stardust/state.json` exists with at least one `directed` page. 3. Verify project-root `DESIGN.md` and `DESIGN.json` exist with `DESIGN.json.extensions.canon` populated. 4. Verify `stardust/canon/` exists with at least `header.html`, `footer.html`, `canon.css`. **Canon auto-bootstrap (when steps 3–4 find no canon).** The documented `prototype → migrate → deploy` happy path does not run `prepare-migration`, so a first migrate legitimately arrives with no canon (observed on 4 of 6 e2e sites, where every run had to derive canon by hand to proceed — this is the fix). When canon is absent **and** at least one `approved` prototype exists, do not stop: run the canon write-back inline from the first approved prototype (the canon-author, default `home`) per `../prototype/reference/canon-extraction.md` § Five-step procedure — extract `header.html` / `footer.html` / `canon.css` to `stardust/canon/`, pin tokens + compositional moves to `DESIGN.json.extensions.canon`, and record `canon.source: "auto-bootstrap: <slug>"`. This is exactly what `prototype --prep` does on first approval; migrate performs it on demand so the core pipeline never dead-ends. Only stop and recommend `$stardust prepare-migration` when canon is absent **and** no approved prototype exists (there is nothing to derive canon from). Under `state.json.handsOff` the bootstrap is automatic and logged; interactively, surface it as a one-line notice before proceeding. 5. Verify `stardust/direction.md` has an active (not pending) direction. 6. Read `state.json.pages[]` and partition into: - **inScope**: status `directed`, `prototyped`, or `approved`, `stale: false` (or `--all` / explicit `<slug>`). - **skipped**: everything else, with reason captured. 7. **Validate provenance on every in-scope page.** Call `validateProvenance(page)` per `skills/stardust/reference/state-machine.md` § Provenance validation for every page in `inScope`. Abort with the helper's error when any page lacks live-render evidence — migrating a synthesized page record produces deployable HTML that misrepresents the source site, the exact failure mode that motivated the validator. Surface `Provenance OK on N pages` in the migrate-plan output before Phase 1. 8. **Mobile-adapt audit on every Path A / Path A′ source.** For every page whose render branch consumes a proposed or archetype HTML file (Path A, Path A′ per `reference/template-and-module-rendering.md` § Render path selection), run the audit per `skills/prototype/SKILL.md` § Mobile-adapt audit: - `<meta name="viewport" content="width=device-width, ...">` present, width not pinned to a fixed pixel value. - At least one `@media (max-width: ...)` rule. - At least one mobile-targeted breakpoint at ≤ 640px. Refuse pages that fail — the audit is mandatory; there is no skip flag. The user fixes the proposed file (re-prototype or chat-driven impeccable command) and re-invokes migrate. Record the audit result per page in the migrate report and in the post-render `_meta.json#audit.adapt` sidecar. Path B (unique-renders) skips the audit because adapt hasn't run on those pages — a Path B page that needs mobile coverage gets it via `$impeccable adapt` invoked separately by the user. Surface this distinction in the report so it's not read as a silent skip. ## Procedure ### Phase 1 — Plan **Dynamic-surface precondition (safety net).** If `stardust/dynamic-features.md` is missing, the hand-run flow (`extract → direct → prototype → migrate`) never passed a pre-import gate: run the stardust `dynamics` skill Phases 1–3 now (`extract --dynamics` for reach if needed, detector on the archetypes, triage draft, curate) before rendering any page. Never import a site as static without a decision per dynamic row. Per page, rows of the inventory that touch it become `contentDeviations[]` `kind: "dynamic-dependency"` entries (`reference/content-preservation.md § Dynamic dependencies`). **Gated-archetype precondition (`flow: replica`).** Before rendering any `sibling`-tier page, read `stardust/replica/progress.json`: the page type's archetype must have a gate result at every configured breakpoint that is `pass: true`, or over the bar with every residual carrying a `cause` (`../replica/reference/source-fidelity-gate.md` § Residual logging format — a documented residual is a pass with an asterisk). An archetype never gated, or over the bar with no residual entries, blocks its page type: report the archetype slug and `$stardust replica <archetype>`, and render nothing for that type. The same rule guards `rollout` Setup; the published-origin re-gate is unchanged. Thresholds are the gate's. Print the plan and wait for confirmation when the scope is large: ``` migrate plan ============ In scope: 127 pages Path A (approved) 6 pages: home, news/post-housing-summit, news, ... Path A' (template-applied) 118 pages: 84 article, 5 listing, 11 program, 2 form, 16 static Path B (unique) 3 pages: 404, search, faq Skipped: 0 stale, 0 unscoped DESIGN.md sha: 1a2b3c4 DESIGN.json sha: 5d6e7f8 Canon shas: header:7g8h9i footer:9i0j1k css:1k2l3m Output: stardust/migrated/ + per-page _meta.json sidecars Idempotent skip: enabled (run with --force to override) Reply "go" to proceed. ``` For 1-3 pages or `<slug>` invocation, skip the confirmation. ### Phase 2 — Per-page render In the replica flow a sibling renders only after its archetype is deployed and gated at the published origin (`../replica/reference/handoff-contract.md` § 3 row C, unit `C-archetype`; #126); the render is the encoder's input, its gate is the row after PUT. For each page in scope, follow `reference/migration-procedure.md` and `reference/template-and-module-rendering.md`: - **Idempotent skip check** first (sha-compare across `designMd`, `designJson`, `sourceCurrent`, `sourceProposed`, `canonShas`, `archetypeSource`). - **Placeholder gate** (Path A and Path A′ only — pages with a proposed file or archetype). Refuse when `[data-placeholder]` elements or non-empty `_provenance.unsourcedContent[]` are present — the user fills the missing content in the proposed file before re-invoking migrate. No bypass flag: shipping placeholders to a public site is the failure mode this gate exists to prevent. Widget mounts are links (`a[data-widget]`), never `data-placeholder`, so the gate does not fire on them. - **Render branch selection** (LLM judgment per T&M § Render path selection): A / A′ / B. **Declare the page's `fidelityTier`** from the branch — A → `archetype` (craft-gated), A′ → `sibling` (canon-fork, the cheap default for breadth — variance-probed once per template before cloning, `reference/fidelity-tiers.md` § Sibling variance probe; deltas become variant classes, never per-page forks), B/bodyless → `thin` — per `reference/fidelity-tiers.md`. Record `fidelityTier`, `archetypeSource`, `template` (the archetype's slug; the archetype's own sidecar leaves it null), `modules[]` (the page's block ids — composite sections only), `widgets[]` (derived from the page's `a[data-widget]` mounts) and `gatesPassed[]` in `_meta.json` so coverage shows what was craft-gated vs cloned and rollout groups and dedups from the sidecars, without a census. - **Render** per the chosen branch's procedure in T&M. - **Canon application** — chrome injection, canon.css injection, deviation logging. - **Module rendering** — render module instances via `stardust/canon/modules/<id>.html`; bespoke slots logged with `data-bespoke`. - **Apply content-preservation rules** per `reference/content-preservation.md`. Internal-link rewriting always emits migrated-tree paths; missing slugs flagged broken. - **Content-count acceptance** per `reference/fidelity-tiers.md` § Content-count acceptance: compare role-classified node counts (headings, body/list nodes, CTAs, images) between the captured source page JSON and the rendered result. A count drop in any class not covered by a logged `contentDeviations[]` entry fails the page — dropped-content importer bugs must surface here, while the importer is still cheap to fix, not at a downstream fidelity gate. Record the pass in `_meta.json#gatesPassed[]` as `"content-count"`. - **Compose `<head>` metadata** per `reference/metadata-and-jsonld.md` (five categories; page-type-driven JSON-LD). - **Validate** per T&M § Validation contracts. Strict contracts refuse the page; soft contracts log and continue. - **Compute output path** per migration-procedure.md § Output path mapping. - **Asset bundling.** Scan the final HTML for asset references (six detection shapes per `reference/asset-bundling.md` § Detection), copy each unique referenced subpath from `stardust/current/assets/<subpath>` to `stardust/migrated/assets/<subpath>` (preserving subdir structure), then rewrite every reference to the root-relative form `/assets/<subpath>`. Cross-page dedup uses a module-level Set seeded from `state.json.migrate.bundledAssets[]`. Missing source assets warn-and-skip per § Edge cases; the bundle stays internally consistent. - **Media reconciliation.** For every image **not** bundled to same-origin (reused source-CDN URLs under Mode A image-reuse), decide optimize/keep/rewrite/omit per `reference/media-reconciliation.md`. Cross-origin `<img>` kept as source URLs must **skip `createOptimizedPicture`** (it drops the `?v=` key and corrupts the rendition); broken URLs are repaired (missing `?`-delimiter, wrong host) or omitted, never shipped as `about:error`. `rollout` re-runs the authoritative network resolve at delivery (`media-reconcile.mjs`). - **Cinematic sibling (when `<slug>-cinematic.html` exists).** Migrate consumes the STATIC prototype only — the cinematic layer is never merged. Copy the motion assets (`lenis.min.js`, `lenis.min.css`) from `stardust/prototypes/` to `stardust/migrated/assets/motion/` (idempotent) for downstream consumers (deploy/rollout decide whether to wire them), and record `cinematic-variant-not-consumed` in the page's `_meta.json#migrationDecisions[]`. - **Write** the migrated `index.html` and the `_meta.json` sidecar in the same directory. Provenance block as first child of `<head>`. Record `assetsBundled` (count of unique asset refs on this page) in `_meta.json`. The mechanics of this phase are scripted: `node skills/migrate/scripts/migrate.mjs render <slug…|--all>` (the project copy runs from `stardust/scripts/migrate/`) builds the page map, places each page at its URL-literal path, rewrites internal links, bundles assets, composes `<head>` (provenance, `:root`, canonical and JSON-LD defaults), validates strictly and writes the sidecar, skipping pages whose input shas are unchanged. The judgments it cannot make are recorded on the sidecar with `gate`, `deviation`, `decision`, `variant` and `modules` on the same script; it never advances `pages[].status`. **Recorded units.** A whole-site render is bookkept as units in `stardust/migrate/progress.json`, the shape of rollout's C-deliver unit ledger (`../replica/reference/handoff-contract.md` § 3 row C): `units.<name>: {status: pending|running|done|failed, kind: plan|render|assets|report, templates[], pages: n, startedAt, endedAt, verdict}`. One `plan` unit (Phase 1), one `render` unit per template cluster of at most ~8 siblings (`migrate.mjs render <slug…>` takes the cluster's slug list), one `assets` unit when Phase 3 rehosts media, one `report` unit (Phase 4). Per unit, in this order: the `progress.json` write → the checkpoint commit → the next unit; a unit end is a safe resume point, not a stop. A session resumed inside migrate reads the file and runs only the units not `done`; a unit left `running` re-renders its slugs — the driver is idempotent, unchanged pages are skipped. A recorded run rendered 31 siblings through five parallel builders as one unit in one session — its most expensive — with no resume point between the plan and the final report. ### Phase 3 — Sitewide assets and bundle finalisation Per-page asset bundling already happened in Phase 2 (every referenced media subpath is on disk under `stardust/migrated/assets/`). Phase 3 fills in the **sitewide assets** that no individual page references explicitly: 1. Copy `stardust/current/assets/logo.<ext>` to `stardust/migrated/assets/logo.<ext>` (only if missing or stale). Record under `state.json.migrate.bundledAssets[]`. 2. Verify favicon variants and font files were generated by `prepare-migration` Phase 4. If absent, log a warning and continue (the migrated site renders without them, just missing some platform-specific affordances). 3. Add `stardust/migrated/robots.txt` and `sitemap.xml` derived from the migrated page inventory per `reference/metadata-and-jsonld.md` § Sitemap entry. 4. If `--clean` was passed, compute `stale = priorBundle.filter(p => !bundledAssets.has(p))` from `state.json.migrate.bundledAssets[]` and remove each stale subpath from `stardust/migrated/assets/`. Record the deletions under `state.json.migrate.cleanedAssets[]`. Per `reference/asset-bundling.md` § Stale asset cleanup. 5. Verify **portability**. The bundle must work via `file://`, at a webserver root, and at any subpath — "one shape, works everywhere". Run every audit; any non-empty grep output or non-zero fixture exit fails the run with the cited error message: ```bash # No source-tree escapes