| name | design-google-ingest |
| version | 1.0.0 |
| description | Crawl design.google end-to-end via Edge headless (SPA-aware) — discover every /library article + site page, scrape post-hydration DOM via scripts/edge-mass-scrape.ts, then run the design-google-curator agent to assemble docs/DESIGN.md as the canonical aphrody design reference. |
| when_to_use | User types "/design-google-ingest", says "scrape design.google", "refresh DESIGN.md", "rebuild the design reference", "ingest design.google", or asks to keep the DESIGN.md in lockstep with what Google publishes at design.google. Pair with /loop weekly ("/loop 7d /design-google-ingest") to keep the cache fresh. |
design-google-ingest — One-shot SPA crawl + curate
Mode /goal permanent : décider seul, ne pas demander confirmation, finir le refresh + commit.
Crawls design.google end-to-end, materialises the post-hydration DOM of
every discoverable URL into the local edge-cache, then dispatches the
design-google-curator agent to assemble docs/DESIGN.md from the raw
HTML. The result is a single Markdown file that aphrody contributors can
read in 5 minutes to know what Google's design org has shipped + the
canonical references each article exposes.
Operating contract
- One invocation = one full refresh. Re-running is idempotent: cached
URLs are skipped unless
--force is passed.
- SPA-aware crawl.
design.google is a Single Page Application; most subroutes
ship only the empty shell when fetched without JS. The skill uses
scripts/edge-mass-scrape.ts (Microsoft Edge --headless=new --dump-dom --virtual-time-budget=15000) so every page receives the
full client-side hydration before snapshot.
- No mocks. The curator only ever cites text that actually appears
in the captured DOM. If a page failed to scrape, the curator says so
in the audit footer rather than inventing content.
- Two-output split.
var/data/edge-cache/<sha256-of-url>.html — raw HTML per URL
(gitignored, regenerable).docs/DESIGN.md — distilled Markdown ledger (checked into git).
The pipeline, one tick
1. Discover URLs (scripts/design-google-ingest.ts):
- Re-scrape `https://design.google/library` for the freshest
/library/* href list.
- Union with the canonical seed list
(home, about, events, products, library).
- Drop duplicates, normalise to lowercase, write
`scripts/design-google.urls.json` (schema v1).
2. Run mass-scrape:
bun run scripts/edge-mass-scrape.ts
--urls=scripts/design-google.urls.json
--cache=var/data/edge-cache
--concurrency=4
--virtual-time=15000
--retry=2 --timeout=60000
-> manifest.json (engine=edge) lands next to the HTML files.
3. Dispatch the `design-google-curator` agent:
prompt = "Read every URL in scripts/design-google.urls.json,
match it to its cached HTML, extract title + meta
description + first prose paragraph + every #RRGGBB
color stop appearing in <code> or inline style, then
assemble docs/DESIGN.md per the SKILL.md output schema."
4. Verify:
- `docs/DESIGN.md` exists, non-empty.
- Every URL in scripts/design-google.urls.json maps to either a
successful section or an explicit `> _Scrape failed_` block
in DESIGN.md.
- Run `bun run scripts/m3-coverage-audit.ts` to update the
coverage table that DESIGN.md links to.
5. Commit (single-deliverable):
scripts/design-google.urls.json + docs/DESIGN.md +
docs/audits/m3-coverage.md (regenerated).
6. ScheduleWakeup({ delaySeconds: 604800, prompt: "/design-google-ingest",
reason: "weekly refresh design.google cache" })
— only when invoked under /loop.
docs/DESIGN.md output schema
<!-- SPDX-License-Identifier: Apache-2.0 -->
<!-- GENERATED by .claude/skills/design-google-ingest — DO NOT EDIT BY HAND. -->
# Aphrody Design Reference
Last refreshed: <ISO-8601 UTC timestamp>
Source: design.google + scripts/edge-mass-scrape.ts (Edge headless, virtual-time=15000)
Articles ingested: N / M (M = URLs requested)
## 1. Quick-reference index
| Section | Source URL | Bytes |
|---|---|---|
| ... | ... | ... |
## 2. Foundations
(Articles in the design-systems / typography / accessibility / color categories.)
## 3. Material 3
(Articles tagged m3, material-design-3, design-tokens.)
## 4. Gemini visual identity
(gemini-ai-visual-design + related.)
## 5. Brand assets
(google-sans-flex-font, open-source-custom-fonts-google, transparent-screens.)
## 6. Other library articles
(Everything else, alphabetical.)
## 7. Site pages
(home, about, events, products.)
## 8. Raw failures
(URLs that returned <= 5 KB or an empty shell — needs re-run.)
## 9. Audit
- M3 coverage snapshot (token=X% bridge=Y% catalogue=Z% html=W% overall=V%)
- Aphrody bridge mappings already in lock-step.
- Open follow-ups.
Files this skill touches
| Path | Direction | Role |
|---|
scripts/design-google.urls.json | write | fresh URL list |
scripts/design-google-ingest.ts | invoke | URL discovery + crawl orchestration |
scripts/edge-mass-scrape.ts | invoke | SPA-aware scrape engine |
var/data/edge-cache/<sha256>.html | write | per-URL DOM cache (gitignored) |
var/data/edge-cache/manifest.json | write | run manifest with per-URL status |
docs/DESIGN.md | write | distilled Markdown reference |
docs/audits/m3-coverage.md | refresh | coverage table linked from DESIGN.md |
When NOT to use
- For single-URL scrapes (use
scripts/edge-mass-scrape.ts directly).
- For Linux/Mac native max-power scrape — use Chrome headless.
Related skills / agents / references
design-google-curator agent (.claude/agents/design-google-curator.md) —
the agent the skill dispatches to read the cache + write DESIGN.md.scripts/edge-mass-scrape.ts — the underlying SPA-aware scraper.scripts/m3-coverage-audit.ts — coverage snapshot consumed by §9.docs/audits/2026-05-17-edge-headless-spa-fallback.md — why we
use Edge instead of a static scraper for design.google.docs/audits/2026-05-17-gemini-clone-and-google-sans-flex.md —
prior application of design.google intel to crates/m3-tokens.
