setup-wizard

# setup-wizard — guided config onboarding Turns a freshly-unpacked research-brain into a configured and truthfully activated one. The user opens the project root in Cowork / Claude Code, says "let's set up sources", and this skill leads with the supported product default: > We'll set up the full research-brain path. Advanced users can choose RSS Digest Lite or headless transports. The normal path does not ask the user to choose an implementation mode. It talks in user outcomes: sources, delivery, schedule, first value, and the next action. Advanced setup is explicit and preserves RSS Digest Lite, SMTP, and Telegram Bot API for users who need a headless/no-agent path. It is **conversational** and **idempotent**: on first run it fills in the shipped templates (or copies the starter); on a later run ("let's reconfigure email", "add a topic", "change the timezone") it re-reads the current config and changes only what the user asks. "Add a topic" also updates `tags.yaml`; "change the timezone" only touches `schedule.yaml`. The config files this skill owns: | File | Format | Written in step | |------|--------|-----------------| | `config/topics.yaml` | YAML (schema CONVENTIONS §2.3) — the topics config | a — Topics | | `config/tags.yaml` | YAML (fixed registry, CONVENTIONS §11) | a — derived from topics | | `config/sources.yaml` | YAML (schema §2.3, incl. per-source `tier`) — the sources config | b — Sources | | `config/channels.yaml` | YAML (fixed schema, CONVENTIONS §2.1) | c — Delivery | | `config/schedule.yaml` | YAML (fixed schema, CONVENTIONS §2.2) | d — Schedule | **Config format rule (§2.3):** `config/sources.yaml` / `config/topics.yaml` are the ONLY sources/topics config the runtime reads — every ordinary path in this skill reads/writes the YAML twins via the `manage_sources.py` backend. The legacy `.md` tables are retired (0.8.0): when Step 0 detects an unmigrated legacy install (real `.md` config, no YAML twin), the wizard STOPS and offers exactly one action — `python3 scripts/schema.py migrate-config` — then continues on the YAML files. The shipped archive carries the twins as empty structures; the backend fills them. Secrets are **never** written by this skill — they go in `config/.env` (copied from `config/.env.example`); the YAML only ever holds the *name* of the env var. --- ## When to Use **Use when:** - The user says: "set up sources", "configure", "let's set up", "first-time setup", "onboarding", "starter pack", "настроить источники", "онбординг", "давай настроим". - A fresh checkout where `config/` files still hold the shipped EXAMPLE placeholders. - The user wants to change one part of an existing config ("reconfigure Telegram", "add a topic", "change the digest day", "change the timezone") — run only the relevant step. Adding/removing a topic re-derives `tags.yaml` too. **Skip / don't use for:** - Filling in secret values (tokens, passwords, webhook URLs). Direct the user to `config/.env`. - Running a collection or building a digest — collection is a scheduled task (`schedule/collect-*.md`, not a skill); building is the `digest-build` skill (sending is `digest-deliver`). setup-wizard only writes config (Step f *offers* a rehearsal but sends nothing). - Editing `brain/`, `pipeline/`, or any file outside `config/`. - Hand-gardening `tags.yaml` long-term — after the first derivation it grows automatically via the weekly brain-hygiene pass (`scripts/validate_brain.py`). --- ## DO / DON'T **DO:** - Inspect repo state first; tell the user whether this is a first run or a reconfigure. - On first run, OFFER the starter-pack fast path before the full interview (Step 0). - Ask one topic at a time. Confirm the collected values back before writing each file. - Derive `config/tags.yaml` from the topics (nested `domain/topic`), show it, confirm. - Ask a per-source `Tier` (high/medium/low credibility), default `medium`. - For each chosen channel, explain its PREREQUISITES (MCP connected / webhook created / secret in `.env`). - Write `channels.yaml`, `schedule.yaml`, `tags.yaml` to their EXACT fixed schemas (CONVENTIONS §2.1 / §2.2 / §11). - Write the FULL schedule schema — `collect` / `curate` / `synthesis` / `digest` + `timezone`. - Use relative paths only (`config/...`, `pipeline/1-incoming/...`). Never absolute paths. - Keep topic names in sync between `topics.yaml`, `sources.yaml`, and the `tags.yaml` synonyms. - Detect source type from the URL (rss / blog / medium-tag / webpage); prefer `rss` when a feed exists. - After writing, run `python3 scripts/setup_check.py --runtime agentic --scheduler-state ready-not-registered` and report its activation checklist (Step f). - Before claiming `complete`, verify recurring execution is actually registered. A written `config/schedule.yaml` is only `ready, schedule not registered`. **DON'T:** - Don't write secrets (tokens, passwords, webhook URLs) into any `.yaml` or `.md`. Ever. - Don't change the structure / keys of the YAML schemas — only the values. - Don't wipe a configured file on a reconfigure run — re-read and patch only what changed. - Don't invent topics or sources the user didn't give. Leave a section empty rather than guess. - Don't interrogate the user about tags — derive them, show the result, ask one yes/no confirm. - Don't write absolute paths or any brand/project-specific hardcoding. - Don't actually send a digest — Step f is a dry-run rehearsal only. --- ## Inputs | Input | Type | Required | Default | Description | |-------|------|----------|---------|-------------| | `MODE` | enum `first-run` / `reconfigure` | no | auto-detected | Decided by inspecting config state (see Step 0) | | `USE_STARTER` | bool | no | asked on first run | Copy the starter pack instead of the full interview (Step 0) | | `TARGET` | string | no | all steps | For reconfigure: which part to change ("email", "topics", "schedule", "timezone"...) | | User answers | interactive | yes | — | Collected through the step-by-step interview | Use AskUserQuestion-style prompts: ask one focused question, wait, confirm, then move on. --- ## Steps ### Step 0 — Inspect repo state + offer the starter pack Determine where we are before asking anything. 1. Check which config files exist: `config/sources.yaml` + `config/topics.yaml` (the only sources/topics config the runtime reads), any leftover legacy `config/topics.md` + `config/sources.md`, plus `config/channels.yaml`, `config/schedule.yaml`, `config/tags.yaml`. Note any missing. 2. Detect whether the install is still a **shipped template** (not yet configured). Markers: - `sources.yaml` / `topics.yaml` exist (possibly as the shipped empty structures) → configured iff they carry real topics/sources (`python3 scripts/manage_sources.py list`). - **no YAML twin at all**, but `topics.md` / `sources.md` hold real topic/source content → an **unmigrated legacy install** (the state `scripts/sources/store.py` refuses loudly everywhere) — go to 2a. `.md` files carrying only `# EXAMPLE — replace` rows or the literal `Updated: YYYY-MM-DD` header are templates, not user data — they never block; treat the install as unconfigured. - `channels.yaml`: every channel has `enabled: false` AND `to: []` is empty AND `chat_id: ""` is empty → not yet configured. - `schedule.yaml`: present with default crons → treat as configurable (always offer). 2a. **Unmigrated legacy install → STOP; offer exactly one action — the migration** (never auto-apply): "Your sources/topics config is in the retired markdown format, which the runtime no longer reads — nothing (collection, validation, this wizard's writes) works until it is imported. Run `python3 scripts/schema.py migrate-config`? One command, nothing is lost — your `.md` files stay untouched as the rollback." On yes run it, show its output, and continue the wizard on the YAML files. On no, stop here — leave the config untouched and name the command for later; there is no `.md` path to continue on. 3. Decide `MODE`: - All config files are still templates → **first-run**. Greet, then go to the **starter-pack fork** below. - At least one file is configured → **reconfigure**. Ask which part the user wants to change (`TARGET`) and jump to only that step. Re-read the current file first. 4. Tell the user what you found: e.g. "Fresh setup — none of the config is filled in yet." or "Looks configured already — what would you like to change? (topics / sources / delivery / schedule / timezone / language)". On first run, lead with this exact default copy: > We'll set up the full research-brain path. Advanced users can choose RSS Digest Lite or headless transports. Do **not** ask the user to choose `agentic`, `headless`, or `RSS Digest Lite` unless they explicitly ask for advanced setup. ### Advanced setup fork Use this fork only when the user explicitly asks for advanced/headless setup, RSS Digest Lite, SMTP, Telegram Bot API, cron-only operation, or when the current runtime cannot support the recommended MCP transport. Advanced choices preserve existing capability: - **RSS Digest Lite** — deterministic RSS collection + sanitize + digest + headless delivery. This path requires at least one active `rss` source. - **Email SMTP** — set `provider: smtp`, collect SMTP host/user/recipient, and keep `password_env: SMTP_PASSWORD`. - **Telegram Bot API** — set `transport: bot-api`, collect `chat_id`, and keep `token_env: TELEGRAM_BOT_TOKEN`. The normal first-run path stays full agentic with MCP-first delivery defaults. **Starter-pack fork (first-run only).** Before the full interview, ask one question: > "Want a ready-to-go starter so you get a working setup in one step, then tweak? > It imports `config/sources.example-starter.md` + `config/topics.example-starter.md` > (public feeds for **tech-news** + **ai-ml**) into your `sources.yaml` / `topics.yaml`." - **Yes (`USE_STARTER`)** → copy the two starter files in as migration input: `cp config/sources.example-starter.md config/sources.md` and `cp config/topics.example-starter.md config/topics.md`. Stamp today's date over the `Updated: YYYY-MM-DD` header in both, then land them in the YAML config: `python3 scripts/schema.py migrate-config --force` (writes `config/sources.yaml` + `config/topics.yaml` from the copied starters; the copied `.md` files are import material only — they can be deleted after). Then **still derive `config/tags.yaml`** from the starter topics (Step a.4), and continue to delivery (Step c), schedule (Step d), language (Step e), and verify (Step f). Skip the topic/source interview (a.1–a.3, b). - **No** → run the full interview a → e, then f. Read commands (illustrative, relative paths): - `ls config/sources.yaml config/topics.yaml` — do the YAML twins exist yet? - `python3 scripts/manage_sources.py list` — what is actually configured (reads the YAML; on an unmigrated legacy install it fails with the migrate-config pointer — Step 2a). - `grep -l "EXAMPLE — replace" config/*.md` — leftover `.md` files that are templates, not user data (they never block; real `.md` content without YAML twins is Step 2a). - Read `config/channels.yaml` and check the `enabled:` flags. ### Step a — Topics / categories → `config/topics.yaml` (+ derive `config/tags.yaml`) 1. Ask: "What topics do you want to track? Give me a short name for each (e.g. `ai-agents`, `ai-governance`)." Collect a list. 2. For each topic, ask (can batch sensibly): include rules (what belongs), exclude rules (what to drop), and frontmatter tags (default to `[<name>]` if the user has no preference). 3. Confirm the full topic list back to the user. 4. **Derive `config/tags.yaml`** from the topics (auto-derive + grow — CONVENTIONS §11). For each topic, split a `<domain>-<topic>` name into a nested `domain/topic` tag: - `ai-agents` → domain `ai`, topic `agents` → tag `ai/agents` - `ai-governance` → domain `ai`, topic `governance` → tag `ai/governance` - `tech-news` → domain `tech`, topic `news` → tag `tech/news` - a single-word topic (`security`) → use it as the topic under a sensible domain, or `misc/<topic>` if no domain is implied; ask only if genuinely ambiguous. Then write the registry: - `domains:` — each domain mapped to its list of topics (e.g. `ai: [agents, governance]`). - `synonyms:` — map the **raw topic name** and obvious aliases to the canonical nested tag (`ai-agents: ai/agents`, `agentic: ai/agents`, `tech-news: tech/news`). This is what RSS `<category>` values and concept tags normalize through. - `pending: []` — empty quarantine for tags the system proposes later. Show the user the resulting tag list (e.g. "domains: ai → agents, governance; tech → news") and confirm with ONE yes/no — don't interrogate. Tell them: **"You never hand-manage tags — this vocabulary grows automatically. When a new tag recurs across ≥3 notes the weekly hygiene pass *recommends* it (`scripts/validate_brain.py`, advisory) and `scripts/promote_tags.py` *applies* the promotion from `pending:` into the registry."** Keep the file's header comment block intact; only fill `domains:` / `synonyms:` / `pending:`. 5. Write the topics into `config/topics.yaml`. First create each topic through the deterministic backend (it writes the YAML twins and keeps names in sync): `python3 scripts/manage_sources.py add-topic <name>` — then set the interview's include/exclude rules and tags on each entry by editing `config/topics.yaml` — one `topics:` entry per topic (schema CONVENTIONS §2.3); replace the backend's default prose with the user's answers: ```yaml topics: - name: ai-agents tags: [ai-agents, agents] include: "Substantive pieces about autonomous agents, orchestration, tool use." exclude: "Generic LLM hype with no agent angle; route product news elsewhere." ``` > On a **reconfigure** "add/remove a topic": use `manage_sources.py add-topic` / > `remove-topic` (it patches BOTH `config/topics.yaml` and `config/sources.yaml`), > AND re-derive the `tags.yaml` entry for that topic (add the `domain/topic` to > `domains:` + a `synonyms:` alias; never delete a tag that existing concepts may use — > leave it or fold it up to its domain). ### Step b — Sources per topic → `config/sources.yaml` (Skipped when the starter pack was copied — the starter already carries sources with tiers.) Source establishment should reuse the Phase 1 `intake` path when the user supplies OPML, a plain URL, or an explicit shortcut: - OPML import: run the `intake` flow, preview first, then apply only after confirmation. - Bare URL: run `python3 scripts/intake.py URL --json`; approve the proposed recurring-source handoff before persistence. - Explicit `/add <url>` or explicit monitor/add intent: route directly to `source-manage`. - Explicit `/save <url>` remains a read-it-later capture path and is not part of source setup unless the user deliberately wants to capture an article during onboarding. 1. For each topic from Step a, ask: "Which URLs should we collect `<topic>` from?" 2. For each URL, **detect the type** and confirm: - ends in `.xml` / `/feed` / `/rss` / `/atom`, or is a known feed path → `rss` - `medium.com/tag/<tag>` (optionally `/latest`) → `medium-tag` - a blog *index* page (e.g. `.../blog`) → `blog` - anything else (a single page, no feed) → `webpage` **Prefer `rss` whenever a feed exists** — say so to the user. The verified feed