daily-brief

name

daily-brief

description

Operational knowledge for the daily-brief digest pipeline (this project). RSS/API fetchers, pluggable LLM enrichment (default claude CLI on Max; also anthropic/openai/deepseek/minimax API), trading section, HTML rendering, cross-platform scheduler integration (Windows Task Scheduler / macOS launchd / Linux cron). Load when the user asks about running daily / regenerating sections / debugging a failed run / adding or disabling sources / LLM quota / scheduler / why a tab shows wrong data / why a source failed / switching LLM backend. Always prefer the documented npm commands over re-implementing logic. Diagnose by reading logs/daily-*.log first, then logs/llm-calls.jsonl for LLM-side issues.

daily-brief — Operational Skill

This project generates a single-page HTML daily digest covering tech / finance / politics / market data / community discussion. The pipeline runs locally via the OS scheduler (Windows Task Scheduler / macOS launchd / Linux cron, default 08:00 local time) and emits daily_reports/<YYYY-MM-DD>/<YYYY-MM-DD>.html + sidecar files (each date gets its own subdir). The date label uses the system local timezone by default — set REPORT_TZ (e.g. Asia/Shanghai, UTC) in .env.local to override.

Detailed architecture lives in code; this skill is a cheat sheet for operating and diagnosing, not a re-explanation of the system.

Project root assumption

All paths in this skill are relative to the project root (the directory that contains package.json, lib/, scripts/).

Before any command, ensure the working directory is the project root. Two cases:

1. Claude Code session opened inside the project — already there, no action needed

2. Session opened elsewhere — read the config file and cd:

   # Cross-platform Node one-liner (prints the project root path):
   node -e "const fs=require('fs'),os=require('os'),path=require('path');const cfg=path.join(os.homedir(),'.daily-brief-config');if(fs.existsSync(cfg))console.log(fs.readFileSync(cfg,'utf8').trim());else process.exit(1)"
   

   Use the printed path: cd "$(...)" on bash / Set-Location (...) in PowerShell.

The config file is written by node scripts/install.mjs --global. If it's missing the user hasn't done a global install — tell them to run it.

Quick command reference

Need	Command	Cost
Full pipeline	npm run daily	~5-8 min, ~6 Sonnet calls
Fetch sanity only (no LLM)	npm run dry-run	~30s
Re-render existing sidecar	npm run render [date]	<1s
Re-run trading section	npm run regen-trading [date]	~2 min, 1 LLM call
Top-up missing summary	npm run regen-enrich <cat:sub> [date]	~20-40s, 1 LLM call
Open today's report in Chrome	npm run open	instant
Sonnet quota + call history	npm run quota-report	instant

[date] defaults to today's date in the report timezone (system local, or REPORT_TZ if set). The pipeline and the OS scheduler both run in local time, so the report's date label = the date when the trigger fired in the report timezone. A user with REPORT_TZ=Asia/Shanghai whose machine fires the trigger at 23:00 UTC-8 will get a "next-day Shanghai" file, e.g. daily_reports/2026-05-17/2026-05-17.html.

<cat:sub> accepted by regen-enrich: finance:news, politics:world, tech:ai-news. Single-source X 推文 (tech:x-viral) is enriched as part of daily only — no top-up path.

File map — where to change what

Task	File
Add / disable / re-categorize a source	sources.config.json (project root — single source of truth; lib/sources/registry.ts is just a loader)
Rename L1 tab labels	lib/output/render.ts CATEGORY_LABELS
Reorder / rename L2 subcategories	SUBCATEGORY_ORDER + SUBCATEGORY_LABELS in same file
Change per-source item cap	SOURCE_DISPLAY_LIMITS
Change merged-timeline cap	MERGED_SUBGROUP_LIMITS
Add a Sonnet enrichment prompt	lib/ai/enrich.ts — copy XVIRAL_SYSTEM_PROMPT pattern
Wire an enrichment into pipeline	scripts/daily.ts — await enrichXxx(articles) in main()
Add a new fetcher type	New file in lib/sources/ + branch in lib/sources/dispatch.ts
Adjust HTML styling	inline <style> block in renderHtml() in lib/output/render.ts
Change scheduler trigger time	node scripts/install.mjs --at HH:MM (re-registers)
Wrapper script the scheduler invokes	scripts/run-daily.mjs

How LLM enrichment works (mental model)

• Each merged L2 subcategory gets a Sonnet pass: GH-trending (per-source), finance:news, politics:world, tech:ai-news, tech:x-viral.
• Each pass = one batched Sonnet call for all items in that subgroup. Don't iterate per-item.
• Sources with lang: "zh" in registry skip enrichment (already Chinese).
• Failures are non-fatal: skipped articles just render without summary.

Diagnostic flow

Order matters — top-to-bottom:

"今天日报没出来" / "Chrome 没弹"

1. Check scheduled task state — platform-specific:
   • Windows: Get-ScheduledTaskInfo -TaskName DailyBrief → LastRunTime + LastTaskResult (0=success, 267009=running, else failed)
   • macOS: launchctl list | grep com.daily-brief (PID column + last exit code)
   • Linux: cron doesn't track per-job state; look at logs/cron.log
2. Tail today's log (date = local, not UTC):

   node -e "const fs=require('fs'),d=new Date(),pad=n=>String(n).padStart(2,'0');console.log(fs.readFileSync('logs/daily-'+d.getFullYear()+'-'+pad(d.getMonth()+1)+'-'+pad(d.getDate())+'.log','utf8').split('\n').slice(-40).join('\n'))"
   

3. Check report files exist: ls daily_reports/<date>/ (any platform) or Get-ChildItem daily_reports\<date>\ (Windows)

"某个源数据不对 / 0 条"

1. Look at fetch lines near top of log — <id> <count> or <id> FAILED — <reason>
2. If specific source failed: read its fetcher in lib/sources/<source>.ts
3. If Cloudflare-related: see "LinuxDo lesson" below
4. Single-source failure must never kill the run (try/catch per source in daily.ts)

"LLM 调用炸 / 中文摘要缺失"

1. npm run quota-report — per-backend summary; for claude-cli shows 5h window, for API backends shows 24h spending
2. If quota hot on claude-cli: wait or temporarily switch via .env.local (LLM_BACKEND=openai etc.)
3. If specific phase missing summaries: npm run regen-enrich <cat:sub>
4. Each call logged to logs/llm-calls.jsonl (legacy claude-calls.jsonl still read for backwards-compat) — grep "success":false, see errorCategory (quota / timeout / auth / other)
5. Which backend is active = LLM_BACKEND env in .env.local; not set → claude-cli

"UI 出错 / 某个 tab 显示异常"

1. npm run render (1 second) — often fixes display-only bugs
2. If still wrong: read rendered HTML for the affected panel
3. renderRawCategoryPanel / renderSubContent chain in render.ts is where panel structure lives

Recurring failure patterns (institutional knowledge)

LinuxDo / Cloudflare WAF

• LinuxDo is behind Cloudflare and frequently flags datacenter-IP exits with "Just a moment..." challenges
• Do NOT add aggressive retry to its fetcher — burst requests escalate the WAF flag, causing persistent blocks
• May ship with enabled: false depending on current IP rep
• Browser works because of cookies + JS challenge; curl can't do either
• If re-enabling: keep single attempt, accept intermittent failures

Run-daily.mjs wrapper notes

• Tees npm run daily stdout+stderr to logs/daily-<local-date>.log via stream pipes (real-time, not buffered)
• Exit code from npm run daily is propagated to the OS scheduler
• On exit 0: spawns npm run open detached so Chrome opens without blocking
• Cross-platform: same .mjs file works on Windows / macOS / Linux

"X 推文 出现非英文"

• API's lang=en param is best-effort; some slip through
• Fix in lib/sources/attentionvc.ts isEnglish() — checks langsDetected (most reliable) + lang === "zxx" (image/code-only, kept)

"社区讨论 tab 偶发空白"

• Was a JS scope bug: sub-tab/source-tab handlers used data-cat="tech" selector. Tech main panel AND community panel both had sub-content with data-cat="tech", so clicking AI 媒体 in tech panel deactivated cn-community in community panel
• Fixed: handlers use btn.closest('.panel') + btn.closest('.sub-content'). If regression, look at inline <script> block at end of renderHtml()

Trading commentary "watchlist empty"

• Sonnet occasionally returns valid JSON with empty watchlist. 1-shot retry built into lib/ai/trading-commentary.ts with stronger prompt
• If retry also fails, render falls back to empty trading panel — run isn't aborted

Source registry conventions

Sources live in sources.config.json at the project root. lib/sources/registry.ts only loads + validates that JSON at module-init; never hardcode sources in TS.

• Every source has: id, name, type (rss/api/scrape), url, category, subcategory?, enabled?, useCurl?, lang?, locales?
• useCurl: true for sources behind Cloudflare-style TLS-fingerprint blocks
• lang: "zh" (or "en") for sources already in a specific language — enrich skips them when REPORT_LOCALE matches
• locales: ["zh"|"en"] filters which REPORT_LOCALE keeps the source. Omit → both
• Disabled sources stay in the JSON with enabled: false + a notes field explaining why — don't delete
• Run npm run sources to see the table by category with current enable/filter status

Render layout (current, may evolve)

L1 tabs in order: tech / trading / politics / finance / community

技术动态 (tech)
  L2: GitHub Trending  (per-source, cap 20)
  L2: X 推文           (single source attentionvc-ai, cap 20, preserve fetch order)
  L2: AI 媒体          (merged 7 RSS sources, cap 15, summary)

市场行情 (trading)
  asset-group tabs: macro / 美股 / 加密 / 中港 / 商品外汇

时政观察 (politics:world)
  merged single timeline, cap 15, summary, sports filtered

财经要点 (finance:news)
  merged single timeline, cap 12, summary

社区讨论 (community)
  Source tabs: V2EX / LinuxDo (cap 10 each)
  Note: cn-community is registered under category=tech but rendered as its
  own L1 panel — see TECH_MAIN_SUBS vs TECH_COMMUNITY_SUBS in render.ts

Scheduler integration (cross-platform)

scripts/install.mjs registers the daily trigger via the OS-native scheduler:

OS	Mechanism	Wake-from-sleep
Windows	Task Scheduler "DailyBrief" (WakeToRun, AllowStartIfOnBatteries, StartWhenAvailable) + power-plan wake timers	✓ wakes laptop
macOS	launchd plist ~/Library/LaunchAgents/com.daily-brief.plist	✗ doesn't wake; configure pmset separately if needed
Linux	crontab entry tagged # daily-brief	✗ cron doesn't fire while suspended — run skipped

Common:

• Default trigger: 08:00 local time (--at HH:MM to change)
• Runs as current interactive user — required because claude CLI's OAuth token lives in user profile
• Execution timeout: 30 min (Windows only; macOS/Linux no built-in timeout)
• Set up: node scripts/install.mjs [--at HH:MM] [--global]
• Tear down: node scripts/uninstall.mjs
• Inspect: Get-ScheduledTask DailyBrief | fl or taskschd.msc GUI

What NOT to do

• Don't console.log debugging that won't survive — use structured logger or write to logs/
• Don't add process.exit(1) deep in a fetcher; let daily.ts's per-source try/catch handle it
• Don't bypass runLlm (lib/ai/llm.ts) by importing a specific backend directly — that defeats the LLM_BACKEND switch and pins call sites to one provider
• Don't change the default backend silently; if user has Max subscription they almost certainly want claude-cli to keep using it. Switching to API costs them money
• Don't put AI-generated digest fields (hero_headline, editor_note, keywords) back into the HTML view — they're intentionally hidden. Still generated and live in <date>.json and <date>.md for archive
• Don't add Playwright / Puppeteer dependencies casually — project uses curl + JSON APIs to stay light