trending-digest

name	trending-digest
description	Builds a self-contained multi-source trending / hot-topic Markdown digest for any subject (AI, gaming, crypto, research, news, ...). Pulls fresh content in parallel from X/Twitter, Hacker News, Reddit, GitHub trending, arXiv, 微博 (Weibo), 36kr — plus any new source modules you drop into scripts/sources/. Use this skill whenever the user asks to find trending/hot/popular content on a topic, monitor what people are saying across social media or news sites, build a "top N posts" digest, aggregate AI/tech/research signals into a report, summarize cross-platform discussion, track Chinese (国内) + international (国外) coverage of an event, or curate a multimedia weekly digest with embedded images and videos. Triggers on phrases like "find hot posts about X", "X 热门帖", "trending now on AI", "monitor what people are saying about Z", "整理热点", "AI 资讯汇总", "Twitter + HN + Reddit trending report", "中文 + 英文 热点", "tech news roundup", "summarize discussion across platforms". The skill ships ready-made scrapers for 7 sources and is designed to be extended — each source is one file in scripts/sources/ exposing a `discover(topic, limit)` function.
argument-hint	[topics, comma-separated] [optional: sources to use]
allowed-tools	Bash, Read, Write, Edit, WebFetch, Glob, Grep

Trending Digest — Multi-source Info Retrieval → MD Report

A pipeline that fans out across many public APIs and HTML scrapers, normalizes everything into a single canonical Post schema, and writes a polished Markdown report with embedded images, HTML5 video players, summary tables, and methodology transparency.

Why this skill exists

People who track AI / tech / news want one report covering everything — not 5 separate tabs of X, HN, Reddit, arXiv. Building such an aggregator yourself means each source has its own quirks (X has a login wall, Reddit needs a UA, GitHub trending is HTML-only, 微博 uses a mobile JSON endpoint, etc.). This skill encodes all those workarounds and gives a single orchestrate.py runner that does the fan-out in parallel.

The output is a self-contained folder: <output>/<slug>/{report.md, assets/*} that renders correctly in GitHub / VSCode preview / Typora with centered fixed-width media that preserves aspect ratio.

Pipeline overview

Stage 1: ORCHESTRATE  — per topic × per source, in parallel
  Social / community:
    ├─ scripts/sources/x.py        → Twitter (syndication API + Nitter discovery)
    ├─ scripts/sources/reddit.py   → Reddit (.json endpoint)
    └─ scripts/sources/hn.py       → Hacker News (Algolia)
  AI news (国外):
    ├─ scripts/sources/ainews.py   → 10 RSS feeds (the-decoder / VentureBeat AI /
    │                                 TechCrunch AI / Ars / Verge / MIT Tech Review /
    │                                 Marktechpost / Synced / HF blog / Wired AI)
    └─ scripts/sources/huggingface_papers.py → HF Daily Papers (community-curated trending)
  Research / code:
    ├─ scripts/sources/arxiv.py    → arXiv Atom query
    └─ scripts/sources/github_trending.py → GitHub trending HTML
  国内 AI 资讯:
    ├─ scripts/sources/weibo.py    → 微博 mobile API
    ├─ scripts/sources/kr36.py     → 36kr search / hot rank
    └─ scripts/sources/ainews_cn.py → 机器之心 / 量子位 / InfoQ 中国 AI

  → (extend by dropping any new source.py here)

Stage 2: CURATE  — rank, dedup, filter spam/marketing, distribute across topics

Stage 3: DOWNLOAD MEDIA  — images + video posters into <out>/assets/

Stage 4: RENDER MD  — TL;DR table + Top-N table + per-topic detail + methodology
                      with centered <img>/<video> blocks, fixed widths

How to use this skill

Step 0 — clarify scope (if not obvious)

Ask only if missing:

• Topics (1-5): what subjects? e.g. "AI agent", "Grok Ani", "3D gesture generation"
• Sources (default: x,hn,reddit,github,arxiv): which ones? For 国内 topics also include weibo,kr36
• N (default 10): how many posts in the final report?
• Output location (default claude_workspace/analysis/shared/<slug>/)

If the user gave a clear request like "AI 热点 + 国内国外", default to:

• Topics: the AI subject they imply (agent, LLM news, etc.)
• Sources: x,hn,reddit,arxiv for 国外 + weibo,kr36 for 国内
• N=10
• Output: analysis/shared/<topic-slug>-<YYYY-MM-DD>/

Step 1 — orchestrate discovery across sources

python3 scripts/orchestrate.py \
    --topics "AI agent,Grok Ani,3D gesture" \
    --sources x,hn,reddit,arxiv \
    --limit-per-source 15 \
    --out <work-dir>/candidates.json

Or pass a YAML/JSON config for per-topic source customization:

# config.yaml
defaults:
  limit_per_source: 15
topics:
  - {key: agent, query: "AI agent autonomous", sources: [x, hn, reddit]}
  - {key: 3d-gesture, query: "3d gesture generation", sources: [arxiv, x]}
  - {key: 国内ai, query: "AI", sources: [weibo, kr36]}

python3 scripts/orchestrate.py --config config.yaml --out candidates.json

The output candidates.json shape:

{
  "posts": [<canonical Post dict>, ...],
  "stats": {
    "jobs": 12, "raw_posts": 184, "deduped": 172, "elapsed_s": 18.3,
    "per_source": {"x": 45, "hn": 30, ...},
    "per_topic":  {"agent": 60, ...}
  },
  "by_source_topic": [...]
}

See references/post-schema.md for the canonical Post dict.

Step 2 — curate

The default curator is judgment-driven. Read candidates.json and apply this rubric per topic, distributing the final N picks proportionally:

Signal	Effect
engagement.score (likes / points / stars)	primary ranking
engagement.comments	secondary (real discussion)
source_specific.verified (X) / contributor reputation	quality boost
Text length > 30 chars	filter empty / shilly one-liners
has_image / has_video	richer for the MD report
Spam patterns	DEMOTE: marketing copy, crypto/Web3 promo, follower-pitch lead-ins

If reproducibility matters, write a curate.py that scores by engagement.score × log(1 + comments) and dedupes. For most one-off requests, manual curation in the conversation is faster.

Write the picks into curated.json:

{"selected": [<picked posts>, ...]}

Step 3 — download media locally

python3 scripts/download_media.py \
    --curated curated.json \
    --assets-dir <out>/assets \
    --rel-from <out>

Downloads each post's images[] + video_poster and writes back local_images[] / local_video_poster to curated.json.

Step 4 — render the MD

python3 scripts/build_md.py \
    --curated curated.json \
    --out-dir <out> \
    --slug <slug> \
    --title "<title>"

Creates <out>/<slug>.md plus assets/. The MD uses:

• <p align="center"><img src="assets/..." width="720"/></p> for centered images
• <div align="center"><video controls width="720" poster="..."><source ...></video></div> for videos
• Default 720px width; 800px for 16:9 / text-dense; native width for small (no upscale)
• TL;DR table + Top-N table + per-topic detail + mandatory methodology section

Source modules

Each scripts/sources/<name>.py exposes:

def discover(topic: str, limit: int = 20, **kwargs) -> list[dict]:
    """Return list of canonical Post dicts matching the topic."""

Available (2026-06):

Source	Module	Domain	Auth?
X / Twitter	x.py	English/international social	❌ (syndication API + Nitter)
Hacker News	hn.py	Tech/startup discussion	❌
Reddit	reddit.py	Broad community discussion	❌ (UA required)
GitHub trending	github_trending.py	Open-source projects	❌
arXiv	arxiv.py	Research papers	❌
AI news (intl)	ainews.py	10 RSS feeds: the-decoder / VentureBeat AI / TechCrunch AI / Ars / Verge / MIT Tech Review / Marktechpost / Synced / HF blog / Wired AI	❌
HuggingFace Daily Papers	huggingface_papers.py	Community-curated trending arXiv papers	❌
AI news (国内)	ainews_cn.py	机器之心 / 量子位 / InfoQ 中国 AI	❌
微博 (Weibo)	weibo.py	中文 social	❌ (rate-limited)
36kr	kr36.py	国内 AI/创业资讯	❌

See references/sources-matrix.md for endpoints, rate limits, known issues.

Adding a new source

# scripts/sources/my_source.py
from _base import make_post, http_get_json, register

@register('mysource')
def discover(topic: str, limit: int = 20, **kw) -> list[dict]:
    data = http_get_json(f'https://api.example.com/search?q={topic}&n={limit}')
    return [
        make_post(
            source='mysource',
            source_id=item['id'],
            url=item['url'],
            title=item['title'],
            author=item['author'],
            text=item['snippet'],
            topic=topic,
            published_at=item['ts'],
            score=item['likes'],
            comments=item['comments'],
        )
        for item in data.get('items', [])
    ]

After adding, invoke via --sources mysource. orchestrate.py discovers modules dynamically from sources/.

Reproducibility / methodology transparency

The MD report MUST include a Methodology section (auto-generated by build_md.py) documenting:

1. Which sources were called, and the API/method used
2. Per-source counts in the final report
3. Known limitations (X retweet missing, no time filter, etc.)

Non-negotiable — without provenance, the report is just summarized hearsay.

Bundled scripts

Script	Purpose
scripts/orchestrate.py	Multi-source fan-out runner (main entry)
scripts/sources/_base.py	Shared Post-schema helpers + register() decorator
scripts/sources/x.py	X via syndication API + Nitter discovery
scripts/sources/hn.py	Hacker News Algolia search
scripts/sources/reddit.py	Reddit .json endpoint
scripts/sources/github_trending.py	GitHub trending HTML scraper
scripts/sources/arxiv.py	arXiv Atom query API
scripts/sources/ainews.py	10 国外 AI 资讯站 RSS 聚合（the-decoder / VentureBeat AI / TechCrunch / Ars / Verge / MIT TR / Marktechpost / Synced / HF blog / Wired AI）
scripts/sources/huggingface_papers.py	HuggingFace Daily Papers JSON API（community upvote 排序）
scripts/sources/ainews_cn.py	国内 AI 媒体（机器之心 / 量子位 / InfoQ 中国 AI）
scripts/sources/weibo.py	微博 mobile site API
scripts/sources/kr36.py	36kr search / hot rank
scripts/download_media.py	Download images + video posters
scripts/build_md.py	Render MD with centered media + tables
scripts/verify_candidates.py	(X-specific) bulk verify URL/ID list

Bundled references

• references/sources-matrix.md — per-source API endpoints + rate limits + known issues
• references/post-schema.md — canonical Post dict shape (multi-source)
• references/syndication-api.md — X syndication API reference + token derivation
• references/discovery-strategies.md — discovery technique catalogue
• references/md-template.md — MD output layout + rendering rules

Output location convention

Default: claude_workspace/analysis/shared/<slug>/

<slug> is <topic-keyword>-<YYYY-MM-DD>. Keep the MD + assets/ together — never put assets in a global tmp/ folder where they'd get cleaned.

When NOT to use this skill

• User wants live API metrics — most sources cache ~5-15 min
• User wants raw data (just JSON) — skip build_md step
• User wants to post / reply / DM — this skill is read-only
• User wants TikTok / Facebook / Instagram — heavily gated, not covered

Failure modes & graceful degradation

Failure	Behavior
One source 503 / empty	Other sources still proceed; methodology notes the failure
All sources fail for a topic	Empty topic in report; gap is visible
Image download fails	Skip that image, render text-only
Video CDN URL 403 later	Local poster <img> fallback still works
Rate-limit hit	Per-source discover() sleeps + retries

Pushy reminder

If the user mentions "热门", "trending", "热点", "viral", "hot posts", "什么火", "最近大家在聊什么", "AI 资讯", "weekly digest", or wants to summarize discussion across multiple platforms (X + HN + Reddit + 微博) — invoke this skill. The "multi-source + canonical schema + MD output" loop is exactly what it does; otherwise you'd spend an hour writing per-source scrapers from scratch.