news-aggregator-skill

name	news-aggregator-skill
version	1.0.0
author	G-HunterAi
license	MIT
description	Real-time news aggregation and deep analysis from 8+ major sources including Hacker News, GitHub Trending, Product Hunt, tech news, and finance. Fetch, filter, and interpret hot topics across domains.
homepage	https://news.ycombinator.com/
tags	["news","aggregation","trending","tech","finance","research"]
platforms	["all"]
category	research
emoji	📰
metadata	{"clawdbot":{"emoji":"📰","requires":{"bins":["python3"],"env":[]}}}

News Aggregator Skill

Fetch real-time hot news from multiple sources, filter by interests, and generate deep interpretations of trending topics.

Supported News Sources

Source	Focus	Coverage	API
Hacker News	Technology, startups, AI	Silicon Valley/global	REST
GitHub Trending	Open source projects	Developer tools, libraries	Web scrape
Product Hunt	New products, launches	Tech products	Web scrape
36Kr	Chinese startups, tech	China/Asia	Web scrape
Tencent News	Technology, general news	China-focused	Web scrape
WallStreetCN	Finance, markets, crypto	Financial markets	Web scrape
V2EX	Developer community topics	Chinese developer forums	Web scrape
Weibo	Social trending, hot search	Chinese social trends	Web scrape

---

When to Use

• Daily intelligence gathering — Scan multiple sources each morning
• Briefings by topic — AI news, finance updates, open source trends
• Competitive intelligence — Track startup launches, product releases
• Trend spotting — Identify emerging topics across domains
• Deep analysis — Understand why specific stories are trending
• Multi-language coverage — Monitor both English and Chinese sources

When NOT to Use

• Breaking news response — This is best-effort aggregation, not real-time alerts
• Social media monitoring — Use dedicated social listening tools instead
• Detailed research — Use in-depth-research skill for thorough investigation
• Trading decisions — Financial data is for informational purposes only

---

Prerequisites

• Python 3.10+ — For running the aggregator script
• requests library — For HTTP requests (pip install requests)
• Internet connection — Access to news sources
• Optional: Pandas — For advanced filtering and export

---

Core Commands

Single Source Fetch

python3 fetch_news.py --source hackernews --limit 10 --deep

Parameters:

• --source — hackernews, github, producthunt, 36kr, tencent, wallstreetcn, v2ex, weibo, or all
• --limit — Number of items per source (default: 10, max: 50)
• --deep — Enable deep fetching (downloads full article content)
• --keyword — Comma-separated filters (e.g., "AI,LLM,GPT")

Keyword Filtering

Always expand simple keywords to cover entire domain:

# User asks: "AI news from Hacker News"
# You expand keywords and run:
python3 fetch_news.py --source hackernews --limit 20 \
  --keyword "AI,LLM,GPT,Claude,Generative,Machine Learning,RAG,Agent,Neural" \
  --deep

Global Scan (All Sources)

# Fetch from all sources for comprehensive view
python3 fetch_news.py --source all --limit 15 --deep

Output: ~120 items total across all sources (requires semantic filtering).

Specific Topic Search

# Look for specific, unique term
python3 fetch_news.py --source all --limit 10 --keyword "DeepSeek" --deep

---

Output Format

Default Output (JSON)

[
  {
    "title": "OpenAI Releases GPT-4 Turbo",
    "url": "https://news.ycombinator.com/item?id=...",
    "source": "hackernews",
    "score": 1240,
    "timestamp": "2026-03-21T10:30:00Z",
    "metadata": {
      "comments": 430,
      "age_hours": 2
    }
  }
]

With --deep Flag (Includes Content)

[
  {
    "title": "...",
    "url": "...",
    "source": "...",
    "score": 1240,
    "timestamp": "2026-03-21T10:30:00Z",
    "content": "Full extracted article text..."
  }
]

---

Smart Filtering & Reporting

Time Window Handling

If user requests specific time range (e.g., "past 5 hours") and results are sparse (< 5 items):

1. Prioritize requested window — List items strictly within time window first
2. Smart fill — Add high-value items from wider range (past 24h) if needed
3. Clearly annotate — Mark supplementary older items (e.g., "⚠️ 18h ago", "🔥 24h Hot")
4. Maintain quality — Always prioritize SOTA (state-of-the-art), major releases, high heat

GitHub Trending Exception

For purely list-based sources (GitHub Trending):

• List ALL fetched items (e.g., Top 10-25)
• Do NOT apply "smart fill" to extend beyond original list
• Perform deep analysis on each item:
  • Core Value: What problem does it solve? Why trending?
  • Inspiration: Technical insights, product lessons
  • Scenarios: 3-5 keywords describing use case (e.g., #RAG #LocalFirst #Rust)

---

Response Guidelines

Language Options

Make output language configurable (default: English):

python3 fetch_news.py --source all --language english --deep
python3 fetch_news.py --source all --language 中文 --deep

Format & Style

• Style: Magazine/newsletter vibe (like The Economist or Morning Brew)
• Professional yet engaging — Respect reader intelligence but stay accessible
• Concise — Value reader time; no fluff

Report Structure

## 🌍 Global Headlines
[Top 3-5 most critical stories across all domains]

## 🚀 Technology & AI
[AI, LLM, and tech-specific items with deep analysis]

## 📈 Finance & Markets
[Market news, crypto, economic updates]

## 🔬 Science & Innovation
[Research breakthroughs, emerging tech]

## 🌟 Open Source & Developer Tools
[GitHub trending, dev tools, libraries]

Item Format (Template)

### 1. [Title as Markdown Link](https://url)

**Source:** Hacker News | **Heat:** 1,240 points | **Time:** 3 hours ago

A punchy one-liner explaining the "so what" of this news.

**Why it matters:**
- Key implication 1
- Key implication 2
- Connection to broader trends

**For your context:**
[Optional: relevance to your research/interests]

---

Complete Workflow Examples

Example 1: Daily Tech Briefing

# Scan all sources for tech/AI news from past 24h
python3 fetch_news.py --source all --limit 15 --deep \
  --keyword "AI,LLM,GPT,Tech,Software,Startup,Innovation"

# Output: Generate briefing report
# Format: Markdown with links, scores, timestamp
# Save to: reports/tech_briefing_2026-03-21_0800.md

Report output:

# Tech & AI Briefing — March 21, 2026

## 🚀 Top Stories

### 1. [OpenAI Releases GPT-5 Preview](https://...)
**HN: 2,430 points | 4 hours ago | 580 comments**

The most significant AI release this quarter...

**Why this matters:**
- New capabilities in...
- Implications for...
- Competition from...

---

Example 2: Open Source Trend Scout

# Deep scan GitHub Trending + V2EX tech
python3 fetch_news.py --source github --limit 25 --deep
python3 fetch_news.py --source v2ex --limit 20 --deep

# For each project, analyze:
# 1. What problem does it solve? (核心价值)
# 2. What's the technical insight? (启发思考)
# 3. Use case tags (场景标签)

Example 3: Finance Market Watch

# Scan financial news sources
python3 fetch_news.py --source wallstreetcn --limit 15 --deep \
  --keyword "Market,Crypto,Stock,Finance,Economy,Trading"

# Highlight:
# - Major market moves
# - Policy announcements
# - Crypto developments

---

Advanced Features

Concurrent Fetching

The Python aggregator uses concurrent requests for speed:

# Fetches from multiple sources simultaneously
# 8 sources × 15 items each = ~120 items in 3-5 seconds

Deep Content Extraction

With --deep flag, the aggregator:

1. Fetches each article URL
2. Extracts main text content
3. Cleans HTML, removes ads/junk
4. Returns plain text for analysis

This enables:

• Full content analysis
• Summarization
• Quote extraction
• Fact checking

Semantic Filtering

For large result sets (100+ items), filter by topic:

# Get all items, then manually filter by domain
python3 fetch_news.py --source all --limit 50 --deep \
  | grep -E "AI|LLM|Agent" \
  | jq '.[] | {title, url, score}'

---

Error Handling

Error	Cause	Solution
Connection timeout	Source unavailable	Retry after 5 min; check internet
Empty results	Keyword too specific	Broaden keyword search
Rate limit (429)	Too many requests	Add delay between calls; reduce limit
Partial results	Source down	Proceed with other sources
Bad HTML parsing	Page structure changed	Check source status; update parser

---

Output Artifacts

Save to File

python3 fetch_news.py --source all --limit 15 --deep \
  > reports/news_$(date +%Y%m%d_%H%M).json

# Or directly generate Markdown report
python3 fetch_news.py --source all --deep --output markdown \
  > reports/briefing_$(date +%Y%m%d_%H%M).md

File Naming Convention

reports/
├── hn_news_20260321_0800.md          (Hacker News focused)
├── tech_briefing_20260321_0800.md    (Multi-source tech)
├── github_trends_20260321_0800.md    (GitHub Trending deep dive)
└── global_scan_20260321_0800.md      (All sources, all topics)

---

Works Well With

• in-depth-research — Deep dive into individual stories from aggregator output
• web-scraper — Scrape specific websites not covered by aggregator
• translation — Translate Chinese news sources to English
• email-automation — Send daily briefings as email newsletters
• workflow-orchestrator — Schedule daily/weekly news runs

---

Tips & Best Practices

1. Expand keywords broadly — User says "AI" → expand to include LLM, GPT, agents, neural, ML, etc.
2. Respect time windows — If user says "past 5 hours," primarily show items from that window
3. GitHub exception — Always list ALL trending items (don't artificially limit), then analyze
4. Generate context — Don't just list items; explain why each matters
5. Use tags — Mark items with #Category tags for easy filtering
6. Preserve links — Always include original URLs as markdown links
7. Cite scores — Show HN points, GitHub stars, etc. to indicate popularity
8. Track time — Include "X hours ago" or "3 hours old" for freshness context

---

Links & Resources

• Hacker News: https://news.ycombinator.com/
• GitHub Trending: https://github.com/trending
• Product Hunt: https://www.producthunt.com/
• 36Kr: https://36kr.com/
• WallStreetCN: https://wallstreetcn.com/

---

Last updated: March 2026