| name | wechat-mp-rss-debug |
| version | 1.0.0 |
| description | Diagnose why wechat-mp-rss-extractor produces 0 new files - deleted articles (status=1000), slug collisions, and API refresh failures |
| category | wiki |
| related | ["wechat-mp-rss-extractor","wiki-pipeline"] |
we-mp-rss article debugging
When articles appear in RSS feeds but the extractor script produces 0 new files, systematically diagnose using this checklist.
Diagnosis Flow
1. Count new URLs vs inbox
inbox_dir = os.path.expanduser('~/wiki/raw/wechat-inbox/')
inbox_urls = set()
for f in os.listdir(inbox_dir):
if f.endswith('.md'):
with open(os.path.join(inbox_dir, f)) as fh:
for line in fh:
if line.startswith('source_url:'):
inbox_urls.add(line.strip().split('source_url:')[1].strip())
break
2. Check DB status for each new URL
docker exec we-mp-rss python3 -c "
import sqlite3
conn = sqlite3.connect('/app/data/db.db')
c = conn.cursor()
# NOTE (2026-06-09): schema column is `mp_id` (not `fakeid`), date is `updated_at_millis` (BIGINT ms, not unix seconds)
c.execute('SELECT id, mp_id, status, has_content, length(content), length(content_html), updated_at_millis FROM articles WHERE id LIKE \"%ARTICLE_ID%\" ORDER BY updated_at_millis DESC LIMIT 1')
row = c.fetchone()
print(row)
conn.close()
"
Schema note (2026-06-09 verified): The we-mp-rss DB schema uses mp_id (not the old fakeid field). Timestamps are updated_at_millis (BIGINT milliseconds) and created_at (DATETIME string). Use ORDER BY updated_at_millis DESC for "most recently updated". Quick schema check:
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db').cursor()
print([r[1] for r in c.execute('PRAGMA table_info(articles)').fetchall() if r[1] in ('id','mp_id','status','has_content','content','updated_at_millis')])
"
3. Interpret DB results
| status | content | has_content | Meaning |
|---|
| 1 | >0 | 1 | Valid article - should be written to inbox |
| 1000 | DELETED (7 chars) | 1 | Deleted from WeChat - API returns 404 |
| 1000 | None | 0 | Never crawled / removed - API returns 404 |
| 1 | None | 0 | Inconsistent state - rare |
Tables present (2026-06-09 verified): articles, feeds, users, message_tasks, config_management, access_keys, cascade_nodes, cascade_sync_logs, cascade_task_allocations, filter_rules, message_tasks_logs, tags — note that mp_accounts does NOT exist (use feeds table for account metadata).
Aggregate sanity check (use when extractor returns 0 writes for many URLs):
docker exec we-mp-rss python3 -c "
import sqlite3, time
c = sqlite3.connect('/app/data/db.db').cursor()
now_ms = int(time.time()*1000)
day_ago = now_ms - 24*3600*1000
print('total active:', c.execute('SELECT COUNT(*) FROM articles WHERE status=1 AND has_content=1 AND length(content)>100').fetchone()[0])
print('total deleted (status=1000):', c.execute('SELECT COUNT(*) FROM articles WHERE status=1000').fetchone()[0])
print('last 24h active updated:', c.execute('SELECT COUNT(*) FROM articles WHERE status=1 AND has_content=1 AND length(content)>100 AND updated_at_millis > ?', (day_ago,)).fetchone()[0])
"
If last 24h active updated is 0 even though cron has been firing, the upstream WeChat source is silent — not a script bug. See failure mode #7.
Key insight: status=1000 means the article is filtered out by the API (WHERE status != 1000). The API returns 404 even though the row exists in DB.
4. Check RSS content for valid articles
For articles with status=1, check if RSS content:encoded has content by querying the we-mp-rss RSS endpoint for the specific fakeid.
5. Check for slug collision
If article has content but script skips it, the slug file may already exist with a different URL:
ls ~/wiki/raw/wechat-inbox/ | grep -i "slug-pattern"
Common Failure Modes
-
All new articles deleted (status=1000): The WeChat articles were removed by authors. Script correctly skips them. No action needed.
-
Slug collision: Existing file has same title slug but different URL. The script's os.path.exists(slug + '.md') check prevents overwriting. Fix: rename existing file or update its content.
-
API refresh fails: Articles with empty RSS content trigger POST /refresh, but if status=1000, the refresh task is created but never completes (article not in active set). Script marks as pending_refresh.
-
Script fails to run: ModuleNotFoundError: No module named 'feedparser' - use /usr/bin/python3 (system Python has feedparser + html2text pre-installed).
-
Dry-run produces no output (2026-05-23): python3 scripts/wechat-mp-rss-extractor.py --dry-run silently exits with no output. This means the script hit an import error or silent exit path. Run with full Python to see the error:
/usr/bin/python3 -c "import sys; sys.path.insert(0, 'scripts'); import wechat_mp_rss_extractor" 2>&1
Also check if Docker container is running: docker ps | grep we-mp-rss
-
Docker DB has no tables (2026-05-23): sqlite3 db.db .tables returns nothing. The database file exists but was never initialized. This happens when the Docker volume was created but the container's init script never ran. Fix: docker exec we-mp-rss python3 -c "from app.database import init_db; init_db()" or restart the container to re-init.
-
All new URLs are status=1000 (upstream silent) — extractor returns 0 writes, this is correct (2026-06-09 verified): When the user runs python3 scripts/wechat-mp-rss-extractor.py --latest=10 and the summary shows rss_empty / api_empty = 8/8 with exists_in_articles: 202 and 0 new writes, the cause is usually that every candidate article in DB is status=1000 (deleted from WeChat). The extractor cannot write what doesn't exist. This is NOT a script bug — it's upstream WeChat state. Diagnose with the aggregate sanity check above: if last 24h active updated is 0, the source is silent. Don't waste cycles restarting the container or re-running the cron — verify state and report honestly:
Quick Diagnostic Script
source ~/.wiki-cron.env
/usr/bin/python3 scripts/wechat-mp-rss-extractor.py --latest=10
echo "Inbox count: $(ls raw/wechat-inbox/*.md | wc -l)"
echo "Status log: $(tail -1 cron-status.log)"
