we-mp-rss-troubleshooting

name	we-mp-rss-troubleshooting
title	we-mp-rss Docker Troubleshooting
description	Diagnose and fix common we-mp-rss Docker container issues: container exited state, Docker Desktop not running, proxy misconfiguration, QR code auth failure, cookie expiry, cascade worker registration failure, ContentTaskQueue stuck (Playwright anti-bot → api mode fix), dual-DB feed sync, extractor hanging on contentless articles (placeholder fallback fix), web.py API endpoint/response rewrite (appmsgpublish→appmsg, app_msg_list format), and cron job failures.
version	1.2.0
author	Hermes Agent
tags	["docker","we-mp-rss","wechat","cron","troubleshooting"]

we-mp-rss Docker Troubleshooting

Trigger: When wechat-mp-rss-extractor cron fails with "Connection refused", "Cannot reach we-mp-rss", when the Docker container is in an unexpected state, or when the extractor produces 0 new files despite the container responding HTTP 200.

Common Failure Patterns

1. Docker Desktop Not Running (most frequent)

Symptoms:

Cannot connect to the Docker daemon at unix:///Users/jinguo/.docker/run/docker.sock

or

Cannot reach we-mp-rss at http://localhost:8001: <urlopen error [Errno 61] Connection refused>

Cause: Docker Desktop was not running when the cron job fired. The we-mp-rss container cannot start without Docker.

Fix:

# Start Docker Desktop
open -a Docker

# Wait for it to initialize
sleep 15

# Verify Docker is available
docker info >/dev/null 2>&1 && echo "Docker is running"

2. Container in Exited State

Symptoms:

we-mp-rss Exited (255) 15 seconds ago ghcr.io/rachelos/we-mp-rss:latest

Cause: Container crashed or was stopped. Exit code 255 typically means the process terminated abnormally.

Diagnosis:

docker logs we-mp-rss 2>&1 | tail -30

Fix:

docker restart we-mp-rss
sleep 10
curl -s -o /dev/null -w "%{http_code}" http://localhost:8001/rss?limit=1
# Should return 200

3. Service Running but Slow to Respond

Symptoms: Cron runs but extractor gets connection refused even after docker restart.

Cause: Container is starting up but the web server hasn't bound port 8001 yet.

Fix:

# Wait and retry
sleep 10
curl -s -o /dev/null -w "%{http_code}" http://localhost:8001/rss?limit=1

3b. Port Mapping Mismatch — Connection reset by peer

Symptoms: curl -v http://localhost:8001/ shows Connected to localhost then Recv failure: Connection reset by peer. docker ps shows the port mapping but the service is unreachable. Inside the container, urllib to localhost:8001 works but localhost:3000 is refused.

Cause: The -p flag mapped host:8001 → container:3000 but the Node.js server listens on port 8001 (not 3000). The container's internal listening port is 8001 — confirmed by docker port we-mp-rss showing 3000/tcp -> 0.0.0.0:8001 (wrong direction) vs the correct 8001/tcp -> 0.0.0.0:8001.

Fix: Always use -p 8001:8001 (not -p 8001:3000). Verify with:

docker port we-mp-rss
# Expected: 8001/tcp -> 0.0.0.0:8001

# Quick health check (from host)
curl -s -o /dev/null -w "%{http_code}" http://localhost:8001/rss?limit=1
# Should return 200

# Inside container test (if curl to host fails)
docker exec we-mp-rss python3 -c "import urllib.request; print(urllib.request.urlopen('http://localhost:8001/rss?limit=1', timeout=5).read()[:50])"

Reference: references/2026-07-02-proxy-empty-port-browser-fix.md — full recovery transcript including proxy-empty-variant, port mapping mismatch, and Playwright browser_type fix.

Cron Job Pre-Run Checklist

Before running wechat-mp-rss-extractor.py:

# 1. Ensure Docker is running
docker info >/dev/null 2>&1 || { open -a Docker; sleep 15; }

# 2. Ensure container is running
docker ps --filter "name=we-mp-rss" --filter "status=running" | grep we-mp-rss
# If not running: docker restart we-mp-rss && sleep 10

# 3. Verify service responds
curl -s -o /dev/null -w "%{http_code}" http://localhost:8001/rss?limit=1
# Should return 200

# 4. Run extractor
source ~/.wiki-cron.env
export PATH="$PATH:/Users/jinguo/Library/Python/3.14/bin"
cd ~/wiki && python3 scripts/wechat-mp-rss-extractor.py --latest=10

4. DB Mismatch After Container Recreate

Symptoms: After docker compose down && up -d, the web UI shows "共 0 条" (no subscriptions) even though the volume mounted correctly.

Root Cause: The docker-compose-sqlite.yaml configures DB=sqlite:///data/we_mp_rss.db but the real data may be in db.db (created by a previous version or config). The new container creates a fresh we_mp_rss.db while the subscriptions remain in db.db.

Fix:

# Check which DB has data
sqlite3 /Users/jinguo/data/we_mp_rss.db "SELECT COUNT(*) FROM feeds;"
sqlite3 /Users/jinguo/data/db.db "SELECT COUNT(*) FROM feeds;"

# Restore data (backup first)
cp /Users/jinguo/data/we_mp_rss.db /Users/jinguo/data/we_mp_rss.db.bak
cp /Users/jinguo/data/db.db /Users/jinguo/data/we_mp_rss.db
docker restart we-mp-rss

5. Broken PASSWORD Line in docker-compose YAML

Symptoms: Login returns "Could not validate credentials" for any password.

Root Cause: The compose file has a merged line like PASSWORD=*** - GATHER.CONTENT=True — two env vars concatenated. The running container stores the literal broken value as the password hash.

Fix: Split into separate YAML lines:

- PASSWORD=*** GATHER.CONTENT=True

After fixing, either delete the DB to force recreation with the correct password, or restore from a known-good backup (db.db).

6. env_file Pattern (Recommended)

Instead of hardcoding secrets in docker-compose:

services:
  we-mp-rss:
    env_file:
      - ../.env
    environment:
      - DB=sqlite:///data/we_mp_rss.db

Add .env to .gitignore. Restart to apply: docker compose -f compose/docker-compose-sqlite.yaml up -d

7. List Subscriptions via CLI

curl -s http://localhost:8001/rss | grep '<title>' | grep -v 'WeRSS订阅'

Verification After Scan

Important: The heartbeat file is typically touched by the cron preamble (cron-heartbeat.py touch), so find -newer heartbeat will NOT find new files (they're older than the heartbeat).

Instead, verify by:

# Method 1: Count total files
ls ~/wiki/raw/wechat-inbox/*.md | wc -l

# Method 2: Check cron-status.log
tail ~/wiki/cron-status.log

# Method 3: Check most recent files
ls -lt ~/wiki/raw/wechat-inbox/*.md | head -5

Environment Setup

Required in cron shell (bash, non-interactive — .zshrc is NOT loaded):

source ~/.wiki-cron.env
export PATH="$PATH:/Users/jinguo/Library/Python/3.14/bin"

The .wiki-cron.env file contains WERSS_AK and WERSS_SK for API authentication.

4. "Alive but Stale" — Service HTTP 200 but Upstream Sync Broken (Proxy Misconfiguration)

Symptoms:

• curl http://localhost:8001/rss?limit=1 returns HTTP 200
• RSS feeds return valid XML with article entries
• BUT all articles' pubDate are weeks/months old (e.g., all from May when today is July)
• Docker logs show 成功0条 for every account in every sync cycle
• wechat-inbox stays at 0 new files indefinitely

⚠️ Diagnostic trap: The container's internal articles table create_time can be stale even when the RSS has current articles. These are SEPARATE data paths — the RSS endpoint generates from one source, the articles table from another. Always check the per-feed RSS endpoint directly before concluding the pipeline is stuck.

# CORRECT first step — check RSS per-feed (fast, accurate)
curl -s "http://localhost:8001/rss/MP_WXS_3236757533?limit=3" | grep -o '<pubDate>[^<]*</pubDate>'
# If these show today's dates -> pipeline is healthy

Root Cause (two variants):

1. 127.0.0.1 loopback — HTTP_PROXY=http://127.0.0.1:10808 (points to container's own loopback, not host). Error: ProxyError('Unable to connect to proxy').
2. Empty proxy env — HTTP_PROXY= / http_proxy= (variable present but empty). No error logged — requests just timeout silently behind the Great Firewall. This was the 2026-07-02 failure: container rebuilt without setting proxy vars, leaving them as empty strings.

Diagnosis Sequence:

# Step 1: Check RSS data freshness (pick 2-3 representative accounts)
for fakeid in MP_WXS_3006407565 MP_WXS_3073282833 MP_WXS_3236757533; do
  name=$(curl -s --connect-timeout 5 "http://localhost:8001/rss/$fakeid?limit=1" | grep -oE '<title>[^<]+</title>' | tail -1 | sed 's/<[^>]*>//g')
  date=$(curl -s --connect-timeout 5 "http://localhost:8001/rss/$fakeid?limit=1" | grep -oE '<pubDate>[^<]+</pubDate>' | head -1 | sed 's/<[^>]*>//g')
  echo "$name: $date"
done
# If all dates are >7 days old → upstream sync has stopped

# Step 2: Check Docker logs for proxy errors
docker logs we-mp-rss --tail 200 2>/dev/null | grep -iE 'error|proxy|refused|connect' | head -10

# Step 3: Check container env vars for proxy misconfiguration
docker inspect we-mp-rss --format '{{range .Config.Env}}{{println .}}{{end}}' 2>/dev/null | grep -i proxy
# WRONG: HTTP_PROXY=http://127.0.0.1:10808  (container's own loopback!)
# RIGHT: HTTP_PROXY=http://host.docker.internal:10808

# Step 4: Verify host.docker.internal resolves inside container
docker exec we-mp-rss python3 -c "import socket; print(socket.gethostbyname('host.docker.internal'))"
# Should return 192.168.65.254 (Docker Desktop's host gateway)

# Step 5: Check DB for newest article date
docker cp we-mp-rss:/app/data/db.db /tmp/we-mp-rss.db
sqlite3 /tmp/we-mp-rss.db "SELECT title, publish_time FROM articles ORDER BY publish_time DESC LIMIT 5;"
python3 -c "
import sqlite3, datetime
conn = sqlite3.connect('/tmp/we-mp-rss.db')
for row in conn.execute('SELECT title, publish_time FROM articles ORDER BY publish_time DESC LIMIT 5'):
    dt = datetime.datetime.fromtimestamp(row[1])
    print(f'{dt} | {row[0][:40]}')
"

Fix: Stop old container and recreate with corrected proxy env vars:

# Stop the broken container
docker stop we-mp-rss
docker rm we-mp-rss

# Recreate with correct proxy (host.docker.internal instead of 127.0.0.1)
docker run -d \
  --name we-mp-rss \
  --restart unless-stopped \
  -p 8001:8001 \
  -v /Users/jinguo/data:/app/data \
  -e USERNAME=admin \
  -e PASSWORD=*** \
  -e GATHER.CONTENT=True \
  -e GATHER.CONTENT_MODE=web \
  -e GATHER.MODEL=web \
  -e GATHER.CONTENT_AUTO_CHECK=True \
  -e GATHER.CONTENT_AUTO_INTERVAL=59 \
  -e BROWSER_TYPE=chromium \
  -e DB=sqlite:///data/we_mp_rss.db \
  -e TZ=Asia/Shanghai \
  -e WERSS_AK=<AK> \
  -e WERSS_SK=<SK> \
  -e PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple \
  -e HTTP_PROXY=http://host.docker.internal:10808 \
  -e HTTPS_PROXY=http://host.docker.internal:10808 \
  -e http_proxy=http://host.docker.internal:10808 \
  -e https_proxy=http://host.docker.internal:10808 \
  -e NO_PROXY=localhost,127.0.0.1,::1,*.local \
  -e no_proxy=localhost,127.0.0.1,::1,*.local \
  ghcr.io/rachelos/we-mp-rss:latest

# Wait for startup
sleep 15

# Verify sync is working
curl -s --connect-timeout 5 "http://localhost:8001/rss?limit=30&offset=0" | grep -oE '<pubDate>[^<]+</pubDate>' | head -3
# Should show recent dates (today or yesterday)

Data safety: The -v /Users/jinguo/data:/app/data bind mount preserves the SQLite DB across container recreation. All feed subscriptions and article history are retained.

Verification after fix:

# Check that sync_time updates (run after the first sync cycle completes)
docker cp we-mp-rss:/app/data/db.db /tmp/we-mp-rss.db
sqlite3 /tmp/we-mp-rss.db "SELECT mp_name, datetime(sync_time, 'unixepoch') FROM feeds ORDER BY sync_time DESC LIMIT 5;"
# Should show recent timestamps

5. WeChat Session Token Expired (after proxy fix)

Symptoms:

• Proxy is fixed (no more Connection refused errors)
• But Docker logs show Invalid Session, stop at 0 for every account
• Web UI "授权管理" page shows Token 到期时间 in the past
• 成功0条 continues despite proxy working
• cascade_task_allocations table shows 273+ entries with status=timeout (error: "任务超时（>30分钟）")

Critical Pitfall: Redis Caches Old Token

Even after injecting a new cookie into /app/data/wx.lic, the system may continue using the expired token from Redis cache. This causes the "Invalid Session" error to persist indefinitely until Redis is cleared.

Redis flush is AUTOMATED (2026-07-06) — redis-cli NOT in container PATH (confirmed 2026-07-13): wechat-cookie-renew.py runs a Redis flush after health check, but the container image does NOT ship redis-cli. The command always fails with OCI runtime exec failed: exec: \"redis-cli\": executable file not found in $PATH.

⚠️ Python interpreter pitfall (confirmed 2026-08-07): The container's system python3 does NOT have the redis module (ModuleNotFoundError). The module exists only in /app/env_x86_64/bin/python3 (the venv). ALL docker exec we-mp-rss python3 -c "import redis..." invocations below must use /app/env_x86_64/bin/python3 instead, or the flush silently fails again.

Fix wechat-cookie-renew.py: Replace the subprocess.run(["docker", "exec", "we-mp-rss", "redis-cli", ...]) block with:

try:
    r = subprocess.run(
        ["docker", "exec", "we-mp-rss", "/app/env_x86_64/bin/python3", "-c",
         "import redis; r=redis.Redis(host='localhost',port=6379,db=0); r.flushall(); print('OK')"],
        capture_output=True, text=True, timeout=10
    )
    if r.returncode == 0 and 'OK' in r.stdout:
        print(f"Redis flushed via Python redis client: {r.stdout.strip()}")
    else:
        print(f"WARN: Redis flush failed: {r.stderr.strip() or '(no output)'}")
except Exception as e:
    print(f"WARN: Redis flush error: {e}")

To manually verify Redis was flushed:

docker exec we-mp-rss /app/env_x86_64/bin/python3 -c "
import redis
r = redis.Redis(host='localhost', port=6379, db=0)
r.flushall()
print('Redis flushed via python redis client')
" 2>&1 | grep -v '^\['

Legacy manual fix (still valid if bypassing cookie-renew script):

# Step 1: Stop container
docker stop we-mp-rss

# Step 2: Clear Redis-like cache (we_mp_rss.db)
# Option A: Delete the cached token data
docker exec we-mp-rss python3 -c "
import sqlite3
conn = sqlite3.connect('/app/data/we_mp_rss.db')
c = conn.cursor()
c.execute(\"DELETE FROM kv_store WHERE key='token_data' OR key LIKE '%cookie%'\")
conn.commit()
print('Cleared', c.rowcount, 'cached entries')
"

# Option B: If using Redis directly (rare), flush it:
# docker exec we-mp-rss redis-cli FLUSHDB

# Step 3: Verify wx.lic has fresh cookie
docker exec we-mp-rss cat /app/data/wx.lic | head -5

# Step 4: Restart container
docker restart we-mp-rss
sleep 15

# Step 5: Verify new articles appear
docker logs we-mp-rss --tail 50 | grep -E "Added article|成功"

The 42-day outage pattern (2026-05-20 to 2026-07-01):

• Cookie expired on 2026-05-20
• All article collection returned "Invalid Session"
• Redis cached the expired token
• Even after QR re-scan and new wx.lic injection, collection failed
• Root cause: Redis cache was NOT cleared (step 5 in fix table)
• After clearing Redis, collection resumed successfully

The 9-day silent outage (2026-07-29 → 2026-08-07) — two false-healthy masks:

Symptom chain: no new articles since 07-29, but cron showed all OK. Two script bugs masked it:

1. wechat-cookie-renew.py only checked cookie PRESENCE (slave_sid exists), not server-side validity. When the WeChat session dies server-side, Chrome still holds a stale slave_sid cookie → renewal "succeeded" daily, writing dead cookies into wx.lic. The real signal is the home page URL token (token=\d+ in mp.weixin.qq.com/cgi-bin/home?...) — logged-out sessions have no token.

   • Fix (2026-08-07): cookie-renew now exits 1 with QR-scan instructions if slave_sid is present but no token is found after navigation + refresh.

2. wechat-article-discover.sh reported Done: 30/30 ok, 0 err while every feed fetched 0 articles (成功0条). The inline python only counted exceptions, never article yields → exit 0 → cron "OK".

   • Fix (2026-08-07): the script now counts articles table rows before/after the run and exits 1 with ALERT: ... new_articles=0 if any feed errored or zero new articles were stored.

Recovery sequence that worked (2026-08-07): 10:28 cookie renew wrote valid cookies (Chrome session was actually alive) → flush Redis (/app/env_x86_64/bin/python3 -c "import redis; redis.Redis(host='localhost',port=6379,db=0).flushall()") → docker restart we-mp-rss → "Invalid Session" gone, replaced by transient frequencey control, stop at 0 (WeChat rate limit after repeated failed attempts; lifts within ~30-60 min). Confirm recovery by single-feed do_job(mp=feed) test showing no Invalid Session.

⚠️ Never run wechat-cookie-renew while wechat-article-discover is in flight — the renew script restarts the container, SIGKILLing the discover docker exec (exit 137). See section 24.

Root Cause: The WeChat login session token expired during the period when proxy was broken (45+ days of no sync). The container's WxGather module cannot refresh the token automatically — it requires manual re-authorization via the web UI.

Diagnosis:

docker logs we-mp-rss --tail 200 2>/dev/null | grep -i 'invalid session'

# Check cascade task timeout count as proxy for "sync broken"
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db').cursor()
print('timeout:', c.execute(\"SELECT COUNT(*) FROM cascade_task_allocations WHERE status='timeout'\").fetchone()[0])
print('pending:', c.execute(\"SELECT COUNT(*) FROM cascade_task_allocations WHERE status='pending'\").fetchone()[0])
"

# Check if any articles were synced after a specific date
docker exec we-mp-rss python3 -c "
import sqlite3, time
c = sqlite3.connect('/app/data/db.db').cursor()
may15 = int(time.mktime(time.strptime('2026-05-15', '%Y-%m-%d')))
rows = c.execute('SELECT id, title, datetime(publish_time, \"unixepoch\"), status, has_content FROM articles WHERE publish_time > ? ORDER BY publish_time DESC LIMIT 10', (may15,)).fetchall()
print(f'Articles after May 15: {len(rows)}')
for r in rows:
    print(f'  {r[3]} {r[2]} {r[1][:40]}')
"

Fix (requires manual intervention):

1. Open http://localhost:8001/ in a browser
2. Log in (see "Web UI Password Reset" below if password unknown)
3. Navigate to 授权管理 (left sidebar)
4. Click "扫码授权" button
5. Scan the QR code with phone's WeChat
6. After successful scan, verify Token status shows "有效" with a future expiry date
7. Wait for the next scheduled sync (*/30 * * * *)

Headless Container Workaround (QR code in Docker with no display):

The "扫码授权" button triggers the container's internal Playwright WebKit browser (headless mode) to open https://mp.weixin.qq.com/ and save the QR code screenshot to /app/static/wx_qrcode.png. The QR code file exists only during the 5-minute login window and is cleaned up on timeout.

To extract the QR code from a headless container:

# Step 1: Click "扫码授权" in the web UI
# Step 2: IMMEDIATELY copy the QR code from the container (5-min window)
docker cp we-mp-rss:/app/static/wx_qrcode.png /tmp/wx_qrcode.png

# Step 3: Open it for scanning
open /tmp/wx_qrcode.png

# Step 4: Scan with phone's WeChat

# Step 5: Verify login succeeded
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db').cursor()
# Check wx.lic for new cookie expiry
print(open('/app/data/wx.lic').read()[:200])
"

If QR code file doesn't appear:

• The container's Playwright may have failed to screenshot (check docker logs we-mp-rss for "二维码" or "qrcode" messages)
• The lock file at /app/data/lock.lock being present means the login process is running
• The browser process can be verified: docker exec we-mp-rss ps aux | grep -i "webkit\|playwright"
• If the QR code is never generated, the container may need --cap-add=SYS_ADMIN or --shm-size=2g for headless browser screenshot support

Note: The QR code is generated by the container's internal Playwright (server-side), not by the Vue.js frontend. Clicking "扫码授权" triggers a server-side API that starts a headless browser inside the container.

23b. web.py API Endpoint Outdated — New Subscriptions Return 0 Articles

Symptoms: Proxy/cookies are correct, cascade tasks show timeout, but specific new subscription feeds return 0 articles via RSS. logs show successful sync but no article entries in the DB.

Root Cause: Three code-level bugs in /app/core/wx/model/web.py (we-mp-rss v1.5.2):

1. Wrong API endpoint — uses deprecated cgi-bin/appmsgpublish (returns 0 articles) instead of cgi-bin/appmsg
2. Response format — parser only reads data.publish_page but the active API returns data.app_msg_list
3. Early exit — if 'publish_page' not in msg: break kills collection before app_msg_list is read

Fix: Patch web.py inside the container and restart:

# Apply all three fixes via sed
docker exec we-mp-rss sed -i 's|cgi-bin/appmsgpublish?type=10|cgi-bin/appmsg?type=9|' /app/core/wx/model/web.py
docker exec we-mp-rss sed -i '/def parse_response/a\    if data.get('\"'\"'app_msg_list'\"'\"'):\n        for item in data['\"'\"'app_msg_list'\"'\"']:\n            articles.append({...})' /app/core/wx/model/web.py
docker exec we-mp-rss sed -i 's|if '"'\"'\"'publish_page'\"'\"' not in msg:|if '"'\"'\"'publish_page'\"'\"' not in msg and '\"'\"'app_msg_list'\"'\"' not in msg:|' /app/core/wx/model/web.py
docker restart we-mp-rss

See references/2026-07-04-webpy-appmsg-api-fix-chain.md for exact code and verification script.

Persistence: These are container-level patches. They must be reapplied after every Docker image update (pull→recreate).

⚠️ 2026-08-07: patch lost again after container rebuild + account-level freq control (ret=200013)

Symptom chain (9-day outage 07-29 → 08-07): all feeds 成功0条 silently → today escalated to explicit Invalid Session (redis cached stale auth) → after redis flush + fresh login, still 0 articles. Direct API probe with the CORRECT appmsg?action=list_ex&type=9 endpoint, with BOTH old and NEW tokens, via BOTH direct and proxied connections: all return {"base_resp":{"err_msg":"freq control","ret":200013}}. Conclusion: the block is ACCOUNT-LEVEL (the WeChat MP login account), NOT token/IP/fakeid-specific.

Key facts:

• The deprecated appmsgpublish endpoint was back in web.py (container rebuilt, lost the 2026-07-04 patch). Repeated hits on the deprecated endpoint are abuse signals that keep the account freq-controlled.
• Frequency control cooldown is ≥ 60-90 min from the LAST request; the internal job sweeping all feeds every 59 min (enable_job=True) perpetually resets it → self-sustaining throttle (see section 17).
• Recovery requires: (1) web.py re-patched to appmsg, (2) ALL collection requests stopped for 24-48h (internal scheduler off + wechat-article-discover cron paused), (3) then a single verification request.

Follow-up (2026-08-08): 24h of complete silence did NOT lift the block — single-feed do_job AND direct API probe still return 200013. The account dashboard (mp.weixin.qq.com/cgi-bin/home) loads normally with no risk-control banner — the ban is on the appmsg DATA API only, account-level, multi-day (7-day tier likely). Lessons: (a) do NOT probe daily — each probe is a fresh violation that extends the ban; probe at 48-72h intervals; (b) re-login (new token) does NOT help; (c) while banned, article DISCOVERY is impossible — only manual URL ingestion (CDP fetch of known mp.weixin.qq.com/s/xxx links) keeps the wiki pipeline alive; consider RSSHub-style mirrors as an alternative discovery path.

✅ FINAL SOLUTION (2026-08-13): Third-party mirror RSS channel — Wechat2RSS (wechat2rss.xlab.app)

The account-level 200013 ban (15 days at this point) has NO local fix. The working recovery path is to bypass the account entirely via a third-party mirror service that maintains its own WeChat MP collection channel:

How it works:

• wechat2rss.xlab.app hosts 395+ WeChat MP accounts as RSS feeds (avg 6h latency, full-text output, free). It collects via ITS OWN account — your banned account is never touched, zero rate-limit risk.
• Match your feeds against their list: curl -sL -x http://127.0.0.1:10808 https://wechat2rss.xlab.app/list/all/ → regex href="(https://wechat2rss\.xlab\.app/feed/[^"]+\.xml)"[^>]*>([^<]{2,30})</a>. NOTE: href is the FULL URL with .xml suffix — a regex matching only /feed/ relative paths silently returns 0 matches.
• Result: 11/30 of this wiki's feeds were covered (机器之心/量子位/新智元/腾讯技术工程/阿里技术/字节跳动技术团队/PaperWeekly/美团技术团队/阿里云开发者/夕小瑶科技说/小米技术). Remaining 19 can be requested via GitHub issue, or wait for unban.

Implementation (3 touch points, all verified):

1. blogwatcher DB (~/.blogwatcher-cli/blogwatcher-cli.db, table blogs: id,name,url,feed_url,scrape_selector,last_scanned): insert one row per mirror feed, name prefixed WeChat-. ⚠️ Do NOT use blogwatcher-cli add — it dedupes by URL and all feeds share domain https://wechat2rss.xlab.app, so only the first insert succeeds. Direct SQL INSERT with the unique .xml feed_url instead.
2. rss-inbox-recovery.py GOOD_FEEDS dict (in ~/.hermes/skills/wiki/rss-to-wiki-pipeline/scripts/): hardcoded feed list — the WeChat-* entries MUST be added here or recovery never fetches them (it does NOT read the blogwatcher DB). Python urllib reads HTTPS_PROXY=http://127.0.0.1:10808 automatically — no code change needed for proxy.
3. rss-feed-scan cron prompt (job f2231ed6eda8): append note that WeChat-* feeds need --unsafe-client (see pitfall below) + proxy env.

Pitfalls (all hit and fixed 2026-08-13):

• blogwatcher Go SSRF guard rejects loopback proxies: blogwatcher-cli scan "WeChat-机器之心" with HTTPS_PROXY=127.0.0.1:10808 fails with proxyconnect tcp: dial tcp 127.0.0.1:10808 ... not authorized by the client: "127.0.0.1" address is loopback. Fix: HTTPS_PROXY=http://127.0.0.1:10808 blogwatcher-cli scan --unsafe-client. The Python recovery path is unaffected (urllib has no such guard).
• Zombie cron round: a wechat-inbox-pipeline round showed running in executions.db with no agent subprocess (gateway idle, inbox files untouched for 20+ min) — agent died mid-run without status update. Fix: verify with ps aux | grep -E 'cron|48ab4689ca54' (empty = dead), then cronjob action=run to re-trigger. Do NOT wait for the next scheduled round.
• WeChat vendor content gets quick-classify skipped by design: most mirror articles are vendor official-account posts (阿里/腾讯/字节/小米/美团) — the inbox-screener deterministically skips them (not a bug). Expect a small ingest fraction (39/220 ingested in the first round).
• First scan shows New:0 for a feed you manually scanned earlier: manual test scans mark articles read — Found: 20, New: 0 is normal, subsequent runs pick up only genuinely new posts.
• Feed-name matching needs fuzzy check: exact-name match missed nothing here, but verify with difflib (ratio > 0.6) + substring before concluding a feed is uncovered.

Verification sequence (end-to-end):

# 1. scan (proxy + unsafe-client)
HTTPS_PROXY=http://127.0.0.1:10808 blogwatcher-cli scan --unsafe-client 2>&1 | grep -E 'WeChat|New'
# 2. recovery writes inbox files
cd ~/wiki && python3 ~/.hermes/skills/wiki/rss-to-wiki-pipeline/scripts/rss-inbox-recovery.py 2>&1 | tail -20
# expect "WeChat-XX: 20 written"
# 3. inbox-pipeline (hourly cron) scores + ingests
# 4. verify: ls raw/rss-inbox | wc -l drops; grep -c '\[2026-08-13\]' log.md increases

Unban recovery path (when the account comes back): UPDATE message_tasks SET status=1 WHERE id='5da2b787-...' + container restart + resume wechat-article-discover cron + manual trigger. Until then keep the account at ZERO collection requests (internal scheduler off via status=0, discover cron paused) so the ban window isn't extended.

---

24. Cron Timing Conflict — Manual Triggers of cookie-renew + article-discover

Symptoms: Manually triggering wechat-cookie-renew and wechat-article-discover at roughly the same time causes wechat-article-discover to exit with code 137 (SIGKILL). The cookie-renew script restarts the we-mp-rss container (Container restarted), which kills the docker exec session that article-discover is running inside.

Root Cause: Under normal cron scheduling the two jobs are intentionally 10 minutes apart: wechat-cookie-renew at 09:00, wechat-article-discover at 09:10. The container restart from cookie-renew invalidates any in-flight docker exec from article-discover. When triggered manually without this gap, article-discover always fails.

Fix: Never trigger both manually at the same time. Run cookie-renew first, wait until it completes (check output for "Done! Cookie auto-renewal successful."), then trigger article-discover. Under normal cron scheduling this conflict does not occur.

Symptoms: ContentTaskQueue keeps retrying content fetch but has_content stays 0. RSS content:encoded remains empty. Container logs show Playwright launch failures or captcha pages. Old articles have content (fetched before container rebuild) but new ones don't.

Root Cause: GATHER.CONTENT_MODE=web (default) makes ContentTaskQueue use Playwright to open article pages. WeChat anti-bot blocks headless browsers from data center IPs. The api mode uses HTTP GET + proxy (requests library) instead, bypassing the browser entirely.

Fix — Set GATHER.CONTENT_MODE=api on container creation:

-e GATHER.CONTENT_MODE=api

Also accelerate the queue (default is 59 minutes):

-e GATHER.CONTENT_AUTO_INTERVAL=5

Verify:

# Check a specific article's HTML content
docker exec we-mp-rss python3 -c "
import sys; sys.path.insert(0,'/app')
from core.wx.model.api import MpsApi
fetcher = MpsApi()
content = fetcher.content_extract('https://mp.weixin.qq.com/s/XXXXX')
print(f'Content: {len(content) if content else 0} chars')
"

# Check RSS content:encoded
curl -s 'http://localhost:8001/rss/{fakeid}?limit=1' | python3 -c "
import sys, xml.etree.ElementTree as ET
root = ET.fromstring(sys.stdin.read())
item = root.find('.//item')
if item:
    enc = item.find('{http://purl.org/rss/1.0/modules/content/}encoded')
    print(f'content:encoded: {len(enc.text) if enc is not None and enc.text else 0} chars')
"

# Check DB content status
sqlite3 ~/data/we_mp_rss.db 'SELECT COUNT(*) || \" of \" || COUNT(*) || \" articles have content\" FROM articles WHERE has_content=1'

Note: The api mode's content_extract calls requests.get(url) through the proxy configured in HTTP_PROXY. It does NOT use Playwright at all, so browser_type changes have no effect. The extracted content is stored as content_html and served in RSS <content:encoded>.

Extractor hanging fix: When articles have no content, the extractor's 10-attempt poll loop blocks processing of subsequent feeds. Reduce poll count or use --no-refresh:

# Option 1: Run with --no-refresh (contentless articles get placeholder files)
/usr/bin/python3 scripts/wechat-mp-rss-extractor.py --latest=5 --no-refresh

# Option 2: Patch poll count in the script (range(10) → range(2))

The script also now creates placeholder inbox files for contentless articles (metadata + original link) instead of skipping them entirely, preventing pipeline blocking.

6. Web UI Password Reset

Symptoms: Cannot log into http://localhost:8001/ — password unknown or PASSWORD=*** env var is literally the string ***.

Root Cause: The docker inspect output shows PASSWORD=*** which looks like redacted output but is actually the literal password value (three asterisks). If that doesn't work, the password hash in the DB needs to be reset.

Fix (reset via SQLite):

sqlite3 /Users/jinguo/data/db.db ".schema users"
# Note: column is password_hash, NOT password

pip3 install --break-system-packages --user bcrypt
python3 -c "
import bcrypt
pw = b'admin'
hashed = bcrypt.hashpw(pw, bcrypt.gensalt(rounds=12))
print(hashed.decode())
"

HASH='<paste output>'
sqlite3 /Users/jinguo/data/db.db "UPDATE users SET password_hash='$HASH' WHERE username='admin';"
docker restart we-mp-rss
sleep 10

7. Full Recovery Sequence (Proxy Fix → Session Re-auth)

When the pipeline has been broken for an extended period (weeks+), two issues compound:

1. Proxy misconfiguration → all sync requests fail silently for weeks
2. WeChat session token expires → even after proxy fix, sync still fails

Recovery steps in order:

1. Fix proxy (see section 4 above)
2. Reset Web UI password if needed (see section 6 above)
3. Re-authorize WeChat via QR code scan (see section 5 above)
4. Wait for the first successful sync cycle
5. Run extractor to pull new articles into wechat-inbox
6. Run wiki-pipeline to ingest

4. Alive but Stale — Container Responds but No New Articles (WeChat Cookie Expired)

Symptoms:

• docker ps shows container running, curl localhost:8001 returns HTTP 200
• Extractor runs but produces 0 new files — rss_empty / api_empty count equals total candidate articles
• All cascade_task_allocations entries show status=timeout with error "任务超时（>30分钟）"
• DB query shows last 24h active updated: 0 and last 7 days: 0
• Latest article publish dates are weeks/months old across ALL accounts

Root Cause: The WeChat public platform login cookie has expired. we-mp-rss uses Playwright (headless WebKit) to log into mp.weixin.qq.com and obtain a session cookie. This cookie expires ~14 days after the last login. Once expired, all article sync tasks hang until timeout because the WeChat API rejects the stale session.

Diagnosis:

# 1. Check cascade task allocation status
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db').cursor()
rows = c.execute('SELECT status, COUNT(*) FROM cascade_task_allocations GROUP BY status').fetchall()
for r in rows: print(f'  status={r[0]}: {r[1]}')
"

# 2. Check latest article dates per account
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db').cursor()
rows = c.execute('''
    SELECT f.mp_name, a.title, datetime(a.publish_time, \"unixepoch\") as pub
    FROM articles a JOIN feeds f ON a.mp_id = f.id
    WHERE a.status = 1 AND a.has_content = 1 AND length(a.content) > 100
    GROUP BY a.mp_id HAVING MAX(a.publish_time)
    ORDER BY a.publish_time DESC LIMIT 25
''').fetchall()
for r in rows: print(f'{r[0]:30s}  {r[2]}  {r[1][:40]}')
"

# 3. Check WeChat cookie expiry (check wx.lic file)
docker exec we-mp-rss cat /app/data/wx.lic | grep -o 'expiry_time: [^"]*' 

# 4. Check recent articles count (should be 0 when stale)
docker exec we-mp-rss python3 -c "
import sqlite3, time
c = sqlite3.connect('/app/data/db.db').cursor()
week_ago = int(time.time()) - 7*86400
cnt = c.execute('SELECT COUNT(*) FROM articles WHERE status=1 AND has_content=1 AND length(content)>100 AND publish_time > ?', (week_ago,)).fetchone()[0]
print(f'最近7天有content新文章: {cnt} 篇')
"

Fix: Re-authenticate WeChat by scanning the QR code:

1. Log into we-mp-rss web UI at http://localhost:8001/ (admin/admin)
2. Navigate to 授权管理 (Authorization Management)
3. Click 扫码授权 (Scan to Authorize)
4. Scan the QR code with WeChat on your phone
5. Verify: docker exec we-mp-rss cat /app/data/wx.lic | grep expiry_time shows a future date
6. Trigger a manual sync or wait for the next cron cycle

Alternative (via API):

# Get auth token
TOKEN=$(curl -s -X POST "http://localhost:8001/api/v1/wx/auth/token" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "username=admin&password=admin" | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")

# Trigger QR code generation
curl -s "http://localhost:8001/api/v1/wx/auth/qr/code" \
  -H "Authorization: Bearer $TOKEN"

# The QR code is generated inside the container at /app/static/wx_qrcode.png
# Copy it out to view:
docker cp we-mp-rss:/app/static/wx_qrcode.png /tmp/wx_qrcode.png
open /tmp/wx_qrcode.png

Alternative (via browser — recommended when Browserbase is available): Navigate to https://mp.weixin.qq.com/ in the browser tool, click "使用账号登录" then switch to QR code login tab by clicking .login__type__container__select-type.login__type__container__select-type__scan. The QR code image URL will be https://mp.weixin.qq.com/cgi-bin/scanloginqrcode?action=getqrcode&random=.... Screenshot and present to the user. Note: the container's headless WebKit often fails to save the QR code screenshot to /app/static/wx_qrcode.png (no error logged, just silently missing), so the browser tool approach is more reliable for getting the QR code to the user.

Prevention: Use the monitoring script at scripts/check-cookie-health.py (in this skill directory) to proactively detect cookie expiry before it causes a silent outage:

# Run manually
python3 ~/.hermes/skills/devops/we-mp-rss-troubleshooting/scripts/check-cookie-health.py

# Or set up a weekly cron job
# Example: hermes cron create --schedule "0 9 * * 1" \
#   --prompt "Run we-mp-rss cookie health check" \
#   --script "scripts/check-cookie-health.py" \
#   --no-agent

The script checks:

1. Container is running
2. Cookie expiry timestamp from wx.lic
3. Recent article sync activity (last 7 days)

Exits 0 (healthy) or 1 (stale/expired), suitable for cron alerting.

18. Cascade Worker Registration Failure — Redis Port Conflict

Symptoms:

• Container starts normally, HTTP 200
• Logs show: [WARN] 端口 6379 已被占用，内置 Redis 服务启动失败
• cascade_task_allocations table has no completed tasks — everything is pending forever
• cascade_nodes table is empty (0 entries) in both db.db and we_mp_rss.db
• New subscriptions (added via Web UI or SQLite) appear in feeds list but get 0 articles — RSS /rss/{fakeid} returns empty
• Old subscriptions that had articles before a container rebuild still work (articles persist in DB)
• Queue Redis 连接成功 log line still appears (content queue uses a different Redis connection that succeeds)

Root Cause: The container has GATHER.CONTENT_AUTO_CHECK=True and GATHER.CONTENT_AUTO_INTERVAL=59, which means the built-in cascade task scheduler creates "全量公众号定时采集" tasks in cascade_task_allocations. But the actual worker nodes that claim and execute these tasks register themselves via the built-in Redis server on port 6379. When port 6379 is already occupied on the host (e.g., another Redis instance), the container logs 内置 Redis 服务启动失败, no cascade nodes register, and all tasks stay pending forever with no worker to claim them.

This is distinct from the "All tasks timeout" pattern (section 5) — timeout means workers existed but WeChat API calls failed (cookie expired). pending forever means no worker ever claimed the task.

Diagnosis:

# 1. Check cascade_nodes — should have >=1 entry when healthy
docker exec we-mp-rss python3 -c "
import sqlite3
for db_name in ['db.db', 'we_mp_rss.db']:
    conn = sqlite3.connect('/app/data/' + db_name)
    cnt = conn.execute('SELECT COUNT(*) FROM cascade_nodes').fetchone()[0]
    print(db_name + ' cascade_nodes: ' + str(cnt))
    conn.close()
"

# 2. Check task status — if all are 'pending' with no 'timeout'/'completed', workers aren't claiming them
docker exec we-mp-rss python3 -c "
import sqlite3
conn = sqlite3.connect('/app/data/db.db')
rows = conn.execute('SELECT status, COUNT(*) FROM cascade_task_allocations GROUP BY status').fetchall()
for r in rows: print('  ' + r[0] + ': ' + str(r[1]))
conn.close()
"

# 3. Check for Redis port conflict in logs
docker logs we-mp-rss 2>&1 | grep -i 'redis.*port.*占用\|redis.*6379'

Fix — Option A: Stop host Redis to free port 6379 (recommended)

# Check what's using port 6379 on host
lsof -i :6379

# Stop host Redis (if it's not needed by other services)
brew services stop redis

# Recreate container (port 6379 now available for internal Redis)
docker stop we-mp-rss && docker rm we-mp-rss
docker run -d ...(same flags as section 4 fix)...

# After restart, cascade nodes should auto-register
sleep 20
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db')
print('Nodes: ' + str(c.execute('SELECT COUNT(*) FROM cascade_nodes').fetchone()[0]))
print('Pending: ' + str(c.execute('SELECT COUNT(*) FROM cascade_task_allocations WHERE status=\"pending\"').fetchone()[0]))
"

Fix — Option B: Configure container Redis to use different port

-e REDIS_SERVER_PORT=6380
-e REDIS_URL=redis://127.0.0.1:6380/0

Fix — Option C: Accept limitation, manage new feeds manually New feeds require manual article injection when cascade workers can't register.

Key diagnostic signal: ALL cascade tasks pending + 0 timeout + 0 nodes = Redis/cascade worker issue, not cookie expiry. "timeout" means workers ran but hit WeChat API limits; "pending forever" means the system is fundamentally stuck.

19. New Feed Has 0 Articles — No Cascade Task Dispatched

Symptoms: RSS lists the subscription but /rss/{fakeid} returns 0 items. Feed has status=1 and recent sync_time but no entries in articles table.

Root Cause: Adding a feed to feeds table only registers subscription metadata. Article list collection requires cascade workers to execute. Without registered nodes (section 18), no collection happens.

Diagnosis:

# Step 1: Confirm feed in both DBs
docker exec we-mp-rss python3 -c "
import sqlite3
for db_name in ['db.db', 'we_mp_rss.db']:
    c = sqlite3.connect('/app/data/' + db_name)
    cnt = c.execute('SELECT COUNT(*) FROM feeds').fetchone()[0]
    print(db_name + ': ' + str(cnt) + ' feeds')
    c.close()
"

# Step 2: Check cascade nodes
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db')
n = c.execute('SELECT COUNT(*) FROM cascade_nodes').fetchone()[0]
p = c.execute('SELECT COUNT(*) FROM cascade_task_allocations WHERE status=\"pending\"').fetchone()[0]
print('Nodes: ' + str(n) + ', Pending: ' + str(p))
c.close()
"

# Step 3: Check RSS articles
curl -s \"http://localhost:8001/rss/{fakeid}?limit=3\" | grep -c '<item>'

Workaround (when cascade workers unavailable):

1. Sync feed to BOTH databases (section 9 dual-DB fix)
2. Creating pending cascade tasks manually helps only if workers exist
3. /api/v1/wx/sync?fakeid=XXX (GET 200) does NOT trigger collection — it's a status endpoint
4. Real fix: resolve Redis port conflict so cascade workers can register

20. Manual Article Collection from Inside the Container

Pitfall — wechat-article-discover.sh: script hangs due to set -euo pipefail + grep pipe at line 52

The cron script ~/.hermes/scripts/wechat-article-discover.sh has set -euo pipefail at the top and pipes the entire docker exec output through grep -v to filter noise:

docker exec we-mp-rss /app/env_x86_64/bin/python3 -c "..." 2>&1 | grep -v "^\[SQL\|^\[参数\|^FROM\|..."

When grep -v filters ALL lines (which is common when the Python script produces nothing but SQL chatter in quiet-mode), grep exits with code 1 (no matches). With set -euo pipefail, this causes the entire script to exit with code 141 (pipe error), which shows as:

• Script exited with code 141 if caught by set -e
• OR the script hangs indefinitely if the docker exec process is still running when grep's pipe closes (SIGPIPE is delivered but the docker exec parent process may not propagate the signal immediately)

Fix: Replace the grep pipe with inline Python filtering that never exits nonzero:

# Instead of: print(...) 2>&1 | grep -v "^\[SQL"
# Use Python-level filtering in the inline script:
import sys
# ... existing code ...
# At the very end, use a filter writer:
class FilterWriter:
    def __init__(self, orig): self.orig = orig
    def write(self, s):
        if not s.startswith('[SQL') and not s.startswith('[参数'):
            self.orig.write(s)
sys.stdout = FilterWriter(sys.stdout)

Alternative fix (simpler): Add || true after the grep pipe:

docker exec ... 2>&1 | grep -v "^\[SQL\|^\[参数" || true

When to use: Cascade workers are stuck (section 18), new feeds have 0 articles, or you need to force a one-shot collection for specific accounts without waiting for the scheduler.

Prerequisites:

• The correct Python interpreter is /app/env_x86_64/bin/python3, not the system python3 (system Python lacks sqlalchemy)
• Playwright browsers may need installing first: docker exec we-mp-rss /app/env_x86_64/bin/python3 -m playwright install chromium

Trigger full collection (ALL feeds — SLOW, ~10-15 min):

docker exec we-mp-rss /app/env_x86_64/bin/python3 -c "
import sys; sys.path.insert(0, '/app')
from jobs.mps import fetch_all_article
fetch_all_article()
"

Processes feeds sequentially, each with MaxPage=3 (default) in get_Articles() and a 6-second delay between feeds. With 30 feeds, expect ~10-15 minutes per run. The wechat-article-discover cron job must budget enough time.

Timeout pitfall: fetch_all_article() CAN hang on individual feeds. The function iterates ALL feeds with no per-feed timeout. A single stuck feed blocks the entire collection — the cron timeout kills the whole run before processing later feeds. This happens when a feed's Playwright page load hangs or the WeChat API returns a slow response.

Cron-safe approach (batching, used in wechat-article-discover.sh):

The cron script replaces the monolithic fetch_all_article() with a batched inline Python call that:

• Limits to 10 feeds per run (cron runs daily, over time all feeds are covered)
• Sets MaxPage=1 (only first page of articles, sufficient for discovery)
• Uses signal.alarm(20) for per-feed 20-second timeout
• Shorter 3-second delay between feeds (vs default 6s)
• Skips feeds that time out (1-2 per batch is normal)

docker exec we-mp-rss /app/env_x86_64/bin/python3 -c "
import sys, signal, time
sys.path.insert(0, '/app')
import core.db as db
from jobs.mps import UpdateArticle
from core.wx.base import WxGather

mps = db.DB.get_all_mps()
processed = 0; errors = 0
for item in mps:
    try:
        wx = WxGather().Model()
        signal.alarm(20)
        wx.get_Articles(item.faker_id, CallBack=UpdateArticle,
                       Mps_id=item.id, Mps_title=item.mp_name, MaxPage=1)
        signal.alarm(0)
        processed += 1
    except Exception as e:
        signal.alarm(0); errors += 1
    if processed + errors >= 10:
        break
    time.sleep(3)
print(f'Done: {processed}/{len(mps)} ok, {errors} err')
" 2>&1 | grep -v "^\\[SQL\\|^\\[参数\\|^FROM\\|^生成桌面\\|^生成移动\\|^Queue\\|^Redis\\|^ContentTaskQueue\\|^TaskQueueManager\\|^TaskQueue singleton\\|^Redis连接\\|^\\[默认\\]\\|^\\[文章采集API\\]\\|^\\[任务调度\\]"

Trigger collection for ONE feed (requires ORM object, not a string):

docker exec we-mp-rss /app/env_x86_64/bin/python3 -c "
import sys; sys.path.insert(0, '/app')
from core.db import DB
from core.models.feed import Feed
from jobs.mps import do_job
session = DB.get_session()
feed = session.query(Feed).filter(Feed.id == 'MP_WXS_XXXXXXXXX').first()
if feed:
    do_job(mp=feed)
"

Limitations:

• do_job(mp='fakeid_string') does NOT work — mp must be a Feed ORM object, not a bare string ID
• fetch_all_article() processes feeds in id order, starting from the lowest ID — new feeds added last may take 10+ minutes to reach
• GET /api/v1/wx/sync?fakeid=XXX returns HTTP 200 but does NOT trigger collection — it's a status/metadata endpoint
• Only cascade worker nodes can execute article list collection in the background. Manually calling fetch_all_article() is a one-shot workaround

Diagnose collection results:

docker exec we-mp-rss python3 -c "
import sqlite3
conn = sqlite3.connect('/app/data/db.db')
rows = conn.execute('SELECT f.mp_name, COUNT(a.id) FROM feeds f LEFT JOIN articles a ON f.id = a.mp_id GROUP BY f.id HAVING COUNT(a.id) = 0').fetchall()
print('Feeds with 0 articles:', len(rows))
for r in rows: print(' ', r[0])
conn.close()
"

Playwright browsers in the container: The Docker image ships Playwright Python package but NOT browser binaries. Install the correct one matching BROWSER_TYPE:

docker exec we-mp-rss /app/env_x86_64/bin/python3 -m playwright install chromium

Without Playwright browsers, the cascade content fetch step fails silently (content stays empty), though article metadata (title, URL, publish_time) is still added via API calls.

21. RSS content:encoded Content Model — How Articles Actually Reach the Extractor

Critical understanding: The extractor (wechat-mp-rss-extractor.py) does NOT fetch content from WeChat article pages. It reads from the RSS feed's <content:encoded> element, which is populated by we-mp-rss's internal systems before the extractor runs.

Content flow:

WeChat API → we-mp-rss discovers article → ContentTaskQueue fetches HTML
  → stored in DB (content_html + has_content=1)
  → RSS endpoint serves it in <content:encoded>
  → extractor reads RSS → finds content:encoded → writes inbox file

New articles injected directly into articles table (bypassing we-mp-rss discovery) have has_content=0 and content_html=NULL. The RSS endpoint returns them with empty <content:encoded>. The extractor sees the article entry, finds no content, attempts API refresh, then skips.

Diagnosis:

# Check if a feed's RSS has actual content
curl -s "http://localhost:8001/rss/{fakeid}?limit=1" | grep -c "content:encoded"
# 0 = no content → extractor will skip

# Check content_html size in DB
sqlite3 ~/data/we_mp_rss.db "SELECT length(content_html) FROM articles WHERE id='{aid}'"

# Check ContentTaskQueue activity
docker logs we-mp-rss 2>&1 | grep "content=true"

Content vs Metadata distinction: Article METADATA (title, URL, publish_time) is available via RSS even without content. The wiki can still use these as raw supplements with a link to the original article.

---

22. WeChat Anti-Bot Wall — Confirmed Across All Routes (2026-07-02)

Symptom: Any automated attempt to read WeChat article content from a non-residential IP triggers a captcha or "环境异常" page.

Route	Result	Root cause
Container Playwright (chromium)	Captcha ("安全验证")	Docker data center IP
Local Chrome Profile (Tier 3)	"环境异常，完成验证后即可继续访问"	Profile IP not residential
Hermes Browser (Browserbase)	Captcha popup	Free tier: no residential proxy

No fix without residential proxy. Workaround: user opens article on phone, pastes URL to agent for Tier 3 ingestion. Acceptable fallback: RSS metadata (title + URL) is sufficient for raw supplement.

---

23. New Feed Subscription — Complete Add & Initialize Checklist

When user requests "加订阅" for a new WeChat MP account:

1. Ingest the article (always): Playwright Profile → extract text → raw/articles/ → entity → commit
2. Extract __biz from article URL query param (?__biz=...). This is the faker_id.
3. Decode to feed_id: base64.b64decode(faker_id).decode() → MP_WXS_{numeric}
4. Get account name from article page: document.querySelector('#js_name')?.textContent
5. Insert into BOTH databases (we_mp_rss.db AND db.db) with INSERT INTO feeds
6. Restart container: docker restart we-mp-rss
7. Verify: RSS endpoint returns the new feed. Content may be empty (see section 22).

Limitation: Body content fetching is best-effort (anti-bot blocked). Inform user upfront.

Known Behaviors

• Container exit code 255 = abnormal termination, usually recoverable with docker restart
• Docker Desktop must be started before cron jobs can run (cron runs in background, no GUI)
• Service startup takes ~10 seconds after docker restart
• Inbox count is a dynamic equilibrium (~188-198 files), not a bug when stable
• 2 accounts (科技充电站, code秘密花园) will never have full content — we-mp-rss cannot scrape them

8. Host Proxy Environment Leaks Into Container — no_proxy + extra_hosts Required

Symptoms:

• Container inherits HTTP_PROXY=http://127.0.0.1:10808 from host shell
• Inside container, 127.0.0.1:10808 points to container loopback, not host — all external HTTPS requests fail with ProxyError('Cannot connect to proxy')
• QR code generation returns is_exists: false without error (can't reach mp.weixin.qq.com)
• Playwright browser install fails (can't download from cdn.playwright.dev)

Diagnosis:

docker exec we-mp-rss env | grep -i -E "proxy|http" | sort
# Wrong: http_proxy=http://127.0.0.1:10808  (container's own loopback!)

Fix — mp.weixin.qq.com must NOT be in no_proxy (GFW bypass):

    environment:
      - HTTP_PROXY=http://host.docker.internal:10808
      - HTTPS_PROXY=http://host.docker.internal:10808
      - no_proxy=localhost,127.0.0.1,::1,*.local,api.github.com,pypi.org,pypi.tuna.tsinghua.edu.cn,cdn.playwright.dev
      - NO_PROXY=localhost,127.0.0.1,::1,*.local,api.github.com,pypi.org,pypi.tuna.tsinghua.edu.cn,cdn.playwright.dev
    extra_hosts:
      - "host.docker.internal:host-gateway"

Diagnosis: If all REST API calls return ret=200009 or timeout but Playwright works, no_proxy likely includes mp.weixin.qq.com:

docker exec we-mp-rss env | grep no_proxy
# BAD: mp.weixin.qq.com in no_proxy → REST API direct = blocked by GFW

**Apply:** `docker rm -f we-mp-rss && docker compose -f compose/docker-compose-sqlite.yaml up -d`

### 9. QR Code Generation Fails — Playwright Browser Not Installed

**Symptoms:**
- Clicking "扫码授权" returns 404 for `/static/wx_qrcode.png`
- API returns `{"is_exists":false}` with no error message
- The WeChat MP login page uses Vue.js SPA — QR code URL is loaded dynamically

**Root Cause:** `driver/wx.py` uses Playwright headless browser to access `https://mp.weixin.qq.com/` and render JS. The Docker image ships Playwright Python package but NOT browser binaries. Without them, `_extract_qr_info()` returns `None` silently.

**Fix:**
```bash
# Install Playwright browser (~200MB, 1-2 min)
docker exec we-mp-rss pip3 install playwright
docker exec we-mp-rss python3 -m playwright install chromium

Note: Even with Playwright, we-mp-rss v1.5.2 has a code-level bug: _extract_qr_info() regex-scans the raw HTML (before JS renders), but the QR code URL is loaded dynamically by Vue.js. The regex pattern r'(https?://mp\.weixin\.qq\.com/cgi-bin/loginqrcode\?action=getqrcode&param=\d+)' will never match against the static HTML. The QR code flow is fundamentally broken in this version.

10. Adding Subscriptions Without QR Code

Prerequisite: Find the WeChat MP fakeid for the account. This requires either:

• Access to https://mp.weixin.qq.com/ backend with an authorized account
• Extracting __biz parameter from existing article URLs of the target account

Direct SQLite insert:

INSERT INTO feeds (id, mp_name, status, created_at, updated_at)
VALUES ('MP_WXS_xxxxxxx', '公众号名称', 1, datetime('now'), datetime('now'));

The faker_id field is base64-encoded version of the numeric ID.

Verify insertion:

curl -s http://localhost:8001/rss | grep '<title>' | grep -v 'WeRSS订阅'

11. Login API Endpoints — Correct Usage

Purpose	Endpoint	Method	Auth
Web login	/api/login?username=X&password=Y	GET	none
API token	/api/v1/wx/auth/token	POST	form-encoded username+password
QR code info	/api/v1/wx/auth/qr/code	GET	Bearer token
Verify token	/api/v1/wx/auth/verify	GET	Bearer token

Get API token:

curl -s -X POST "http://localhost:8001/api/v1/wx/auth/token" \
  -d "username=admin&password=$PASSWORD" | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])"

13. WeChat MP Login Page Format Change — QR Code URL Pattern Changed

Symptoms: Even after installing Playwright + browser, QR endpoint returns is_exists: false. Container can reach mp.weixin.qq.com (HTTP 200) but _extract_qr_info() returns None.

Root Cause: WeChat MP redesigned their login page. The QR code URL pattern changed:

Aspect	Old (code expects)	New (current)
Path	/cgi-bin/loginqrcode	/cgi-bin/scanloginqrcode
Parameter	param=\d+	random=\d+
App ID	In path	login_appid= (empty in static HTML)

The we-mp-rss v1.5.2 regex never matches the current page. Even Playwright cannot help — the QR code URL requires a login_appid value set dynamically by Vue.js after page init.

No clean fix available. Options:

• Fork we-mp-rss to update regex + parameter extraction
• Use host browser to manually log into mp.weixin.qq.com and inject cookies into wx.lic
• Manage subscriptions via SQLite directly if fakeids are known

14. BROWSER_TYPE Mismatch — webkit vs chromium (+ Default Hardcoded in Code)

Symptoms: Docker logs show "正在启动浏览器..." then "正在生成二维码图片..." then code_src:None. No error is logged. The QR code endpoint returns is_exists: false silently. Or: 启动浏览器失败: Executable doesn't exist at /root/.cache/ms-playwright/webkit-2203/pw_run.sh even though Chromium is installed and BROWSER_TYPE=chromium is set in env.

Root Cause (two levels):

1. Docker config: The docker-compose file has BROWSER_TYPE=webkit but the Playwright browser installed in the container is chromium. we-mp-rss driver/playwright_driver.py line 123 does getattr(self._playwright, self.browser_type) — when browser_type is "webkit", it tries to launch WebKit which doesn't exist.

2. Code default (critical): Even with BROWSER_TYPE=chromium in env and config, /app/driver/playwright_driver.py line 53 hardcodes browser_type: str = "webkit" as the default parameter value. All callers (wx.py:38, wx.py:368, wx.py:467, wxarticle.py:29) instantiate PlaywrightController() without passing browser_type, so the default "webkit" sticks regardless of env vars or config.yaml.

Fix — Two Parts:

Part A — Set env var:

# docker-compose-sqlite.yaml
    environment:
      - BROWSER_TYPE=chromium  # not "webkit"

Part B — Patch the code default (necessary fix — env var alone is NOT sufficient):

docker exec we-mp-rss sed -i 's/browser_type: str = "webkit"/browser_type: str = "chromium"/' /app/driver/playwright_driver.py
docker restart we-mp-rss

This fix must be reapplied after Docker image updates.

14. Host Proxy Env Leaks Into Container — no_proxy Required

Symptoms: Container inherits HTTP_PROXY=http://127.0.0.1:10808 from host shell. Inside container, 127.0.0.1 is container loopback, not host — external requests fail with ProxyError.

Fix in docker-compose-sqlite.yaml:

    environment:
      - no_proxy=localhost,127.0.0.1,::1,*.local,mp.weixin.qq.com,api.github.com,pypi.org,pypi.tuna.tsinghua.edu.cn,cdn.playwright.dev
    extra_hosts:
      - "host.docker.internal:host-gateway"

Apply: docker rm -f we-mp-rss && docker compose -f compose/docker-compose-sqlite.yaml up -d

15. Password Reset — Delete DB User to Force Recreate

When .env password differs from stored hash:

sqlite3 /Users/jinguo/data/we_mp_rss.db "DELETE FROM users;"
docker restart we-mp-rss
# Container reads USERNAME/PASSWORD from .env and creates fresh hash

Warning: Requires .env password to be correct — no fallback.

16. CDP Tab Creation — Use Target.createTarget (Not /json/new)

Symptoms: wechat-cookie-renew.py or fetch_article_content_via_cdp() returns "Cannot create tab" with HTTP 405 Method Not Allowed. Chrome DevTools /json/new?about:blank returns 405.

Root Cause: Chrome 149+ removed the /json/new endpoint. Must use Target.createTarget via WebSocket.

Fix in scripts:

version = json.loads(urllib.request.urlopen("http://127.0.0.1:9222/json/version").read())
browser_ws = version["webSocketDebuggerUrl"]
ws = websocket.create_connection(browser_ws, timeout=10)
ws.send(json.dumps({"id": 1, "method": "Target.createTarget", "params": {"url": "about:blank"}}))
resp = json.loads(ws.recv())
ws.close()
target_id = resp["result"]["targetId"]
time.sleep(1)
pages = json.loads(urllib.request.urlopen("http://127.0.0.1:9222/json", timeout=5).read())
tab_ws = next(p["webSocketDebuggerUrl"] for p in pages if p["id"] == target_id)

17. ENABLE_JOB vs SERVER_ENABLE_JOB — Which Env Var Controls the Internal Cron

Symptom: Setting SERVER_ENABLE_JOB=False in docker-compose doesn't disable fetch_all_article(). The internal cron continues to run.

Root Cause: The config.yaml reads enable_job: ${ENABLE_JOB:-True} — it's looking for ENABLE_JOB, not SERVER_ENABLE_JOB. The env var name must match the config template's ${VAR_NAME} exactly.

⚠️ Self-sustaining frequency control (2026-08-07): The internal job sweeps ALL feeds every GATHER.CONTENT_AUTO_INTERVAL (59 min). When WeChat frequency control is active (cooldown ≥ 60 min from the LAST request), each hourly sweep lands inside the cooldown window and RESETS the clock — the rate limit never lifts on its own. Observed: sweeps at 11:05 and 12:10 both fully throttled, with zero user requests in between.

Fix without container recreate (in-container config patch):

docker exec we-mp-rss sed -i 's/enable_job: ${ENABLE_JOB:-True}/enable_job: False/' /app/config.yaml
docker restart we-mp-rss
# Verify: server.enable_job must print False
docker exec we-mp-rss /app/env_x86_64/bin/python3 -c "
import sys; sys.path.insert(0, '/app')
from core.config import cfg
print('server.enable_job =', cfg.get('server.enable_job'))
" | grep -v '^\['

The literal enable_job: False (no ${} placeholder) survives the env override. ⚠️ Lost on container recreation — reapply after image updates.

Trade-off: With the internal job disabled, collection relies solely on the daily wechat-article-discover cron (09:10). This is acceptable: the internal job was returning 0 articles anyway during session/throttle issues, and the discover cron now ALERTs on 0-article runs (no more silent failure).

Fix:

    environment:
      - ENABLE_JOB=False  # This works
      # - SERVER_ENABLE_JOB=False  # This does NOTHING — wrong var name

Verify:

docker exec we-mp-rss /app/env_x86_64/bin/python3 -c "
from core.config import cfg
print('server.enable_job:', cfg.get('server.enable_job'))
" | grep -v "^\["

18. Direct Gmail SMTP for Cron Alerting (Bypass Proxy)

Problem: send-report.py uses HTTP CONNECT proxy (port 465) which frequently hangs at SSL handshake. Proxy-independent alerting is needed.

Pattern (used in wechat-monitor.py):

import smtplib, subprocess
from email.mime.text import MIMEText

pw = subprocess.run(
    ["security", "find-generic-password", "-a", "[e-mail]",
     "-s", "himalaya-smtp", "-w"],
    capture_output=True, text=True).stdout.strip()

msg = MIMEText(body, "plain", "utf-8")
msg["Subject"] = "[Wiki Alert] ..."
msg["From"] = "jinguo <[e-mail]>"
msg["To"] = "[e-mail]"

with smtplib.SMTP("smtp.gmail.com", 587, timeout=30) as s:
    s.ehlo(); s.starttls(); s.ehlo()
    s.login("[e-mail]", pw)
    s.send_message(msg)

Port 587 (STARTTLS) direct connection works when the proxy SSL tunnel on 465 hangs.

⚠️ 2026-08-07 update: Direct 587 is ALSO flaky on this network (timed out 10:25; succeeded on retry 11:29) — the connection path is intermittently blocked (see smtp-via-proxy P19-P22). wechat-monitor.py now: retries direct 587 ×3 with 5s backoff, then falls back to send-report.py (direct 465 via send-mail.py). Also fixed: alerted=True is only persisted when the email actually sends — previously a failed send still suppressed future alerts for the same state (permanent silence bug). Password from macOS Keychain (never hardcoded).

19. Adding Subscriptions Without QR Code (Direct SQLite Insert)

Requires fakeid (MP_WXS_xxx). The WeChat MP searchbiz API is DEPRECATED since ~2026-06 (returns ret=200005 "not supported"). Use __biz extraction instead.

Method: From article URL's __biz parameter (requires user's WeChat MP session)

1. User logs into https://mp.weixin.qq.com/ via QR scan in their browser
2. User provides cookies from DevTools -> Application -> Cookies -> mp.weixin.qq.com:
   • token (from URL after login, e.g. ?token=2076662488)
   • slave_sid (long base64 string)
   • slave_user (starts with gh_)
   • slave_bizuin (numeric)
3. Use host Playwright with these cookies to open the target account's article URL, extract window.biz
4. Key: faker_id = original __biz value, NOT re-encoded
5. Insert into SQLite via Python, then docker rm -f we-mp-rss && docker compose -f compose/docker-compose-sqlite.yaml up -d
6. Verify: curl -s http://localhost:8001/rss | grep -A1 '<title>' | head -20

The searchbiz API now returns {"base_resp": {"ret": 200005, "err_msg": "not supported"}} for all search methods. The only working path to add subscriptions is the __biz extraction from individual article URLs.

17. Host Playwright as QR Fallback

When container lacks Playwright browsers:

python3 -m playwright install chromium
# Then run the QR extraction script (references/2026-07-01-host-playwright-qr.md)

Limitations: Even with host Playwright, login_appid is often empty in the extracted URL — WeChat's server returns 0 bytes for the QR image (see section 13).

Disabling Internal Cron Jobs (When Upstream API Is Broken)

When we-mp-rss fetch_all_article() returns 200009: not found for every feed (WeChat deprecated the appmsgpublish API), the internal cron jobs waste resources polling a permanently broken endpoint. Disable them:

# docker-compose-sqlite.yaml
    environment:
      - GATHER.CONTENT=False
      - GATHER.CONTENT_AUTO_CHECK=False
      - GATHER.MODEL=none
      - GATHER.CONTENT_MODE=none
      - ENABLE_JOB=False

After docker rm -f we-mp-rss && docker compose -f compose/docker-compose-sqlite.yaml up -d, verify:

docker exec we-mp-rss /app/env_x86_64/bin/python3 -c "
from core.config import cfg
print('server.enable_job:', cfg.get('server.enable_job'))
print('gather.content:', cfg.get('gather.content'))
"
# All should show False

What's preserved: Web UI, RSS feeds, existing cached articles, the /rss endpoint, and the wechat-mp-rss-extractor.py cron (which reads from RSS independently). New article discovery stops — but it was already broken due to the API deprecation. See also the "Upstream article discovery chain broken" warning in wiki/wechat-mp-rss-extractor.

Skill Overlap

wiki/devops/we-mp-rss-troubleshooting is a shorter, stale copy. The curator may consolidate the two.

Auto Cookie Renewal (CDP-based)

See references/cdp-cookie-auto-renewal.md for the automated Chrome DevTools Protocol-based renewal system — persistent Chrome profile via launchd, CDP extraction script at scripts/wechat-cookie-renew.py, daily cron renewal. Only needs human intervention for the initial QR scan.

Pitfall: launchd Service Not Running — Cannot connect to Chrome DevTools on port 9222

Symptom: wechat-cookie-renew cron fails instantly with:

ERROR: Cannot connect to Chrome DevTools on port 9222.
Run: launchctl start com.hermes.wechat-chrome

(Verified 2026-08-27: the daily 09:00 run errored this way; the launchd service had stopped.)

Diagnosis:

launchctl list | grep wechat-chrome
# PID column `-` = loaded but NOT running (service exited)  → this is the cause
# PID column numeric = running (different problem, see profile-lock below)
lsof -iTCP:9222 -sTCP:LISTEN   # empty = not listening

Fix (simplest):

launchctl start com.hermes.wechat-chrome
sleep 5
# verify
launchctl list | grep wechat-chrome   # now has a PID
curl -s http://127.0.0.1:9222/json/version | head -1   # DevTools reachable
# then re-run the cron:
hermes cron run 08c75f2b6e77

If launchctl start succeeds (exit 0) but 9222 still isn't listening → the service is in the profile-lock failure below; check ~/.hermes/logs/wechat-chrome.err.

Pitfall: Chrome Profile Lock — Second Chrome Instance Cannot Start

Symptom: launchctl start com.hermes.wechat-chrome succeeds (exit 0) but port 9222 is not listening. Error in ~/.hermes/logs/wechat-chrome.err:

ERROR:chrome/browser/process_singleton_posix.cc:365] 另一个 Google Chrome 进程 (PID) 好像正在使用此个人资料。
Chrome 已锁定此个人资料以防止其受损。

Root Cause: The com.hermes.wechat-chrome launchd plist starts Chrome with --user-data-dir=/Users/jinguo/.hermes/chrome-wechat-profile. If any other Chrome process is already using this same profile directory (e.g., the user opened Chrome with this profile manually, or a previous launchd instance didn't clean up), the second Chrome instance will abort with "profile locked".

Diagnosis:

# Check if the profile is in use
lsof ~/.hermes/chrome-wechat-profile/ 2>/dev/null | head -5

# Check launchd status
launchctl list | grep wechat-chrome
# Status `-` means loaded but not running (exited)
# Status `0` means running

# Check port 9222
lsof -i :9222 -P 2>/dev/null | grep LISTEN

Fix — kill the conflicting Chrome process first: If Chrome is running with the same profile but without remote debugging:

# Find the PID using the profile directory
lsof ~/.hermes/chrome-wechat-profile/ 2>/dev/null | awk '{print $2}' | tail -1
# Kill it (careful — it may be the user's main Chrome)
kill <PID> || kill -9 <PID>

# Restart the launchd service
launchctl stop com.hermes.wechat-chrome
launchctl start com.hermes.wechat-chrome
sleep 3
lsof -i :9222 2>/dev/null | grep LISTEN

If port 9222 is already in use by a different process:

lsof -i :9222 -P 2>/dev/null
# Kill the conflicting process if safe
# Or change the port in the launchd plist

Pitfall: wx.lic requires token_data: wrapper format (two files needed)

Two bugs, same symptom — both must be fixed:

The wechat-cookie-renew.py writes wx.lic as a YAML config. The container's driver/token.py reads via wx_cfg.get("token_data", None), so the file must have a token_data: key at the top level. Flat cookie: / token: keys won't work.

# REQUIRED format:
token_data:
  token: '1907733930'
  cookie: 'wxuin=...; slave_sid=...'
  fingerprint: ''
  expiry:
    expiry_timestamp: 1783224289
    remaining_seconds: 7200
    expiry_time: '2026-07-05 10:04:49'

Second required file — key.lic (encrypted cookie array):

The container's Token() method calls Store.load() which reads data/key.lic — an HMAC-SHA256 encrypted JSON array of cookie objects. Without a valid key.lic, add_cookies() gets an empty list and the Playwright session never authenticates.

Encryption (default key "store.csol.store.werss"):

import hashlib, hmac, json
encrypt_key = hashlib.sha256(b"store.csol.store.werss").digest()
filtered = [c for c in cookies if c['name'] not in ('_clck', 'token')]
encrypted = hmac.new(encrypt_key, json.dumps(filtered).encode(), hashlib.sha256).digest() + json.dumps(filtered).encode()
open("/Users/jinguo/data/key.lic", "wb").write(encrypted)

Failing symptom: Every feed shows 请先扫码登录公众号平台 + 认证方式：Web认证=False.

2026-07-05 fix: Rewrote wechat-cookie-renew.py (both copies: ~/.hermes/scripts/ and skill scripts/) to write token_data wrapper + encrypted key.lic + health check retry 6×5s.

CDP Content Fetching (Plan A)

Since 2026-07-04, wechat-mp-rss-extractor.py uses Chrome CDP as Plan A for article content, with old API/Playwright as Plan B fallback.

Flow:

• RSS content missing → fetch_article_content_via_cdp(source_url) → subprocess: wechat-cdp-fetch.py → Chrome CDP (port 9222) → navigate → extract js_content → html2text (3-5s, ≈100% success)
• On failure → fetch_article_content_via_api(article_id) → ContentTaskQueue (old Plan B)

Requirements: Chrome CDP service running (launchd com.hermes.wechat-chrome), port 9222 accessible, Chrome profile logged into mp.weixin.qq.com.

Scripts: ~/wiki/scripts/wechat-cdp-fetch.py (standalone fetcher), ~/wiki/scripts/wechat-mp-rss-extractor.py (updated with CDP as Plan A).

See: references/wechat-pipeline-architecture.md for the full architecture document.

Historical Article Backfill Limitations

WeChat's profile_ext endpoint (公众号 history page) requires separate human verification beyond normal login cookies. CDP-based automation cannot discover articles from lost periods.

• Known URLs → CDP fetch works (≈100% success via wechat-cdp-fetch.py)
• Unknown URLs (lost periods) → Cannot auto-discover; profile_ext blocks all automation
• Backfill results: 12/39 articles recovered from DB (articles with metadata but no body); older URLs (2025, early 2026) return 404

See references/historical-article-backfill-limitation.md for full analysis.

12. Docker Compose Env Line Corruption — Two Env Vars on One Line

Symptoms:

• docker-compose-sqlite.yaml has a line like PASSWORD=*** - GATHER.CONTENT=True
• Password env var and next env var merged on the same line
• Container starts but password is stored as literal value *** followed by spaces
• Web UI login fails with "Could not validate credentials" for any expected password

Root Cause: YAML list item marker was lost during copy-paste or masking:

# WRONG — two env vars on one line (GATHER.CONTENT becomes part of PASSWORD value)
- PASSWORD=***      - GATHER.CONTENT=True

# CORRECT — two separate list items
- PASSWORD=*** Example=True

Fix: Split into two lines, then recreate container:

# Edit docker-compose-sqlite.yaml to have two separate lines:
#   - PASSWORD=my-password
#   - GATHER.CONTENT=True

docker rm -f we-mp-rss
cd /Users/jinguo/projects/we-mp-rss
docker compose -f compose/docker-compose-sqlite.yaml up -d

Login pitfall: The SQLite DB retains a bcrypt hash of the original password. If the original env was corrupted to ***, that's the stored hash. Log in with *** (three asterisks), not the new password.

9. DB File Mismatch — Container Uses Wrong SQLite DB

Symptoms: After container restart, subscription list shows 0 accounts. /Users/jinguo/data/ has db.db (with data) and we_mp_rss.db (empty). Container env says DB=sqlite:///data/we_mp_rss.db but data is in db.db.

Diagnosis:

sqlite3 /Users/jinguo/data/we_mp_rss.db "SELECT COUNT(*) FROM feeds;"
sqlite3 /Users/jinguo/data/db.db "SELECT COUNT(*) FROM feeds;"

Fix:

cp /Users/jinguo/data/db.db /Users/jinguo/data/we_mp_rss.db
docker restart we-mp-rss

Variant: Feeds out of sync between the two DBs

Symptoms: New subscriptions appear in the RSS subscription list (/rss?limit=30) and in we_mp_rss.db but have 0 articles and never get cascade collection tasks. RSS /rss/{fakeid} returns 0 items. we_mp_rss.db.feeds has more subscriptions than db.db.feeds.

Root Cause: The Web UI and RSS endpoint write to we_mp_rss.db (defined by DB= env var), but the cascade task scheduler reads its feed list from db.db (auto-created by the older data migration layer). When new subscriptions are added via Web UI or bulk import, they land in we_mp_rss.db only. The scheduler never dispatches article collection tasks for accounts it doesn't know about.

Diagnosis:

# Compare feed counts
docker exec we-mp-rss python3 -c "
import sqlite3
a = sqlite3.connect('/app/data/we_mp_rss.db').execute('SELECT COUNT(*) FROM feeds').fetchone()[0]
b = sqlite3.connect('/app/data/db.db').execute('SELECT COUNT(*) FROM feeds').fetchone()[0]
print(f'we_mp_rss.db: {a} feeds, db.db: {b} feeds')
"

Fix: Sync feeds from the active DB to the scheduler DB

docker exec we-mp-rss python3 -c "
import sqlite3
n = sqlite3.connect('/app/data/we_mp_rss.db')
o = sqlite3.connect('/app/data/db.db')
cols = [c[1] for c in n.execute('PRAGMA table_info(feeds)').fetchall()]
old_names = set(r[0] for r in o.execute('SELECT mp_name FROM feeds').fetchall())
added = 0
col_names = ','.join(cols)
placeholders = ','.join('?' for _ in cols)
for row in n.execute('SELECT * FROM feeds').fetchall():
    d = dict(zip(cols, row))
    if d['mp_name'] not in old_names:
        o.execute('INSERT OR IGNORE INTO feeds (' + col_names + ') VALUES (' + placeholders + ')', row)
        added += 1
o.commit()
print(f'Synced {added} new feeds to db.db')
"
docker restart we-mp-rss

Both resolve under /Users/jinguo/data bind mount -> container's /app/data.

4. Alive but Stale — Proxy Fixed but Cookie Expired (critical)

Symptom: Docker container HTTP 200, all 21 feeds sync_time updated to today, RSS returns content, but no new articles since May 2026. Extractor returns 0 new writes. Cascade task allocations all show status=timeout with error "任务超时（>30分钟）".

Diagnosis:

# 1. Check cascade task allocation status
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db').cursor()
rows = c.execute('SELECT status, COUNT(*) FROM cascade_task_allocations GROUP BY status').fetchall()
for r in rows: print(f'  status={r[0]}: {r[1]}')
"
# If all are 'timeout' → cookie expired

# 2. Check cookie expiry in wx.lic
docker exec we-mp-rss python3 -c "
import json
with open('/app/data/wx.lic') as f:
    d = json.load(f)
print('Cookie expiry:', d.get('expiry', {}).get('expiry_time', 'unknown'))
"

# 3. Check latest article dates
docker exec we-mp-rss python3 -c "
import sqlite3
c = sqlite3.connect('/app/data/db.db').cursor()
rows = c.execute('''
    SELECT f.mp_name, a.title, datetime(a.publish_time, \"unixepoch\")
    FROM articles a JOIN feeds f ON a.mp_id = f.id
    WHERE a.status=1 AND a.has_content=1 AND length(a.content)>100
    GROUP BY a.mp_id HAVING MAX(a.publish_time)
    ORDER BY a.publish_time DESC LIMIT 25
''').fetchall()
for r in rows: print(f'{r[0]:30s}  {r[2]}  {r[1][:40]}')
"
# If latest dates are 45+ days ago → cookie expired

Root cause: WeChat mp platform login cookie expired. The proxy fix (host.docker.internal:10808) restored sync_time updates (feeds table) but the expired cookie prevents article collection. All 273+ cascade task allocations timeout because do_job() calls wx.get_Articles() which fails silently with expired credentials.

Fix: Re-login to WeChat mp platform and inject cookies into container

The container's built-in "扫码授权" button (API: /api/v1/wx/auth/qr/code) uses headless Playwright WebKit which cannot produce a viewable QR code screenshot — the file /app/static/wx_qrcode.png is never written. Do not rely on the container's internal QR code flow.

Instead, use the browser tool to login externally and inject cookies:

# Step 1: Open mp.weixin.qq.com in browser tool, switch to QR scan mode
browser_navigate(url="https://mp.weixin.qq.com/")
# Click the QR scan tab:
browser_console(expression="document.querySelector('.login__type__container__select-type.login__type__container__select-type__scan').click()")
# Show QR code to user for scanning
browser_get_images()  # Look for the scanloginqrcode image

# Step 2: After user scans, navigate to home to verify login
browser_navigate(url="https://mp.weixin.qq.com/cgi-bin/home")
# Should show dashboard, not "登录超时"

# Step 3: Extract cookies and token from the logged-in browser session
browser_console(expression="document.cookie")
browser_console(expression="// Extract token from URL\nlet m = window.location.href.match(/token=([^&]+)/);\nm ? m[1] : ''")

# Step 4: Format and inject into container's wx.lic
# The wx.lic file format is YAML-like:
# token_data:\n  cookie: '<cookie_string>'\n  expiry:\n    expiry_time: '<YYYY-MM-DD HH:MM:SS>'\n    expiry_timestamp: <unix_ts>\n    remaining_seconds: <N>\n  ext_data: {...}\n  fingerprint: ''\n  token: '<token>'\n
# Write to container via heredoc:
# docker exec -i we-mp-rss sh -c 'cat > /app/data/wx.lic' << 'LICEOF'
# token_data:
#   cookie: '...'
#   ...
# LICEOF