bilibili-page-reader

# Bilibili Page Reader ## Core Rules - **Browser access** via `kimi-webbridge` for page state, login-only data, Bilibili Evolved (BE) providers. - **Do not click BE download buttons.** Call providers directly via evaluate. - **Always use a named session.** A session-less evaluate may land on a different tab and silently return wrong data. - **Network calls outside the browser** (audio download, playurl API) use Node.js — PowerShell's `curl.exe` and Python's requests both get blocked by Bilibili CDN TLS fingerprinting on this platform. --- ## Workflow Overview ``` ┌─ BE downloadSubtitles provider ──→ 投稿字幕 (timestamps) │ BiliBili video ───┼─ BE downloadDanmaku provider ────→ danmaku density/peaks/sample │ └─ No subtitles? ──→ Audio transcription fallback 1. Get audio stream URL (Node.js → playurl API) 2. Download .m4s audio 3. ffmpeg → .m4a 4. FunASR paraformer-zh → SRT with timestamps ``` --- ## Phase 1: Subtitles (preferred — via BE) Use the one-shot evaluate below. It returns both subtitles and danmaku in a single call. ### Setup ```bash ~/.kimi-webbridge/bin/kimi-webbridge status ``` ```json {"action":"navigate","args":{"url":"https://www.bilibili.com/video/BV.../","newTab":true},"session":"bilibili"} ``` Wait 2–3 seconds for BE to fully initialize. ### One-shot evaluate ```js (async () => { const pa = window.bilibiliEvolved.pluginApis; // ── Identifiers ── const s = window.__INITIAL_STATE__ || {}; const vd = s.videoData || {}; const bvid = vd.bvid || s.bvid || location.pathname.match(/BV[\w]+/)?.[0]; const aid = vd.aid || s.aid; const pages = vd.pages || []; const p = parseInt(new URLSearchParams(location.search).get('p') || '1') - 1; const cid = pages[p]?.cid || vd.cid || s.cid || pages[0]?.cid; const title = vd.title || document.title; // ── Register providers ── pa.registerData('downloadVideo.assets', []); // ── Poll for providers (downloadDanmaku loads async, ~1-2s) ── function getProviders() { const g = pa.getData('downloadVideo.assets'); return Array.isArray(g[0]) ? g.flat() : g; } const deadline = Date.now() + 5000; let providers = getProviders(); while (!providers.find(p => p.name === 'downloadDanmaku') && Date.now() < deadline) { await new Promise(r => setTimeout(r, 300)); providers = getProviders(); } // ── Subtitles: try 投稿字幕 first ── let subResult = { count: 0, text: '', source: 'none' }; const subProvider = providers.find(p => p.name === 'downloadSubtitles'); if (subProvider) { try { const subAssets = await subProvider.getAssets([{ input: {} }], { type: 'json', enabled: true }); const subRaw = subAssets[0].data; let subText; if (subRaw instanceof Blob) { const buf = await subRaw.arrayBuffer(); subText = new TextDecoder('utf-8').decode(buf); } else { subText = String(subRaw); } const subtitles = JSON.parse(subText); const subLines = subtitles.map(s => { const totalSec = Math.floor(s.from); const h = Math.floor(totalSec / 3600); const m = Math.floor((totalSec % 3600) / 60); const sec = String(totalSec % 60).padStart(2, '0'); if (h > 0) { return '[' + h + ':' + String(m).padStart(2, '0') + ':' + sec + '] ' + s.content; } return '[' + m + ':' + sec + '] ' + s.content; }); subResult = { count: subtitles.length, text: subLines.join('\n'), source: '投稿字幕' }; } catch(e) { subResult = { count: 0, text: '', source: '投稿字幕_error' }; } } // ── Danmaku: analyze in-page, summary only ── const dmk = providers.find(p => p.name === 'downloadDanmaku'); const dmkAssets = await dmk.getAssets( [{ input: { aid: String(aid), cid: String(cid) } }], { type: 'json', enabled: true } ); const dmkRaw = dmkAssets[0].data; let dmkText; if (dmkRaw instanceof Blob) { const buf = await dmkRaw.arrayBuffer(); dmkText = new TextDecoder('utf-8').decode(buf); } else { dmkText = String(dmkRaw); } const danmaku = JSON.parse(dmkText); // Time density: 30s buckets const bucketSize = 30; const buckets = {}; for (const d of danmaku) { const b = Math.floor(d.progress / 1000 / bucketSize) * bucketSize; buckets[b] = (buckets[b] || 0) + 1; } const density = Object.entries(buckets) .map(([t, c]) => [Number(t), c]) .sort((a, b) => a[0] - b[0]); // Top 5 peak moments const peaks = density.slice().sort((a, b) => b[1] - a[1]).slice(0, 5); // Stratified sample: up to 40 entries across full timeline const sampleCount = Math.min(40, danmaku.length); const step = Math.max(1, Math.floor(danmaku.length / sampleCount)); const danmakuSample = []; for (let i = 0; i < danmaku.length && danmakuSample.length < sampleCount; i += step) { danmakuSample.push({ t: Math.floor(danmaku[i].progress / 1000), c: danmaku[i].content }); } const totalDuration = danmaku.length > 0 ? Math.max(...danmaku.map(d => d.progress)) : 0; // ── Return ── return JSON.stringify({ ok: true, bvid, aid: Number(aid), cid: Number(cid), p: p + 1, title, sub: subResult, dmk: { count: danmaku.length, timeSpanSec: Math.floor(totalDuration / 1000), density, peakMoments: peaks.map(pk => ({ timeSec: pk[0], count: pk[1] })), sample: danmakuSample } }); })() ``` ### Response shape ```json { "sub": { "count": 243, "text": "[0:00] 大家好\n[0:01] 这个视频...", "source": "投稿字幕" }, "dmk": { "count": 247, "density": [[0,11], [30,2], ...], "peakMoments": [{"timeSec": 1170, "count": 16}, ...], "sample": [{"t": 0, "c": "辛苦惹！"}, ...] } } ``` If `sub.count === 0`, no 投稿字幕 was available. Proceed to **Phase 2** below. --- ## Phase 2: Audio Transcription Fallback (when no subtitles exist) Use when Phase 1 returns `sub.count === 0`. This replaces the old "AI subtitle API" fallback which is unreliable — the `/x/player/v2` API often returns stale/empty subtitle data. ### Why this approach The Bilibili CDN uses **TLS fingerprinting** that blocks curl and Python requests. Reliable paths: | Method | Works? | Notes | |--------|--------|-------| | `curl.exe` with browser headers | ❌ Exit code 35 | SSL blocked | | Python `requests` / `urllib` | ❌ Blocked | Same reason | | Node.js `https.get()` | ✅ | Use `User-Agent` + `Referer` headers | | Browser fetch (in-page) | ✅ but evaluate timeout <1s | Only for fast API calls | | Node.js to download audio | ✅ | Backup URL is most stable | ### Step 1: Get audio stream URL via Node.js ```bash node -e " const https = require('https'); const url = 'https://api.bilibili.com/x/player/playurl' + '?bvid=BVxxxxxxxxx&cid=xxxxxxxxx&qn=0&fnval=4048&fourk=1&platform=web'; https.get(url, { headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36', 'Referer': 'https://www.bilibili.com/' } }, (res) => { let data = ''; res.on('data', (chunk) => { data += chunk; }); res.on('end', () => { const d = JSON.parse(data); if (d.code === 0 && d.data && d.data.dash && d.data.dash.audio) { const audio = d.data.dash.audio; const best = audio.reduce((a,b) => a.bandwidth > b.bandwidth ? a : b); console.log(best.baseUrl + '|' + (best.backupUrl ? best.backupUrl[0] || '' : '')); } else { console.log('ERROR: no dash audio data'); } }); }).on('error', (e) => { console.log('ERROR: ' + e.message); }); " ``` The `fnval=4048` flag requests DASH format with separate audio/video streams. ### Step 2: Download audio (use backup URL — most reliable) ```bash node -e " const https = require('https'); const fs = require('fs'); const url = 'BACKUP_URL_FROM_STEP_1'; const file = fs.createWriteStream('output.m4s'); https.get(url, { headers: { 'User-Agent': 'Mozilla/5.0 ...', 'Referer': 'https://www.bilibili.com/' }, timeout: 60000 }, (res) => { if (res.statusCode !== 200) { console.log('HTTP ' + res.statusCode); return; } res.pipe(file); res.on('end', () => { console.log('OK size=' + file.bytesWritten); }); }).on('error', (e) => { console.log('ERR: ' + e.message); }); " ``` ### Step 3: Convert to standard audio (stream copy, no re-encode) ```bash ffmpeg -i output.m4s -c copy output.m4a -y ``` ### Step 4: ASR transcription via FunASR ```bash set MODELSCOPE_CACHE=.\cache\modelscope set MODELSCOPE_CREDENTIAL_PATH=.\cache\modelscope_cred python audio2srt.py output.m4a --srt --model paraformer-zh --punc-model ct-punc --spk-model cam++ ``` ### Step 5: Cleanup — remove intermediate audio files ```bash del output.m4s output.m4a ``` The only artifacts worth keeping are the `.srt` subtitle file. The raw `.m4s`, the converted `.m4a`, and any `.json` debug output are all intermediate and should be cleaned up after the SRT is confirmed valid. **What the script does:** - Loads `paraformer-zh` (Chinese ASR, 220M params) + `fsmn-vad` + `ct-punc` + `cam++` - Outputs SRT with per-sentence timestamps via `res[0]["sentence_info"]` - CPU perf: ~15x realtime (21 min audio ≈ 90 sec) - Handles ModelScope cache/credential path setup automatically **If `cam++` is too slow to download**, omit `--spk-model` and group `res[0]["timestamp"]` (per-character ms array) by punctuation boundaries manually. **If no `sentence_info`** in the output: the `spk_model` triggers sentence segmentation. Without it, you only get raw text + per-character timestamps. ### The `audio2srt.py` script Located next to this `SKILL.md` file. Copy it into the current workspace or call it by its skill-directory path. It handles: - Model loading with ModelScope credential path override (sandbox-safe) - Multiple output formats: SRT, VTT, JSON - Parsing `sentence_info`, `timestamp`, and raw text fallback - Cleaning SenseVoice special tags (`<|zh|>`, `<|HAPPY|>`, etc.) - Segment deduplication and time-sorting