| name | finfluencer-audit |
| description | Audit whether a financial influencer's public calls actually worked, using
pre-registered sampling, blind claim coding, and market backtesting with
honest baselines.
Use when the user asks to evaluate/verify/backtest a 财经博主 / trading UP主 /
finfluencer / stock guru — e.g. "这个博主靠谱吗", "帮我评估 B 站某某的投资视频",
"backtest this YouTuber's calls", "他的胜率是真的吗", "值不值得报他的课".
Covers Bilibili (reference implementation, hardest anti-bot), plus YouTube and
any yt-dlp-listable channel. Produces a graded Markdown report, a claim-level
evidence ledger, and an optional interactive canvas.
NOT for: picking stocks, generating trading signals, or making accusations of
fraud. This measures whether public statements were checkable and whether they
beat a dumb baseline.
|
Finfluencer Audit
Most "this guru has a 70% win rate" claims collapse under two questions: win
rate on what fraction of their content? and versus what baseline? This skill
answers both, reproducibly.
Non-negotiables
- Freeze before you score. Write the sample list and the claim ledger to
disk before loading a single price. Otherwise you will unconsciously read
ambiguous headlines in the direction the market went.
- Always report a baseline. A hit rate without
always-long next to it is
marketing, not evidence. In a bull market, "just hold it" often beats the
guru.
- Report the falsifiable rate. If only 40% of videos contain a checkable
claim, say so in the headline. Accuracy on a self-selected 40% is not
accuracy.
- Right-censor, don't guess. A 12-month call made 3 months ago is
unresolved, not wrong.
- Grade the evidence tier. Coding from titles is much weaker than coding
from transcripts. Never let the reader assume you had transcripts.
- No accusations. Report what was said and what the market did. Flag
unverifiable credentials as unverified, not as false.
Pipeline
fetch_archive → build_sample → fetch_content → code_claims
↓ (freeze)
report ← backtest ← fetch_market
Each step writes JSON into a run directory so any number can be traced back to
a video URL.
1. Build the sampling frame
python3 scripts/fetch_archive.py --platform bilibili --uid 25270495 \
--since 2023-08-06 --until 2026-08-06 --out runs/<name>
Bilibili's space archive needs a WBI signature, dm_img_* anti-bot fields, and
buvid cookies all at once — see references/platforms.md
for the failure codes and what each one means. For YouTube use
--platform ytdlp --channel-url ....
2. Draw a pre-registered sample
python3 scripts/build_sample.py --run runs/<name> --margin 0.08 --seed 20260806 --two-phase
Sample size is Cochran's formula with a finite-population correction. Use
--two-phase when most titles are chit-chat: it screens for directional
language and oversamples that stratum, which is the difference between 42
scoreable claims and ~90 from the same budget.
Publish sample.json before continuing. Substitute only from
alternates.json, and only for deleted or unreadable videos.
3. Collect evidence
python3 scripts/fetch_content.py --run runs/<name> --transcribe whisper --transcribe-limit 30
Subtitles first, audio transcription second, title-only last. Check
content_meta.json — if subtitle + transcript coverage is near zero, your
audit is a headline audit and the report must say so.
4. Code claims, then freeze
python3 scripts/code_claims.py --run runs/<name> --dual-channel
Direction is resolved per asset per clause, because titles like
「原油暴跌，黄金后市可期」 are bearish oil and bullish gold. --dual-channel
runs a second stricter coder and reports Cohen's kappa; below 0.6, say plainly
that coding noise rivals the measured effect.
Coding rules and the claim taxonomy live in
references/coding-protocol.md.
5. Get prices
python3 scripts/fetch_market.py --run runs/<name> --start 2023-07-01 --end 2026-08-07
Every free source fails sometimes; the script walks Yahoo → Stooq → FRED and
records which one answered. See references/market-data.md.
6. Backtest
python3 scripts/backtest.py --run runs/<name> --as-of 2026-08-06
Entry is the first close on or after the publish date. Outputs hit rate with a
Wilson interval, signed returns, MFE/MAE, and three baselines, cut by horizon,
asset, direction, and claim type.
7. Report
python3 scripts/report.py --run runs/<name> --name "BOSS墨" --profile-url https://space.bilibili.com/25270495/
Writes report.md and canvas_payload.json. For an interactive deliverable,
render the payload with the canvas skill.
Beyond price accuracy
A reliability verdict needs more than a hit rate. Also check, and cite sources
for each:
- Credentials — is the self-described qualification independently findable
(regulator registry, patent/trademark office, employer)? Record
verified,
not_found, or unverified_hearsay — never false without documents.
- Track record — brokerage statements or third-party audit, or hearsay?
- Conflicts — paid courses, private groups, referral links, token bags.
- Error handling — do they publish corrections, or only "as planned" recaps?
A high
post_hoc_review count with a low ex_ante_prediction count is a tell.
- Selection — do they delete losing calls? Compare archive count against the
platform's reported total.
Pitfalls
Read references/pitfalls.md before your first run. The
short version:
| Trap | Consequence |
|---|
| Whole-title direction voting | Multi-asset calls get inverted |
| No baseline | Bull-market drift reads as skill |
| Counting unmatured calls as misses | Recent calls drag the score down |
| Title-only coding presented as analysis | Overstated confidence |
| Sampling without a seed | Nobody can reproduce you |
| Scoring before freezing the ledger | Hindsight leaks into coding |
Output contract
A finished audit answers, in this order:
- What fraction of content was checkable at all?
- Of that, what was the hit rate, with a confidence interval?
- How does it compare to always-long, coin flip, and trend following?
- Which specific calls hit and missed, with links?
- What is unverifiable about the creator's own claims?
- What would change the verdict?
