メニュー
Builds the transparent rule-based baseline every model must beat — a hand-written score with reason codes, ranked output, and precision@K evaluation. Use before training any model, or when someone reports model results with nothing to compare against.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Finds label leakage and designs honest validation — leakage taxonomy, grouped and time-aware splits, base rates, and the attack-your-own-model checklist. Use before trusting any metric, when a score looks too good, or when features and labels share time windows or origins.
The FlyRank internship datasets — the 30k-row starter CSV and its gotchas, the ~79M-row warehouse release tables and grains, panel warnings, access, and iteration rules. Load for EVERY task that touches the data. (Project-specific: delete this folder when reusing the skill library elsewhere.)
Works with datasets far too big to download or load in pandas — SQL over remote Parquet with DuckDB, aggregate-then-model, iterate on samples. Use when a dataset has millions of rows, lives on a remote host (hf:// or s3), or a notebook runs out of memory or hits rate limits.
Runs an honest EDA and signal audit — distributions first, heavy-tail handling, verdict-style mini-tests (CONFIRMED/OPPOSITE/MIXED/FALSE), sample-size floors. Use before modeling, when testing whether a popular belief shows up in data, or when a correlation looks too good.
Deploys a static page (research paper, portfolio piece) for free from a GitHub repo using GitHub Pages — setup, file layout, verification, and recording the final URL. Use when publishing a finished write-up as a public web page.
Teaches the working method for using an AI coding assistant on analysis tasks — one task per fresh conversation, search before assuming, interview before big plans, validate everything, commit per finished task. Use at the start of any assignment, whenever a conversation gets long and confused, or when the assistant starts guessing.
| name | building-baselines |
| description | Builds the transparent rule-based baseline every model must beat — a hand-written score with reason codes, ranked output, and precision@K evaluation. Use before training any model, or when someone reports model results with nothing to compare against. |
A model without a baseline is a number without a meaning. The baseline is a rule a human can read — and its job is to be honestly beatable.
1. Say the rule in plain words first. "A page is worth reviewing if it used to get traffic, it's getting old, and its position is slipping." If you can't say it, you can't code it.
2. Code it as a transparent score. Multiply/add simple conditions; no fitted weights:
stale = (df["days_since_update"] >= 180).astype(int)
visible = (df["impressions"] >= 500).astype(int)
df["score"] = stale * visible * df["impressions"] # readable on purpose
3. Attach reason codes. Every scored item carries WHY it scored ("stale_but_visible", "position_slipping"). Reason codes are what make a ranked list trustworthy to a human.
4. Rank and evaluate at K. For "which ones first?" problems the honest metric is precision@K: of the top K the rule flags, how many were actually right?
def precision_at_k(scores, labels, k):
import numpy as np
order = np.argsort(-np.asarray(scores))
return np.asarray(labels)[order[:k]].mean()
Always print the base rate next to it (labels.mean()): precision@50 of 0.60 means little until you know whether random picking gives 0.55 or 0.10.
5. Review the top of the list by hand. The top 20 is where bad logic shows itself. For each: the action, the reason code, a confidence note, and what would make it wrong.