菜单
Audit a Hugging Face dataset before training to confirm splits, columns, format, sample rows, distributions, and duplicates. Triggered before any training/fine-tuning script runs, when a user mentions a new dataset, or when you hit a KeyError / format mismatch in a training job.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/mybigday/ml-intern-kit --skill inspect-dataset命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
正在显示 SKILL.md
基于 SOC 职业分类
| name | inspect-dataset |
| description | Audit a Hugging Face dataset before training to confirm splits, columns, format, sample rows, distributions, and duplicates. Triggered before any training/fine-tuning script runs, when a user mentions a new dataset, or when you hit a KeyError / format mismatch in a training job. |
Never train on a dataset you haven't audited. Looking at the data is the single most reliable way to avoid format mismatches, KeyError-on-step-1 failures, and silently-bad runs.
train_sft.py / train_dpo.py / train_grpo.py invocation.KeyError, ValueError: column not found, or
bad samples.python scripts/inspect_dataset.py <repo_id> # default audit
python scripts/inspect_dataset.py <repo_id> --split train --n 5
python scripts/inspect_dataset.py <repo_id> --config wikitext-2-raw-v1
The script prints schema, row counts per split, length statistics on text /
prompt / messages columns, duplicate counts, label distribution, and n sample
rows.
| Method | Required columns |
|---|---|
| SFT | messages (chat) or text or prompt+completion |
| DPO | prompt, chosen, rejected |
| GRPO | prompt (+ a reward function) |
If the columns don't match the method, stop. Tell the user; don't silently remap or substitute the dataset.
max_seq_length choice.messages already wrapped in a list of dicts vs.
flat text).