data-lineage

name	data-lineage
description	记录与追溯多版本数据/实验的派生血缘。用于回答"X 哪来的"、"X 排除了什么"、"X 和 parent 的差异是什么"。Triggers when user asks about data origin, derivation history, audit trail, removal/addition rationale, or completeness verification across versions.
user-invocable	true
allowed-tools	Read, Write, Edit, Bash, Glob, Grep

Data Lineage

记录每次派生的 parent → child 关系, 形成可追溯的 DAG。

何时写

• 创建任何派生 artifact 时
• 修改 split / manifest / filter 后立即更新
• 通过 verification 后标 verified.*

不可选: 多版本项目里 lineage 是基础设施。

Schema

<artifact>/_provenance/lineage.json:

{
  "version": "<this>",
  "created_at": "ISO-8601",
  "parent": {
    "version": "<parent>",
    "lineage_ref": "<relative-path-to-parent-lineage>"
  },
  "derivation": {
    "method": "<filter | expand | merge | split | rebase>",
    "script": "<entry-script-path>",
    "command": "<reproducible-command>"
  },
  "diff_summary": "<one-sentence single-variable change>",
  "delta": {
    "removed": [],
    "removed_count": 0,
    "removal_criteria": "<doc-ref>",
    "added": [],
    "added_count": 0
  },
  "manifest_summary": {},
  "verified": {
    "diff_is_single_variable": false,
    "manifest_math_consistent": false,
    "isolation_holds": false,
    "completeness_passed": false
  },
  "metrics": {},
  "reports": []
}

字段约束:

• parent.lineage_ref 用相对路径, 避免移动断链
• derivation.command 必须可复现
• verified.* 默认 false, 验证通过后 set true
• diff_summary 一句话, 一次只描述一个变量

三种使用模式

创建

mkdir _provenance/
write lineage.json with verified.* = false
update each verified.* upon respective check passing

反查

walk parent.lineage_ref upward until root
at each level check delta.removed/added for target id
output lineage tree

验证

assert all(verified.values())
assert manifest_summary == parent_manifest - removed + added
assert isolation: train ∩ test = ∅ (and other invariants)

反模式

• 派生不写 lineage → 下游追溯成本爆炸
• verified 全 false 就 commit → 数据不完整
• diff_summary 含多个变量 → 违反单变量原则
• parent.lineage_ref 用绝对路径 → 移动断链
• 一次写完 lineage 不增量 → 不能 fail fast