メニュー
Extracts and maps data lineage from various sources including SQL, dbt, Airflow, and Spark, generating comprehensive lineage graphs for impact analysis.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/a5c-ai/babysitter --skill data-lineage-mapperコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
SKILL.md を表示中
Reference for querying the Atlas knowledge graph through its MCP tools — the SECONDARY enrichment/comparison layer that adds best-practice context to systems you have ALREADY scanned from your real sources (`az`, repos, dirs). Use when you need to look up nodes, edges, kinds, clusters, stats, or wiki pages in Atlas to compare against your real inventory. (atlas graph, query atlas, atlas mcp, search the graph, graph neighbors, atlas record, atlas kinds, enrichment layer)
Atlas turns your STATED NEED into a real systems atlas by SCANNING your actual sources (Azure via `az`, git repos, local dirs) and process/data mining them, THEN enriching against the Atlas knowledge graph. Use this skill when asked to inventory/map your real systems, scan your cloud + repos + directories, mine the real processes or data they contain, or collect their real constraints/gotchas. (atlas, scan my systems, inventory our azure account, map my repos, real systems atlas, process mining, data mining, collect nuances, system discovery)
This skill should be used when the user asks to "find skills in the wild", "assimilate popular workflows", "discover SKILL.md files in repos", "research external skills", "find workflow patterns", "survey the skill landscape", "what skills exist out there", or wants to investigate public repositories for extractable processes, babysitter plugins, and reusable procedural insights. Searches GitHub for SKILL.md files, classifies repos by archetype, and maintains structured research under docs/reference-repos/.
SOC 職業分類に基づく
| name | data-lineage-mapper |
| description | Extracts and maps data lineage from various sources including SQL, dbt, Airflow, and Spark, generating comprehensive lineage graphs for impact analysis. |
| version | 1.0.0 |
| category | Data Governance |
| skill-id | SK-DEA-010 |
| allowed-tools | Read, Grep, Glob, Bash, WebFetch |
| graph | {"domains":["domain:data-engineering"],"specializations":["specialization:data-engineering-analytics"],"skillAreas":["skill-area:data-lineage","skill-area:data-governance"],"roles":["role:data-engineer","role:analytics-engineer"],"workflows":["workflow:data-quality-monitoring"],"topics":["topic:data-mesh"]} |
Extracts and maps data lineage from various sources to provide comprehensive data flow visibility.
This skill parses and extracts data lineage information from SQL queries, dbt projects, Airflow DAGs, and Spark jobs. It generates comprehensive lineage graphs showing data flow from source to destination, enabling impact analysis and data governance.
{
"sources": {
"type": "array",
"required": true,
"items": {
"type": {
"type": "string",
"enum": ["sql", "dbt", "airflow", "spark", "file"]
},
"content": {
"type": "string|object",
"description": "SQL string, file path, or manifest object"
},
"metadata": {
"type": "object",
"properties": {
"database": "string",
"schema": "string",
"catalog": "string"
}
}
}
},
"existingLineage": {
"type": "object",
"description": "Existing lineage graph to merge with"
},
"targetCatalog": {
"type": "string",
"enum": ["datahub", "amundsen", "alation", "openlineage", "json"],
"default": "json",
"description": "Target format for lineage export"
},
"options": {
"type": "object",
"properties": {
"columnLevel": {
"type": "boolean",
"default": true,
"description": "Extract column-level lineage"
},
"resolveViews": {
"type": "boolean",
"default": false,
"description": "Resolve views to underlying tables"
},
"includeTemporary": {
"type": "boolean",
"default": false,
"description": "Include temporary/CTE tables in lineage"
}
}
}
}
{
"lineageGraph": {
"type": "object",
"properties": {
"nodes": {
"type": "array",
"items": {
"id": "string",
"type": "table|view|file|external",
"name": "string",
"database": "string",
"schema": "string",
"columns": "array"
}
},
"edges": {
"type": "array",
"items": {
"source": "string",
"target": "string",
"transformationType": "string",
"sql": "string"
}
}
}
},
"columnLineage": {
"type": "array",
"items": {
"targetColumn": {
"table": "string",
"column": "string"
},
"sourceColumns": {
"type": "array",
"items": {
"table": "string",
"column": "string",
"transformation": "string"
}
},
"transformationLogic": "string"
}
},
"impactAnalysis": {
"type": "object",
"properties": {
"upstream": {
"type": "array",
"description": "All upstream dependencies"
},
"downstream": {
"type": "array",
"description": "All downstream dependents"
},
"criticalPath": {
"type": "array",
"description": "Most important lineage path"
}
}
},
"catalogIntegration": {
"type": "object",
"description": "Export format for target catalog",
"properties": {
"format": "string",
"payload": "object|string"
}
},
"statistics": {
"tablesCount": "number",
"columnsCount": "number",
"edgesCount": "number",
"maxDepth": "number"
}
}
{
"sources": [
{
"type": "sql",
"content": "INSERT INTO analytics.fct_orders SELECT o.order_id, c.customer_name FROM staging.orders o JOIN staging.customers c ON o.customer_id = c.id",
"metadata": {
"database": "warehouse",
"schema": "analytics"
}
}
],
"options": {
"columnLevel": true
}
}
{
"sources": [
{
"type": "dbt",
"content": "./target/manifest.json"
}
],
"targetCatalog": "datahub",
"options": {
"resolveViews": true
}
}
{
"sources": [
{
"type": "dbt",
"content": "./analytics/target/manifest.json"
},
{
"type": "airflow",
"content": "./dags/etl_pipeline.py"
},
{
"type": "sql",
"content": "SELECT * FROM external_db.customers"
}
],
"targetCatalog": "openlineage"
}
{
"sources": [
{
"type": "dbt",
"content": "./target/manifest.json"
}
],
"options": {
"columnLevel": true,
"impactAnalysisTarget": "raw.customers"
}
}
| Statement Type | Extracted Information |
|---|---|
| SELECT | Source tables, column mappings |
| INSERT INTO...SELECT | Target table, source tables |
| CREATE TABLE AS | New table, source lineage |
| MERGE | Target, source, update/insert columns |
| UPDATE...FROM | Target table, source join tables |
Extracts from manifest.json:
ref() and source()catalog.jsonMaps lineage from:
Parses lineage from:
| Type | Example | Lineage |
|---|---|---|
| Direct | SELECT customer_id | 1:1 mapping |
| Rename | customer_id AS cust_id | Rename mapping |
| Expression | CONCAT(first, last) AS name | Multi-column → single |
| Aggregation | SUM(amount) AS total | Many → single with agg |
| Case | CASE WHEN... | Conditional mapping |
{
"columnLineage": [
{
"targetColumn": {
"table": "fct_orders",
"column": "customer_name"
},
"sourceColumns": [
{
"table": "stg_customers",
"column": "first_name",
"transformation": "CONCAT"
},
{
"table": "stg_customers",
"column": "last_name",
"transformation": "CONCAT"
}
],
"transformationLogic": "CONCAT(first_name, ' ', last_name)"
}
]
}
{
"format": "datahub",
"payload": {
"entities": [...],
"relationships": [...]
}
}
{
"format": "openlineage",
"payload": {
"eventType": "COMPLETE",
"run": {...},
"job": {...},
"inputs": [...],
"outputs": [...]
}
}
{
"format": "amundsen",
"payload": {
"tables": [...],
"columns": [...],
"lineage": [...]
}
}
data-lineage.js)data-catalog.js)dbt-project-setup.js)