Skip to main content

eda-engineering

EDA engineering practices

来源信息

仓库
ffsshhttiikk/opencode-agents-skills
最近来源活动
2026年2月28日 22:51
检测到的 SKILL.md 语言
英语
星标
3
Forks
2

安装方式

默认使用会先检查来源的 Prompt;你也可以切换为直接命令,或下载本地副本。

检查来源文件

决定是否安装前,请先阅读 SKILL.md,以及 SkillsMP 当前展示的配套文件。

正在显示 SKILL.md

SKILL.md
来源说明 · 只读预览
name
eda/engineering
description
EDA engineering practices
license
MIT
compatibility
opencode
metadata
{"audience":"data-engineers","category":"data-science"}
## What I do - Build scalable EDA pipelines - Automate data profiling - Create reproducible data analysis workflows - Implement data quality monitoring - Design data exploration tools - Build self-service analytics infrastructure ## When to use me Use me when: - Building data infrastructure - Automating EDA for large datasets - Creating data quality dashboards - Implementing data governance - Building analytics platforms ## Key Concepts ### Data Profiling Pipeline ```python from great_expectations import GreatExpectations import pandas as pd # Define expectations expectations = [ {"expectation_type": "expect_column_values_to_not_be_null", "kwargs": {"column": "id"}}, {"expectation_type": "expect_column_values_to_be_unique", "kwargs": {"column": "id"}}, {"expectation_type": "expect_column_values_to_be_between", "kwargs": {"column": "age", "min_value": 0, "max_value": 120}}, {"expectation_type": "expect_column_distinct_values_to_be_in_set", "kwargs": {"column": "status", "value_set": ["active", "inactive"]}} ] # Validate data ge = GreatExpectations() batch = ge.get_batch("data.csv", "default") results = ge.validate(batch, expectations=expectations) ``` ### Automated Profiling ```python from pandas_profiling import ProfileReport # Generate comprehensive EDA report profile = ProfileReport( df, title="Data Profiling Report", explorative=True, missing_diagrams=True, correlations={"high_cardinality": "include"} ) profile.to_file("eda_report.html") ``` ### Data Quality Metrics - **Completeness**: % non-null values - **Uniqueness**: % unique values - **Validity**: % values in valid range - **Consistency**: Cross-field validation - **Timeliness**: Data freshness ### Monitoring Setup - Great Expectations for validation - Datafold for diff analysis - Great Expectations Cloud - Amundsen for data catalog
在 GitHub 查看