원클릭으로 Manus에서 모든 스킬 실행

cloud-data-infra

스타0

포크0

업데이트2026년 7월 5일 11:17

Umbrella skill for where data pipelines run and what they cost — AWS/GCP/Azure infrastructure (S3/GCS/ADLS layout, BigQuery/Redshift/Snowflake selection, IAM, managed-service choice, performance tuning) and cost control (bytes scanned, partition pruning, slot/credit management, storage tiering, budgets and alerts). Use this whenever the topic is cloud data infrastructure or the cloud data bill. This skill ROUTES to the focused sub-skills (cloud-infra-data, cost-optimization-data) and pulls in both when a task spans architecture and cost. Trigger on: deploying a pipeline to cloud, choosing between managed warehouses, bucket/partition layout, IAM for data access, "our BigQuery bill jumped", expensive queries, or slot/credit utilization.

설치

Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.

Manus에서 실행

출처

Methasit-Pun

Methasit-Pun/data_engineer_claude_skills

GitHub 저장소 열기 Creator 저장소 보기

다운로드

Manus에서 실행

관련 직업SOC

SOC 직업 분류 기준

소프트웨어 개발자컴퓨터 및 수학직·SOC 15-1252

SKILL.md

readonly

이 저장소의 다른 Skills

같은 저장소

data-lifecycle

Methasit-Pun/data_engineer_claude_skills

Umbrella skill for running a data project end-to-end through its lifecycle stages — discover sources → profile the data → architect the platform → build the medallion pipeline → refactor the code. Use this whenever the user is kicking off a new data project, asks "where do I start" or "what are the steps", or is somewhere mid-lifecycle and unsure which stage skill applies. This skill ROUTES to the stage sub-skills (data-sourcing, data-profiling, data-architecture, medallion-design, notebook-refactor) and sequences them, pulling in the right one for the user's current stage.

2026-07-050

data-sourcing

Methasit-Pun/data_engineer_claude_skills

Data collection and readiness — the step BEFORE a pipeline exists. Catalog candidate data sources for an objective, rank each by importance/impact, and record where to get it (internal system vs. open/public dataset), how to access it, its refresh cadence, licensing, and readiness blockers. Use this skill whenever a project is starting and the sources aren't decided yet, when the user asks "what data do I need for X and where do I get it", when weighing internal vs. open-source data, or when assessing whether a source is accessible/usable before committing to a build. This sits upstream of pipeline-design — it decides WHAT to ingest before deciding HOW.

2026-07-050

data-profiling

Methasit-Pun/data_engineer_claude_skills

Profile and map raw data BEFORE designing a schema. One-time exploratory analysis to learn the true shape of a dataset — row/column counts, null rates, cardinality, value distributions, ranges, data types, candidate keys, duplicates, referential relationships — then a source-to-target field mapping and an ER diagram. Use this skill whenever the user has data in hand and needs to understand it before modeling, is about to design a schema, asks "what does this data actually look like", needs to find the primary/composite key of an unfamiliar table, or must map source fields to a target model. This is exploratory and one-time; ongoing production validation is data-quality's job, and modeling patterns are schema-design's job.

2026-07-050

data-modeling

Methasit-Pun/data_engineer_claude_skills

Umbrella skill for shaping data once it has landed — warehouse schema design (star/snowflake/OBT/SCD/grain), SQL for analytics (window functions, CTEs, optimization), dbt model layers/tests/macros/incrementals, and Python transforms (pandas/Polars/PySpark performance). Use this whenever the user is designing tables, writing or reviewing analytical SQL, building dbt models, or writing Python data transformations and it isn't yet clear which tool dominates. This skill ROUTES to the focused sub-skills (schema-design, sql-patterns, dbt-patterns, python-data-patterns) and pulls in more than one when a task spans them. Trigger on: fact/dimension design, table grain, SCD, window functions, deduplication, dbt refs/materializations, or slow/memory-heavy DataFrame code.

2026-07-050

dbt-patterns

Methasit-Pun/data_engineer_claude_skills

dbt model design, ref chains, sources, tests, macros, incremental strategies, materializations, and documentation best practices. Use this skill whenever the user is writing or reviewing dbt models, configuring dbt tests, designing model layers (staging/intermediate/marts), asking about incremental models, choosing materializations, writing macros, setting up sources.yml, or troubleshooting dbt run/test failures. Also trigger when the user mentions dbt refs, lineage graphs, model dependencies, dbt Cloud, the dbt CLI, or when they want to transform data already in the warehouse using SQL. If the project uses dbt at all, this skill should be active for any transformation questions.

2026-07-050

python-data-patterns

Methasit-Pun/data_engineer_claude_skills

Pandas, Polars, and PySpark idioms for production data engineering — chunked reads, memory-safe transforms, vectorized operations, type optimization, and performance patterns. Use this skill whenever the user is writing a Python data transformation script and running into memory issues, slow performance, or correctness bugs with large datasets. Also trigger when the user asks how to handle large CSV/Parquet files, process data in batches, use Polars instead of Pandas, optimize a PySpark job, or reduce DataFrame memory usage. If you see someone iterating row-by-row over a DataFrame, this skill should trigger immediately.

2026-07-050

Cloud Data Infrastructure & Cost (Router)

This is a router skill. It groups the two skills covering cloud data infrastructure and its cost. Diagnose which sub-area(s) the task touches, then invoke the matching sub-skill(s) with the Skill tool.

How to route

If the task is about…

Invoke sub-skill

Provisioning and architecture: S3/GCS/ADLS partitioning, BigQuery slots, Redshift Spectrum, Snowflake warehouses, IAM/cross-account access, managed-service selection, cloud performance tuning

cloud-infra-data

The bill: query cost analysis, bytes scanned, partition pruning, slot reservation vs. on-demand, storage tiering, Snowflake credits, cost alerts and budgets

cost-optimization-data

Routing rules

These two almost always travel together. A "slow BigQuery query" is both a performance (cloud-infra-data) and a cost (cost-optimization-data) problem — invoke both.

"Our bill jumped" or a named expensive query → lead with cost-optimization-data (it starts by finding where the money goes), then cloud-infra-data for the structural fix.

Choosing/standing up a warehouse or storage layout → lead with cloud-infra-data, then cost-optimization-data to price the choice.

Invoke via the Skill tool by name, e.g. Skill(skill="cost-optimization-data"). Combine outputs; don't paraphrase from memory.

Related groups

Pipelines deployed onto this infra → [[data-pipelines]]

Query patterns that drive cost → [[data-modeling]]

Cloud Data Infrastructure & Cost (Router)

How to route

If the task is about…

Invoke sub-skill

Provisioning and architecture: S3/GCS/ADLS partitioning, BigQuery slots, Redshift Spectrum, Snowflake warehouses, IAM/cross-account access, managed-service selection, cloud performance tuning

cloud-infra-data

The bill: query cost analysis, bytes scanned, partition pruning, slot reservation vs. on-demand, storage tiering, Snowflake credits, cost alerts and budgets

cost-optimization-data

Routing rules

These two almost always travel together. A "slow BigQuery query" is both a performance (cloud-infra-data) and a cost (cost-optimization-data) problem — invoke both.

"Our bill jumped" or a named expensive query → lead with cost-optimization-data (it starts by finding where the money goes), then cloud-infra-data for the structural fix.

Choosing/standing up a warehouse or storage layout → lead with cloud-infra-data, then cost-optimization-data to price the choice.

Invoke via the Skill tool by name, e.g. Skill(skill="cost-optimization-data"). Combine outputs; don't paraphrase from memory.

Related groups

Pipelines deployed onto this infra → [[data-pipelines]]

Query patterns that drive cost → [[data-modeling]]