메뉴
Write and optimize PySpark and Spark SQL. Use for Spark DataFrames, joins, windows, UDF decisions, partitioning, caching, AQE, explain plans, OOM, shuffle, skew, Delta writes, and Spark pipeline performance.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | spark-development |
| description | Write and optimize PySpark and Spark SQL. Use for Spark DataFrames, joins, windows, UDF decisions, partitioning, caching, AQE, explain plans, OOM, shuffle, skew, Delta writes, and Spark pipeline performance. |
Use this skill for Spark implementation choices and performance risk checks.
explain("formatted") or platform equivalent.collect(), toPandas(), or repartition(1) on large data.inferSchema=True in production..count() actions just for logging.Before reporting Spark work complete:
Build and maintain an internal technical LLM wiki when the user requests wiki work or a verified major or architectural change qualifies for automatic wiki review. Use for project memory, architecture notes, contracts, runbooks, decisions, glossary, incremental ingest, delta status, and wiki health. Do not use merely because a routine task or plan completed.
Explore ambiguous ideas and solution directions before evidence-backed research or implementation planning. Use for ideation, framing, assumption checks, option generation, product or architecture trade-offs, and deciding whether research or a plan is needed.
Research technical options with source-backed recommendations. Use for technology evaluation, architecture trade-offs, library/framework choices, best practices, scalability, security, maintainability, or when current external facts matter.
Create implementation plans, architecture decisions, and phased roadmaps. Use for Standard or Complex changes, new pipelines, schema changes, migrations, multi-file features, technology choices, or when acceptance criteria and verification strategy are unclear.
Review code and data pipeline changes for correctness, security, performance, maintainability, contract breaks, and missing verification. Use before merge, after implementation, for PRs, commits, pending diffs, or codebase risk scans.
Design data ingestion into landing/bronze layers. Use for source onboarding, transfer pattern selection, full/incremental/CDC scope, change detection, landing zones, schema drift, idempotent file or batch processing, and ingestion quality gates.