Run any Skill in Manus with one click

pyspark-coding

Stars0

Forks0

UpdatedApril 23, 2026 at 09:43

Use this skill when writing PySpark code. Provides PySpark best practices for manageability, testability, and performance including DataFrame patterns, caching strategies, testing approaches, and deployment configurations.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

imarios

imarios/open-vibes

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Software DevelopersComputer and Mathematical Occupations·SOC 15-1252

File Explorer

4 files

SKILL.md

readonly

name	pyspark-coding
description	Use this skill when writing PySpark code. Provides PySpark best practices for manageability, testability, and performance including DataFrame patterns, caching strategies, testing approaches, and deployment configurations.
version	1.0.0

PySpark Coding Guidelines

Comprehensive PySpark development practices for manageability, testability, and high performance.

When to Use This Skill

Use this skill when:

Writing PySpark data processing jobs
Optimizing PySpark performance and reducing compute costs
Testing PySpark transformations and jobs
Deciding between DataFrame API and RDD API
Configuring Spark for local development or cluster deployment
Implementing caching and persistence strategies

Reference Routing Table

Reference	Read when you need to…
`fundamentals.md`	Start a new PySpark project — DataFrame vs RDD decision, code structure, testability patterns, SparkSession management
`performance.md`	Optimize PySpark jobs — caching/persistence strategies, join optimization, partitioning, file format choices, I/O patterns
`testing-and-deployment.md`	Set up PySpark testing — local dev vs cluster configs, parallelism tuning, executor settings, deployment practices

More from this repository

same repository

rest-api

imarios/open-vibes

Use this skill when designing REST APIs. Provides REST API design patterns including URL structure, multi-tenancy, request/response formats, HTTP methods, authorization, and testing requirements.

2026-04-230

splunk-skill

imarios/open-vibes

Use this skill when working with Splunk queries, Search Processing Language (SPL), building Splunk searches, creating dashboards, or analyzing log data. This skill provides comprehensive reference material for SPL commands, eval functions, stats functions, regex patterns, and query optimization techniques.

2026-04-210

docker-dev

imarios/open-vibes

Use this skill when setting up Docker and Docker Compose for local development. Provides development-specific patterns for Dockerfiles and Docker Compose including live reload, volume mounting, and rapid iteration workflows.

2026-04-210

fastapi-coding

imarios/open-vibes

Use this skill when building FastAPI applications. Provides FastAPI-specific best practices including project structure, routing with Pydantic models, async database patterns, dependency injection, security, and testing.

2026-04-210

kubernetes

imarios/open-vibes

Use this skill when working with Kubernetes — writing manifests, managing clusters, debugging workloads, setting up observability, or deploying to EKS. Covers local dev, Helm, kubectl, k9s, Kustomize, OpenTelemetry, Grafana stack, and CI/CD with GitHub Actions.

2026-04-210

terraform-security-review

imarios/open-vibes

End-to-end Terraform and OpenTofu security review with static analysis, policy-as-code, secrets scanning, drift detection, and CI/CD gates. Use when hardening IaC, writing OPA/Rego or Sentinel policies, or setting up Trivy/Checkov PR pipelines.

2026-04-210

name	pyspark-coding
description	Use this skill when writing PySpark code. Provides PySpark best practices for manageability, testability, and performance including DataFrame patterns, caching strategies, testing approaches, and deployment configurations.
version	1.0.0

PySpark Coding Guidelines

Comprehensive PySpark development practices for manageability, testability, and high performance.

When to Use This Skill

Use this skill when:

Writing PySpark data processing jobs
Optimizing PySpark performance and reducing compute costs
Testing PySpark transformations and jobs
Deciding between DataFrame API and RDD API
Configuring Spark for local development or cluster deployment
Implementing caching and persistence strategies

Reference Routing Table

Reference	Read when you need to…
`fundamentals.md`	Start a new PySpark project — DataFrame vs RDD decision, code structure, testability patterns, SparkSession management
`performance.md`	Optimize PySpark jobs — caching/persistence strategies, join optimization, partitioning, file format choices, I/O patterns
`testing-and-deployment.md`	Set up PySpark testing — local dev vs cluster configs, parallelism tuning, executor settings, deployment practices