data-science-python

Estrelas8

Forks3

Atualizado1 de abril de 2026 às 23:54

Python data science: notebook structure, data validation, reproducibility, and model documentation

Instalação

Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.

Fonte

ucdavis

ucdavis/ai-skills-registry

Abrir repositório GitHub Ver repositórios do creator

Download

Ocupações relacionadasSOC

Baseado na classificação ocupacional SOC

Cientistas de dadosInformática e Matemática·SOC 15-2051

SKILL.md

readonly

name	data-science-python
description	Python data science: notebook structure, data validation, reproducibility, and model documentation

Data Science — Python

Notebook Structure

Organize notebooks in this order:

Imports & configuration
Data loading
Exploratory data analysis (EDA)
Feature engineering
Modeling
Evaluation
Conclusions & next steps

Keep notebooks for exploration. Move reusable logic to src/ Python modules with tests.

Data Validation

# Always validate after loading
assert df.shape[0] > 0, "DataFrame is empty"
assert df.isnull().sum().sum() == 0, f"Nulls found: {df.isnull().sum()}"
assert df['price'].between(0, 1_000_000).all(), "Price out of expected range"

# For production pipelines: use pandera
import pandera as pa
schema = pa.DataFrameSchema({
    "price": pa.Column(float, pa.Check.ge(0)),
    "category": pa.Column(str, pa.Check.isin(["A", "B", "C"])),
})
schema.validate(df)

Reproducibility

import random
import numpy as np

SEED = 42
random.seed(SEED)
np.random.seed(SEED)
# sklearn: pass random_state=SEED to all estimators

Pin all dependency versions in requirements.txt or pyproject.toml.
Track experiments: MLflow, or a simple experiments/ log with metadata JSON.
Save models with timestamp + metadata: model_rf_20260115_v1.pkl.

Model Documentation

Document for every model:

Training data: source, date range, size, preprocessing steps.
Feature list and engineering decisions.
Hyperparameters and tuning approach.
Evaluation metrics on held-out test set.
Known limitations and failure modes.

Code Quality

# Extract transforms into sklearn Pipeline
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('classifier', RandomForestClassifier(random_state=SEED)),
])

Write unit tests for data processing functions in tests/.
Use pathlib.Path — never hardcode file paths.
Use logging not print in production code.

Mais deste repositório

mesmo repositório

architecture-general

ucdavis/ai-skills-registry

Architecture decisions: layering, design patterns, microservices trade-offs, and domain-driven design

2026-04-018

architecture-python

ucdavis/ai-skills-registry

Python architecture: PEP-8, type hints, clean layering, dependency injection, and testing strategy

2026-04-018

code-review-general

ucdavis/ai-skills-registry

General code review process: priority ordering, what to block on, how to give actionable feedback

2026-04-018

git-workflow-general

ucdavis/ai-skills-registry

Git workflow: branch naming, conventional commits, PR conventions, and merge strategies

2026-04-018

refactoring-typescript

ucdavis/ai-skills-registry

TypeScript refactoring: SOLID principles, DRY, type safety improvements, and eliminating code smells

2026-04-018

security-general

ucdavis/ai-skills-registry

Security checklist: OWASP top 10, secret scanning, input validation, and auth patterns

2026-04-018

name	data-science-python
description	Python data science: notebook structure, data validation, reproducibility, and model documentation

Data Science — Python

Notebook Structure

Organize notebooks in this order:

Imports & configuration
Data loading
Exploratory data analysis (EDA)
Feature engineering
Modeling
Evaluation
Conclusions & next steps

Keep notebooks for exploration. Move reusable logic to src/ Python modules with tests.

Data Validation

# Always validate after loading
assert df.shape[0] > 0, "DataFrame is empty"
assert df.isnull().sum().sum() == 0, f"Nulls found: {df.isnull().sum()}"
assert df['price'].between(0, 1_000_000).all(), "Price out of expected range"

# For production pipelines: use pandera
import pandera as pa
schema = pa.DataFrameSchema({
    "price": pa.Column(float, pa.Check.ge(0)),
    "category": pa.Column(str, pa.Check.isin(["A", "B", "C"])),
})
schema.validate(df)

Reproducibility

import random
import numpy as np

SEED = 42
random.seed(SEED)
np.random.seed(SEED)
# sklearn: pass random_state=SEED to all estimators

Pin all dependency versions in requirements.txt or pyproject.toml.
Track experiments: MLflow, or a simple experiments/ log with metadata JSON.
Save models with timestamp + metadata: model_rf_20260115_v1.pkl.

Model Documentation

Document for every model:

Training data: source, date range, size, preprocessing steps.
Feature list and engineering decisions.
Hyperparameters and tuning approach.
Evaluation metrics on held-out test set.
Known limitations and failure modes.

Code Quality

# Extract transforms into sklearn Pipeline
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('classifier', RandomForestClassifier(random_state=SEED)),
])

Write unit tests for data processing functions in tests/.
Use pathlib.Path — never hardcode file paths.
Use logging not print in production code.