Use this Skill to make a research project fully reproducible: conda/renv environment locking, DVC data versioning, Dockerfile for analysis, and FAIR data principles.
Standardmäßig ist der Prompt ausgewählt, der zuerst die Quelle prüft. Sie können zu einem direkten Befehl wechseln oder eine lokale Kopie herunterladen.
Quelldateien prüfen
Lesen Sie SKILL.md und alle von SkillsMP angezeigten Begleitdateien, bevor Sie sich für eine Installation entscheiden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
Use this Skill to make a research project fully reproducible: conda/renv environment locking, DVC data versioning, Dockerfile for analysis, and FAIR data principles.
TL;DR — Make a research project fully reproducible end-to-end: lock Python
environments with conda, version large data files with DVC, containerize the
analysis with Docker, protect secrets with .env files, and satisfy FAIR
data principles for long-term reusability.
When to Use This Skill
Use this Skill when you need to:
Share analysis code with collaborators or reviewers who must reproduce your results
Submit to a journal that requires data and code availability
Set up a continuous integration (CI) pipeline that re-runs your analysis
Archive a project at the time of paper submission
Comply with funder open-science mandates (NIH, Wellcome Trust, Horizon Europe)
Task
Tool
Environment locking (Python)
conda + environment.yml
Environment locking (R)
renv + renv.lock
Large file / data versioning
DVC
Containerized execution
Docker
Secrets management
.env file (gitignored)
FAIR compliance
Metadata, DOI, open license
Background & Key Concepts
The Reproducibility Crisis
A 2016 Nature survey found that > 70% of researchers failed to reproduce another
scientist's results. The leading causes are:
Missing or undocumented software dependencies
Unavailable or unversioned raw data
Undisclosed analysis deviations
Platform-specific code that fails on different OS/hardware
FAIR Principles
Letter
Meaning
Practical action
Findable
Persistent identifier, rich metadata
Assign a DOI via Zenodo
Accessible
Open protocol, authentication if needed
Host on GitHub + Zenodo
Interoperable
Standard formats, vocabularies
Use CSV/JSON/HDF5, Dublin Core metadata
Reusable
Clear license, provenance documented
Add LICENSE file, README with methods
Conda vs pip vs Docker
conda: best for scientific Python stacks with compiled dependencies (NumPy, CUDA)
pip + venv: lighter, good for pure-Python projects
Docker: full OS-level isolation; best for exact reproducibility across machines
The recommended stack: conda for local dev, Docker for CI and archival.
Environment Setup
# Install conda (if not already installed)# https://docs.conda.io/en/latest/miniconda.html# Install DVC
pip install "dvc[s3,gs,azure,ssh]>=3.0"# Install Docker Desktop (see https://www.docker.com/products/docker-desktop)
docker --version # verify# Clone your project (or init a new one)
git init my-project
cd my-project
dvc init
git add .dvc .gitignore
git commit -m "Initialize DVC"
Core Workflow
Step 1 — Conda Environment Locking
Create a precise environment.yml with pinned versions, then generate a
fully-resolved lock file for bit-for-bit reproducibility.
# Create the project environment
conda create -n myproject python=3.11 -y
conda activate myproject
# Install packages
conda install -c conda-forge pandas=2.0.3 matplotlib=3.8.0 scikit-learn=1.3.2 -y
pip install dvc>=3.0 pyyaml>=6.0
# Export the human-readable spec
conda envexport --no-builds > environment.yml
# Export a fully-pinned cross-platform lock file (requires conda-lock)
pip install conda-lock
conda-lock lock --file environment.yml --platform linux-64 --platform osx-arm64
# Produces: conda-lock.yml# Reproduce the environment from the lock file on any machine
conda-lock install --name myproject conda-lock.yml
DVC tracks large files outside git, runs pipelines reproducibly, and stores
data in a remote storage backend (S3, GCS, SSH, or local).
# dvc_setup.py — programmatic DVC configurationimport subprocess
import os
import yaml
from pathlib import Path
definit_dvc_project(project_root: str = ".") -> None:
"""
Initialize a DVC project and configure a local remote.
Args:
project_root: Path to the project root directory.
"""
root = Path(project_root)
# Initialize DVC (idempotent)
subprocess.run(["dvc", "init"], cwd=root, check=True)
# Configure a local remote (change to S3/GCS URI in production)
remote_path = root / ".dvc_remote"
remote_path.mkdir(exist_ok=True)
subprocess.run(
["dvc", "remote", "add", "-d", "local_remote", str(remote_path)],
cwd=root, check=True,
)
print(f"DVC initialized with local remote at {remote_path}")
deftrack_data_file(file_path: str) -> None:
"""
Add a data file to DVC tracking.
The file is replaced by a small .dvc pointer file that is committed to git.
Args:
file_path: Relative path to the data file inside the project.
"""
subprocess.run(["dvc", "add", file_path], check=True)
dvc_file = file_path + ".dvc"
subprocess.run(["git", "add", dvc_file, ".gitignore"], check=True)
print(f"Tracked {file_path} with DVC. Commit {dvc_file} to git.")
defcreate_dvc_pipeline(stages: list[dict], output_path: str = "dvc.yaml") -> None:
"""
Write a DVC pipeline YAML file from a list of stage definitions.
Args:
stages: List of dicts, each with keys:
name (str), cmd (str), deps (list), outs (list),
params (list, optional).
output_path: Path to write dvc.yaml.
Example stage:
{
"name": "preprocess",
"cmd": "python src/preprocess.py",
"deps": ["src/preprocess.py", "data/raw.csv"],
"outs": ["data/processed.csv"],
"params": ["params.yaml:preprocessing"],
}
"""
pipeline: dict = {"stages": {}}
for stage in stages:
entry: dict = {"cmd": stage["cmd"], "deps": stage.get("deps", []),
"outs": stage.get("outs", [])}
if"params"in stage:
entry["params"] = stage["params"]
pipeline["stages"][stage["name"]] = entry
withopen(output_path, "w") as f:
yaml.dump(pipeline, f, default_flow_style=False, sort_keys=False)
print(f"DVC pipeline written to {output_path}")
# ── Example usage ─────────────────────────────────────────────────────────────if __name__ == "__main__":
stages = [
{
"name": "preprocess",
"cmd": "python src/preprocess.py",
"deps": ["src/preprocess.py", "data/raw.csv"],
"outs": ["data/processed.csv"],
"params": ["params.yaml:preprocessing"],
},
{
"name": "train",
"cmd": "python src/train.py",
"deps": ["src/train.py", "data/processed.csv"],
"outs": ["models/model.pkl"],
"params": ["params.yaml:training"],
},
{
"name": "evaluate",
"cmd": "python src/evaluate.py",
"deps": ["src/evaluate.py", "models/model.pkl", "data/processed.csv"],
"outs": ["reports/metrics.json"],
},
]
create_dvc_pipeline(stages)
# Then run: dvc repro (executes only changed stages)# Or: dvc dag (visualize the pipeline DAG)
Step 3 — Dockerfile for Reproducible Analysis
# Dockerfile — place in project root
# Build: docker build -t myproject:1.0 .
# Run: docker run --rm -v $(pwd)/data:/app/data myproject:1.0
FROM jupyter/scipy-notebook:python-3.11
# Switch to root to install system dependencies
USER root
RUN apt-get update && apt-get install -y --no-install-recommends \
git \
curl \
&& rm -rf /var/lib/apt/lists/*
# Back to the default non-root user
USER ${NB_UID}
# Copy and install Python dependencies
COPY --chown=${NB_UID}:${NB_GID} environment.yml /tmp/environment.yml
RUN conda env update --name base --file /tmp/environment.yml && \
conda clean --all -f -y
# Install DVC
RUN pip install --no-cache-dir "dvc[s3]>=3.0"
# Copy project source code (not data — mount data volume at runtime)
WORKDIR /app
COPY --chown=${NB_UID}:${NB_GID} src/ /app/src/
COPY --chown=${NB_UID}:${NB_GID} params.yaml /app/params.yaml
COPY --chown=${NB_UID}:${NB_GID} dvc.yaml /app/dvc.yaml
# Default command: reproduce the DVC pipeline
CMD ["dvc", "repro"]
Build and run helpers:
# Build the image
docker build -t myproject:1.0 .
# Run the full analysis pipeline
docker run --rm \
-v "$(pwd)/data:/app/data" \
-v "$(pwd)/models:/app/models" \
-v "$(pwd)/reports:/app/reports" \
myproject:1.0
# Interactive shell for debugging
docker run --rm -it \
-v "$(pwd)/data:/app/data" \
myproject:1.0 /bin/bash
Advanced Usage
.env File Pattern for Secrets
Never commit API keys or passwords to git. Use a .env file that is listed
in .gitignore, and load it with python-dotenv or os.getenv.
# src/config.pyimport os
from pathlib import Path
# Load .env file if it exists (development convenience)try:
from dotenv import load_dotenv
load_dotenv(Path(__file__).parent.parent / ".env")
except ImportError:
pass# In Docker, variables are injected via --env-file or -e flags# export ZENODO_TOKEN="<paste-your-key>"
ZENODO_TOKEN: str = os.getenv("ZENODO_TOKEN", "")
# export S3_BUCKET="<your-bucket-name>"
S3_BUCKET: str = os.getenv("S3_BUCKET", "")
# export OPENAI_API_KEY="<paste-your-key>"
OPENAI_API_KEY: str = os.getenv("OPENAI_API_KEY", "")
defrequire_env(name: str) -> str:
"""Raise a clear error if a required environment variable is missing."""
val = os.getenv(name)
ifnot val:
raise EnvironmentError(
f"Required environment variable '{name}' is not set.\n"f"Add it to your .env file: {name}=<value>"
)
return val
# In R console — one-time setup# install.packages("renv")# renv::init() # creates renv.lock, .Rprofile, renv/# After installing packages# renv::snapshot() # update renv.lock with current package state# On a new machine or fresh clone:# renv::restore() # install exact package versions from renv.lock
ORCID Integration for Author Disambiguation
Add ORCID identifiers to your README and metadata files:
defgenerate_citation_cff(authors: list[dict], title: str, doi: str) -> str:
"""
Generate a CITATION.cff file content for GitHub citation metadata.
Args:
authors: List of dicts with keys: name (str), orcid (str, optional).
title: Repository/paper title.
doi: DOI of the archived version.
Returns:
CITATION.cff file content as a string.
"""import yaml
cff = {
"cff-version": "1.2.0",
"message": "If you use this software, please cite it using the metadata below.",
"title": title,
"doi": doi,
"authors": [
{"name": a["name"], **({"orcid": f"https://orcid.org/{a['orcid']}"} if"orcid"in a else {})}
for a in authors
],
"license": "MIT",
"date-released": "2026-03-17",
}
return yaml.dump(cff, default_flow_style=False, allow_unicode=True)
