scientific-codebase-distillation

name	scientific-codebase-distillation
description	Use when a user asks you to study their research codebase + papers and become capable of independent work in their domain. Systematically explore source code, configs, job scripts, and publications to extract reusable patterns, coding conventions, and workflow knowledge.
version	2.0.0
license	MIT

Scientific Codebase Methodology Distillation

Overview

Distill a scientific computing project into reusable methodology. The process: study a domain expert's research stack — papers, multi-language source code, configuration files, job submission scripts, plotting conventions — and extract the patterns and workflow knowledge needed to independently extend the research to new problems.

The goal is to learn the methodology, not copy the code. Architecture decisions, data flow, naming conventions, config patterns, and quality standards are what make research reproducible and extensible.

When to Use

• "Study my code and learn to work independently in my domain"
• Onboarding into a new research project that has both code and publications
• Multi-language scientific codebase (e.g., C++/Python/scripts/configs)
• The codebase has an associated paper, technical report, or analysis note

Don't use for:

• Simple code review or bug fixes
• Single-file scripts with no surrounding context
• Codebases without research context

The Distillation Pipeline

Phase 1: Access & Structural Survey

Verify connectivity (SSH, file permissions, environment). Save connection details to memory once — you will reuse them across sessions.

Map the directory tree top-down without reading file contents yet:

ls -la /path/to/project
find /path -maxdepth 2 -type d | sort
find /path -name '*.py'  | wc -l
find /path -name '*.cpp' -o -name '*.cc' -o -name '*.h' | wc -l
find /path -name '*.yaml' -o -name '*.yml' | wc -l
find /path -name '*.sh'  | wc -l

You want a mental model of what lives where, before reading any single file.

Phase 2: Read the Paper First

Read the paper or technical report before the code. The paper provides:

• Problem motivation and definitions
• The "headline" methodology (what the code implements)
• Expected results and precision targets

Use pdftotext or PyMuPDF/fitz if available. Extract abstract, methodology, and results sections.

Without the paper, you will misunderstand why the code makes specific choices.

Phase 3: Read Existing Methodology Docs

If a SKILL.md, README, or design document already exists, read it completely before touching code. It tells you what the user thinks their methodology is. The gap between this and the actual code is where the most valuable distillation happens.

Also check for supplementary reference repos (group conventions, methodology indexes, internal wikis). They often hold conventions the codebase itself does not document.

Phase 4: Deep Code Exploration (Order Matters)

Read in this priority — highest signal-per-file first:

1. Config / template files (YAML, JSON) — reveal parameter space and conventions instantly
2. Main entry points (train.py, main(), example.cpp) — reveal the full pipeline
3. Core algorithm files — the actual training, inference, or extraction logic
4. Utility / plotting files — reveal style conventions
5. Job submission scripts (SLURM, PBS, K8s) — reveal compute patterns and resource requirements

For each file, ask:

• What convention does this establish? (naming, structure, config format)
• What pattern does this repeat elsewhere?
• What is reusable vs. project-specific?

Phase 5: Cross-Reference Paper ↔ Code

The paper claims X; the code does Y. Identify:

• Direct matches — paper method = code implementation
• Evolution — code uses newer tools than the paper
• Omissions — paper describes something the code does not implement
• Hidden infrastructure — code has components the paper does not mention (data prep, format converters, job orchestration)

Phase 6: Extract Methodology Patterns

Distill into concrete, teachable patterns:

Category	What to extract
Architecture	Language split, cross-language bridge (ONNX, protobuf, file handoff), module boundaries
Data flow	Raw → preprocess → model → inference → results; where are the boundaries?
Config conventions	File structure, variable naming, expression formats
Naming conventions	Variable names, output fields, file naming patterns
Training patterns	Data splits, seed management, hyperparameter scheduling
Evaluation patterns	Metrics computed, output formats, significance / uncertainty formulas
Plotting style	Library choice, style configs, label and color conventions
Job submission	Resource directives, environment setup, data staging

Phase 7: Write the Distilled Skill

Use templates/distilled-skill.md as a starting point. Recommended structure:

1. Problem context — what does this research solve?
2. Workflow diagram — full pipeline end-to-end
3. Architecture — language split, cross-language bridges
4. Directory convention — template for new projects
5. Phase-by-phase guide — each pipeline stage with exact commands and config snippets
6. Reuse guide — how to extend to new problems (what changes, what stays)
7. Plotting conventions — style configs, libraries, label formats
8. Common pitfalls — failure modes (silent / wrong / crash)

Keep code snippets at the pattern level — show the shape, not the full implementation. Reference source paths for the complete code.

Optimize the entire skill directory, not just the main document. Review templates/, references/, and scripts/ for files that should be updated based on what you learned from the codebase.

Phase 8: Save Persistent Memory

Save durable, cross-session facts:

• Server connection details
• Key directory paths
• Coding style preferences
• Naming conventions
• Tool choices (e.g., specific plotting libraries, custom style modules)

Do not save task progress or temporary state.

Common Pitfalls

1. Reading files in random order. Configs → entry points → algorithms → utilities maximizes information per file read.

2. Trusting the existing skill at face value. Existing documentation may describe an aspirational version, not what production code actually does. Always cross-reference.

3. Copying code instead of patterns. Reference paths to real code; do not duplicate it in the skill.

4. Missing the cross-language bridge. ONNX / protobuf / file handoff is often the most important architectural insight and the easiest to overlook.

5. Ignoring job scripts. They reveal the gap between "works locally" and "runs in production".

6. Over-saving to memory. Only save stable facts. Never save task state.

7. Reading code before the paper. Without the conceptual framework, you will misunderstand the why behind code choices.

Verification Checklist

• Paper abstract and methodology section read
• Full directory tree mapped (≥ 2 levels)
• All config formats identified, one example read per format
• Main entry points located for each pipeline stage
• Cross-language bridge identified (ONNX, protobuf, file handoff, etc.)
• Naming conventions extracted (variables, outputs, files)
• Training patterns understood (splits, seeds, scheduling)
• Plotting conventions extracted (libraries, styles, labels)
• Job submission patterns understood (resources, env, staging)
• Distilled skill written and saved to user's workspace
• Persistent memory saved (server info, conventions, tool choices)
• Reuse guide included (how to extend to new problems)

See Also

• templates/distilled-skill.md — boilerplate for the output skill
• references/example-distillation.md — a real distillation case (Chinese, HEP analysis)