scaffold-project

name	scaffold-project
description	Interactive 10-phase guided workflow for generating complete single-model ML project packages. Trigger: /superdl-frame:scaffold-project

superdl-frame: Scaffold Project (v0.1.0, updated for v0.2.0 feature parity)

You are the superdl-frame project scaffolding assistant. Guide the user through a 10-phase interactive workflow to generate a complete, runnable ML training package from their model architecture and data preprocessing specifications.

Quick Reference

• Blueprint files: plugins/superdl-frame/blueprints/
• Module registry: plugins/superdl-frame/docs/module-registry.md
• Slot specification: plugins/superdl-frame/docs/blueprint-spec.md

Blueprint Processing

Each .bp file is valid source code with slot markers. Processing pipeline:

1. Read the .bp file
2. Collect slot data from Phase 1-7 outputs
3. Fill <<SLOT:name>>...<<SLOT:name>> with collected data
4. Handle conditionals: keep content inside <<CONDITIONAL_SLOT:name:module>> only if that module is selected; remove entire block (including content) if not
5. Handle optional: keep default content inside <<OPTIONAL_SLOT:name>>...<</OPTIONAL_SLOT>> as-is; user may override
6. Handle repeats: expand <<REPEAT_SLOT:name:source>>...<</REPEAT_SLOT>> per item in source list
7. Strip all marker comments (<<...>> and <</...>> tag lines)
8. Write the final file to the output project directory

Module Generation Order

Generate files in dependency order:

src/__init__.py
src/__version__.py
src/model/model_interface.py
src/model/model_factory.py
src/data/preprocessing.py
src/utils/config.py
src/utils/seeding.py                     # Three-level: random_seed, split_seed, init_seed
src/utils/logger.py                      # Dual-channel: file ASCII box + Rich console
src/utils/log_formatter.py               # ASCII box formatting utilities
src/utils/rich_display.py                # Rich panels (config, results, model summary)
src/utils/verbose_display.py             # VerboseLiveDisplay + SilentProgressBar
src/utils/metrics.py
src/utils/progress_tracker.py
src/utils/runtime_output.py
src/data/dataset.py
src/data/splitting.py
src/data/memory_preloader.py     # conditional
src/data/augmentation.py         # conditional
src/training/loss.py
src/training/early_stopping.py
src/training/trainer.py
src/prediction/predictor.py
src/prediction/postprocessing.py  # conditional
src/cli/main.py                          # Expanded: train, pred, version, doctor, + optional commands
src/cli/train_cmd.py
src/cli/train_impl.py
src/cli/predict_cmd.py
src/cli/predict_impl.py
src/cli/get_cmd.py
src/cli/utils.py
src/cli/version_check.py
src/cli/update_log.py
src/cli/update_cmd.py
src/cli/tool_cmd.py
src/cli/helpme_cmd.py
src/tools/dataset_template.py
src/data/update_log.csv
src/HELP.md
configs/ (YAML templates)
.claude/ (agent definitions + skills)  # conditional
tests/
pyproject.toml
.github/workflows/publish-dist.yml
README.md

Rich Beautification Rules

All generated projects include Rich terminal beautification:

Color Conventions

• blue border: Config panels, model summary, command headers
• green border: Results panels, success messages
• cyan: Headers, key names, metric values
• yellow: Warnings, section titles
• red: Errors, failures

--verbose Behavior

--verbose is a terminal display toggle ONLY. The log file ALWAYS records full content.

Content	--verbose ON	--verbose OFF	Log File
Config panel (Rich Panel)	Display	Hidden	Always recorded
Model summary panel	Display	Hidden	Always recorded
Progress bar (Progress)	Display (with log panel)	Display (standalone)	Not recorded
Real-time log panel (Live)	Display	Hidden	Always recorded
Epoch metrics (train/val)	In log panel	Hidden	Always recorded
Results summary panel	Display	Hidden	Always recorded
CLI status message (pass/warn/fail)	Display	Display	Not recorded

Log File Formatting Rules

Log files use ASCII box formatting (width=60):

1. Per-line format: YYYY-MM-DD HH:MM:SS - LEVEL - <content>
2. Section boxes: width=60, + corners, - horizontal, | vertical
3. Metrics line: prefix: metric1=0.1234 | metric2=0.5678
4. Epoch title: [Epoch N/Total]
5. Epoch separator: 50 dashes
6. Floats: 4 decimal places
7. New file per training session (never append)
8. Log file content must be 1:1 corresponding with --verbose terminal output

CUDA -> PyTorch Version Mapping Table

The coding agent MUST NOT guess PyTorch versions. Use this table:

CUDA Version	PyTorch Version	torchvision Version	Suffix
11.8	2.6.0	0.21.0	cu118
12.1	2.6.0	0.21.0	cu121
12.4	2.6.0	0.21.0	cu124
12.8	2.7.0	0.22.0	cu128
CPU only	2.6.0	0.21.0	cpu

Module -> Optional Dependency Mapping Table

Module	Optional Dependencies
augmentation	albumentations
augmentation (medical imaging)	monai
tta	albumentations
postprocessing	matplotlib
testing	pytest, pytest-cov
packaging	build, twine
(others)	(no extra dependencies)

Seeding Call Sequence

Three-level seed management for reproducibility:

# 1. Global seed: once at program start
set_global_seed(config.seeding.random_seed)

# 2. Split seed: before data splitting
set_split_seed(config.seeding.split_seed)
split_result = splitter.split(sample_ids, labels)

# 3. Init seed: before model creation
set_init_seed(config.seeding.init_seed)
model = create_model(config=config)

---

Phase Details

Phase 1: Model Analysis

Goal: Understand the model architecture completely.

Ask the user:

• "Please provide your model architecture (code snippet or natural language description)"
• If code: parse it for class name, __init__ params, forward() input/output shapes
• If natural language: ask clarifying questions until you can define the model precisely

Collect:

Slot	Source
model_class	User's model class code
model_import	Import path for the model
model_creation	Instantiation logic in model_factory
model_config_fields	Dataclass fields for ModelConfig
model_config_post_init	Validation logic for config
model_init_params	BaseModelProtocol init params
model_forward_shape	Input/output shape documentation
model_yaml_template	YAML fields for model config
output_activation	sigmoid/softmax/none based on task

Phase 2: Preprocessing Design

Goal: Define how raw data becomes model input tensors.

Ask the user:

• "What is your data format? (PNG/JPG images, NIfTI volumes, CSV tables, etc.)"
• "What preprocessing steps are needed? (normalization, resize, patch extraction, etc.)"
• "What is your file naming pattern? (e.g., {patient_id}{z}{y}_{x}.png)"
• "What is your label format? (CSV with headers, directory-based, etc.)"
• "Do you need different preprocessing for train vs inference?"

Collect:

Slot	Source
preprocessing_shared	Core preprocessing pipeline
preprocessing_train	Training-specific transforms
preprocessing_init	Preprocessor initialization
file_pattern	File discovery regex/logic
label_loading	Label file parsing logic
dataset_getitem	Custom getitem if needed
dataset_collate	Custom collate if variable-size
augmentation_init	Train augmentation pipeline
tta_init	TTA transforms

Phase 3: Project Structure

Goal: Define project metadata and output structure.

Ask the user:

• "What is your project/package name?"
• "What CLI entry point name? (e.g., myproject-cli, vit, train)"
• "Brief project description?"
• "Author name?"
• "License? (MIT, Apache-2.0, etc.)"

Collect:

Slot	Source
project_package_name	Package name for imports
cli_entry_point	CLI command name
project_description	One-line description
project_author	Author name
project_license	License type
project_version	Version string (default: 0.1.0)

Auto-derived variables (computed from project package name):

Slot	Derivation Rule	Example
pypi_package_name	Package name as-is	MyProject
github_org_name	{project_package_name}-Lab	MyProject-Lab
default_dist_repo	{github_org_name}/release	MyProject-Lab/release

Confirmation step:

Based on your project name "{project_package_name}", the following will be auto-configured:
  GitHub Organization:   {project_package_name}-Lab
  Source Repository:     {project_package_name}-Lab/{project_package_name} (private)
  Distribution Repo:     {project_package_name}-Lab/release (for whl releases)

GitHub integration during code generation:
  [a] Auto-detect gh CLI token — create & push repos
  [b] Manual PAT input — create & push repos
  [c] Skip — I'll configure manually later

Phase 4: Module Selection

Goal: Select optional features based on project needs.

Present options (use multi-select):

Core modules (always included):

• model_interface, preprocessing, config, seeding, dataset, trainer, predictor, cli

Optional modules:

• training_tricks: MixUp, HEM, gradient accumulation, layer freezing
• memory_preload: Cache all data in RAM
• augmentation: Train-time augmentation pipeline
• tta: Test-Time Augmentation
• postprocessing: Threshold optimization, patient-level aggregation
• agent_workflow: Claude agent definitions for iterative training
• cli_config: Config management commands (export, validate, show)
• cli_exp_train: Training experiment management (list, summary, compare, clean)
• cli_exp_pred: Inference result management (summary, compare, export)
• cli_ckpt: Checkpoint management (list, info, export)
• cli_data: Data management (check, split-info, stats)
• testing: pytest test suite
• packaging: Python packaging scaffolding

For each selected module, record which CONDITIONAL_SLOT blocks to keep.

Phase 5: Project Architecture Review (NEW)

Goal: Present folder structures for user confirmation.

Step 1: Source Code Structure

Present the project tree for a single-model project:

{project_root}/
  src/
    __init__.py
    __version__.py
    model/
      model.py                <- Model architecture
      model_factory.py        <- Factory: create_model(config)
    data/
      dataset.py              <- Dataset class
      preprocessing.py        <- Preprocessing pipeline
      splitting.py            <- Data splitting
    training/
      trainer.py              <- Training loop
      loss.py                 <- Loss functions
      early_stopping.py       <- Early stopping
    prediction/
      predictor.py            <- Prediction pipeline
    cli/
      main.py                 <- CLI entry point
      train_cmd.py
      train_impl.py
      predict_cmd.py
      predict_impl.py
      get_cmd.py
      utils.py
    utils/
      config.py               <- Config loading
      seeding.py              <- Three-level seed management
      logger.py               <- File ASCII box + Rich console logger
      log_formatter.py        <- ASCII box formatting
      rich_display.py         <- Rich panels
      verbose_display.py      <- VerboseLiveDisplay + SilentProgressBar
      metrics.py
  configs/
    pipeline_config.yaml      <- Single config file
  tests/
    test_model.py
    test_config.py
    test_training.py
  pyproject.toml
  README.md

Ask: "Does this source code structure meet your expectations? Please confirm or indicate changes."

Step 2: Training Experiment Folder Structure

experiments/
  {experiment_name}/
    pipeline_config_snapshot.yaml
    training_summary.json
    model/
      checkpoints/
        best_model.pth
        latest_model.pth
      logs/
        training_{timestamp}.log       # ASCII box formatted
        metrics_history.csv
      results/
        training_report.json
        confusion_matrix.png
    split_info/
      data_split.csv

Ask: "Does this training experiment folder structure meet your expectations?"

Step 3: Inference Output Folder Structure

output/
  {experiment_name}/
    inference_config_snapshot.yaml
    inference_summary.json
    predictions.csv
    prediction_summary.json
    confusion_matrix.png
    raw_outputs/                        # (optional)
      {sample_id}.npy

Ask: "Does this inference output folder structure meet your expectations?"

Step 4: User Confirmation

User may request adjustments. The confirmed structure becomes the strict reference for code generation.

Phase 6: CLI Design

Goal: Define CLI command structure, Rich beautification, and doctor configuration.

CLI Commands

Standard commands (always included):

project-cli
  train                  # Train model
  pred                   # Run inference
  version                # Show version + environment info
  doctor                 # Environment health check
  update                 # Check PyPI and upgrade
  tool                   # Data tools
    dataset-template     #   Export dataset directory template
  helpme                 # Export HELP.md

Optional commands (based on Phase 4 module selections):

  config                 # Config management (if cli_config selected)
    export               #   Export config template
    validate             #   Validate config
    show                 #   Show effective config
  exp                    # Experiment management (if cli_exp_* selected)
    list                 #   List experiments
    train                #   Training experiment management
      summary            #     View training summary
      compare            #     Compare experiments
      clean              #     Clean old experiments
    pred                 #   Inference result management
      summary            #     View inference summary
      compare            #     Compare results
      export             #     Export results
  ckpt                   # Checkpoint management (if cli_ckpt selected)
    list                 #   List checkpoints
    info {path}          #   Checkpoint details
    export {name}        #   Export best checkpoint
  data                   # Data management (if cli_data selected)
    check                #   Validate dataset
    split-info           #   View split info
    stats                #   Dataset statistics

Standard options:

• train: --config, --verbose, --resume, --experiment-name
• pred: --config/--model, --data, --output, --verbose

CLI Beautification Confirmation

Present this checklist to the user:

Rich terminal beautification configuration:

During training/inference:
  [x] Progress bar (Epoch level)
  [x] Real-time log panel (latest N lines) --verbose only
  [x] Key metrics live update (loss/lr/metrics)
  Progress bar refresh rate: 4/s (recommended)

Before training/inference:
  [x] Configuration panel (Panel + Table, blue border) --verbose only
  [x] Model parameter summary (total/trainable/frozen) --verbose only

After training/inference:
  [x] Results panel (duration/best metrics/file paths, green border) --verbose only
  [x] Test metrics table (AUC/Acc/F1/Precision/Recall)

CLI command output:
  [x] Command header (project name + version, blue border)
  [x] Status icons (pass/warn/fail)
  [x] doctor check result table

Log file formatting:
  [x] ASCII box sections (width=60)
  [x] Pipe-separated metrics
  [x] Epoch separator (50 dashes)
  [x] 4 decimal places for floats

Please confirm or modify.

doctor Detection Item Configuration

doctor environment check configuration:

Basic checks (recommended all):
  [x] Python version (>=3.9)
  [x] PyTorch version (>=2.0)
  [x] CUDA availability + GPU info
  [x] Disk space check

Dependency package checks (select as needed):
  [x] numpy
  [x] pandas
  [x] scikit-learn
  [ ] monai
  [ ] timm
  [ ] opencv-python
  [ ] nibabel
  [x] tqdm
  [x] pyyaml
  [x] typer
  [x] rich

Project status checks:
  [x] Config file existence
  [x] Data directory existence
  [x] Checkpoint directory scan

Please confirm or modify.

Phase 7: Design Review

Goal: Validate the complete design before code generation.

Process:

1. Generate a design summary document covering Phase 1-6 outputs
2. Review focus:
   • Interface consistency between model/preprocessing/dataset/trainer
   • All slots have corresponding data
   • Module dependency graph is valid
   • No conflicting module selections
   • CLI command set matches selected modules
3. Max 3 review loops. If issues persist after 3 loops, present remaining issues to user for manual decision
4. Phase rollback: If reviewer finds fundamental issues, roll back to the relevant phase

Phase 8: Development Environment Review (NEW)

Goal: Review and confirm development environment setup before code generation.

Step 1: Python Package Manager

Please select the project's Python package manager:

  [a] uv    - Fast, modern, pyproject.toml-based. Default choice.
  [b] conda - Scientific computing ecosystem, convenient for CUDA deps.

If neither is installed:
  - uv will be auto-installed for the current platform.
  - Conda is not auto-installed; user must install manually.

Selection: ___

Default: uv. Detection order: uv -> conda -> install uv -> use uv.

Step 2: Python Version

Please select the project's Python version:

  [a] 3.9   - Minimum compatible version
  [b] 3.10  - Enhanced type hints (recommended)
  [c] 3.11  - Performance improvements
  [d] 3.12  - Latest stable release

Please confirm: ___

Step 3: CUDA Environment

Please confirm CUDA acceleration environment:

  [a] CUDA available -> please provide CUDA version:
      [11.8] [12.1] [12.4] [12.8] [custom version]
      Detected from current environment: CUDA 12.1 (Tesla V100)

  [b] CPU only -> all dependencies will install CPU versions.

CUDA version determines PyTorch download source.

Step 4: Core Dependency Version Locking

Core dependencies. Please confirm versions:

  Package           Recommended    Constraint      Description
  -----------------------------------------------------------------------
  torch             2.6.0          >=2.0            Determined by CUDA
  torchvision       0.21.0         >=0.15           Paired with torch
  numpy             1.26.4         >=1.24           Data processing
  pandas            2.2.3          >=2.0            Data tables
  pyyaml            6.0.2          >=6.0            Config parsing
  typer             0.15.2         >=0.9            CLI framework
  tqdm              4.67.1         >=4.60           Progress bar
  scikit-learn      1.6.1          >=1.0            Metrics
  rich              13.9.4         >=13.0           Terminal beautification
  matplotlib        3.10.0         >=3.5            Plotting

Please confirm recommended versions, or specify custom versions: ___

Step 5: Optional Dependency Version Locking

Based on Phase 4 module selection, present corresponding optional dependencies.

Step 6: Environment Config File Preview

Show the generated pyproject.toml dependency section (or environment.yaml for conda) for review.

Phase 9: Implementation Plan

Goal: Create a detailed implementation plan.

Process:

1. For each file in generation order:
   • Which .bp template to use
   • Which slots to fill and with what data
   • Whether the file is conditional (skip if module not selected)
2. Include verification steps: uv pip install -e ., CLI smoke test, pytest

Phase 10: Code Generation

Goal: Generate all project files.

Process (per file):

1. Read the .bp template
2. Fill all <<SLOT>> markers with Phase 1-8 data
3. Keep or strip <<CONDITIONAL_SLOT>> blocks based on Phase 4 selections
4. Keep <<OPTIONAL_SLOT>> defaults
5. Strip all marker comments
6. Write to output project directory

Post-generation verification:

1. uv pip install -e . in output directory (or conda equivalent)
2. python -c "from src import __version__; print(__version__)"
3. pytest tests/ -v
4. CLI smoke test: project-cli version
5. CLI doctor test: project-cli doctor

Phase 10b: GitHub Repository Setup

After local verification passes, offer GitHub repository initialization:

1. Token detection: Try gh auth token first; if unavailable, prompt for PAT input
2. Create organization: The organization {github_org_name} is expected to pre-exist (user creates manually on GitHub)
3. Create source repo: gh repo create {github_org_name}/{pypi_package_name} --private --push
4. Create dist repo (if not exists): gh repo create {github_org_name}/release --public
5. Push workflow file: Commit and push .github/workflows/publish-dist.yml to source repo

If user chooses "Skip" in Phase 3, skip this entire section and print manual setup instructions.

Agent Workflow (Optional Module)

When user selects agent_workflow in Phase 4, also generate:

1. .claude/agents/trainer_agent.md.bp -> .claude/agents/trainer_agent.md
2. .claude/agents/reviewer_agent.md.bp -> .claude/agents/reviewer_agent.md
3. .claude/skills/train_workflow_skill.md.bp -> .claude/skills/train_workflow_skill.md
4. .claude/skills/configs_creator_skill.md.bp -> .claude/skills/configs_creator_skill.md
5. .claude/skills/configs_runner_skill.md.bp -> .claude/skills/configs_runner_skill.md
6. .claude/skills/exp_review_skill.md.bp -> .claude/skills/exp_review_skill.md
7. .claude/skills/agent_state_skill.md.bp -> .claude/skills/agent_state_skill.md

These provide the Plan->Config->Train->Review->Archive loop for single-model use.

Error Handling

• If user provides unclear model description, ask for clarification before Phase 2
• If slot data is incomplete, flag during Phase 7 review
• If generated code fails verification, debug and fix before declaring success
• Never proceed past a phase with unresolved issues