// Generate a Python code skeleton from an experiment blueprint
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization
| name | nanoresearch-experiment |
| description | Generate a Python code skeleton from an experiment blueprint |
| version | 0.1.0 |
Take the experiment blueprint and produce a runnable Python code skeleton that implements the proposed method, baselines, training loops, evaluation harness, and ablation configurations.
None. This skill operates entirely through LLM code generation based on the experiment blueprint.
experiment_blueprint: Path to papers/experiment_blueprint.json produced by the planning skillProduces experiments/ directory containing:
data/: Data loading and preprocessing modulesmodels/: Model architecture implementations (proposed method and baselines)training/: Training loop and optimization utilitiesevaluation/: Metric computation and result aggregationconfigs/: YAML configuration files for each experiment and ablation variantrun.py: Main entry point for launching experimentsrequirements.txt: Python dependencies