Skip to main content Início Criadores synthetic-sciences openscience implementing-llms-litgpt
implementing-llms-litgpt Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.
Ir para a instalação Skills Marketplace Descubra e explore skills de IA criadas pela comunidade.
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Mostrar detalhes do prompt Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.
npx skills add https://github.com/synthetic-sciences/openscience --skill implementing-llms-litgptO comando permanece em uma só linha. Role horizontalmente para revisá-lo antes de copiar.
Prefere uma cópia local? Baixe os arquivos disponíveis atualmente no SkillsMP.
Baixar Zip Baixando... Ocupações relacionadas SOC
Baseado na classificação ocupacional SOC
Explorador de arquivos
5 arquivos name implementing-llms-litgpt description Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers. category ml-training version 1.0.0 author Synthetic Sciences license MIT tags ["Model Architecture","LitGPT","Lightning AI","LLM Implementation","LoRA","QLoRA","Fine-Tuning","Llama","Gemma","Phi","Mistral","Educational"] dependencies ["litgpt","torch","transformers"]
LitGPT - Clean LLM Implementations
Quick start
LitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.
Installation :
pip install 'litgpt[extra]'
Load and use any model :
from litgpt import LLM
llm = LLM.load("microsoft/phi-2" )
result = llm.generate(
"What is the capital of France?" ,
max_new_tokens=50 ,
temperature=0.7
)
print (result)
List available models :
litgpt download list
Common workflows
Workflow 1: Fine-tune on custom dataset
Copy this checklist:
Fine-Tuning Setup:
- [ ] Step 1: Download pretrained model
- [ ] Step 2: Prepare dataset
- [ ] Step 3: Configure training
- [ ] Step 4: Run fine-tuning
Step 1: Download pretrained model
litgpt download meta-llama/Meta-Llama-3-8B
litgpt download microsoft/phi-2
litgpt download google/gemma-2b
Models are saved to checkpoints/ directory.
Step 2: Prepare dataset
LitGPT supports multiple formats:
Alpaca format (instruction-response):
[
{
"instruction" : "What is the capital of France?" ,
"input" : "" ,
"output" : "The capital of France is Paris."
}
,
{
"instruction"
:
"Translate to Spanish: Hello, how are you?"
,
"input"
:
""
,
"output"
:
"Hola, ¿cómo estás?"
}
]
Save as data/my_dataset.json.
Step 3: Configure training
litgpt finetune \
meta-llama/Meta-Llama-3-8B \
--data JSON \
--data.json_path data/my_dataset.json \
--train.max_steps 1000 \
--train.learning_rate 2e-5 \
--train.micro_batch_size 1 \
--train.global_batch_size 16
litgpt finetune_lora \
microsoft/phi-2 \
--data JSON \
--data.json_path data/my_dataset.json \
--lora_r 16 \
--lora_alpha 32 \
--lora_dropout 0.05 \
--train.max_steps 1000 \
--train.learning_rate 1e-4
Training saves checkpoints to out/finetune/ automatically.
tail -f out/finetune/logs.txt
tensorboard --logdir out/finetune/lightning_logs
Workflow 2: LoRA fine-tuning on single GPU Most memory-efficient option.
LoRA Training:
- [ ] Step 1: Choose base model
- [ ] Step 2: Configure LoRA parameters
- [ ] Step 3: Train with LoRA
- [ ] Step 4: Merge LoRA weights (optional)
Step 1: Choose base model
For limited GPU memory (12-16GB):
Phi-2 (2.7B) - Best quality/size tradeoffLlama 3 1B - Smallest, fastestGemma 2B - Good reasoning Step 2: Configure LoRA parameters
litgpt finetune_lora \
microsoft/phi-2 \
--data JSON \
--data.json_path data/my_dataset.json \
--lora_r 16 \
--lora_alpha 32 \
--lora_dropout 0.05 \
--lora_query true \
--lora_key false \
--lora_value true \
--lora_projection true \
--lora_mlp false \
--lora_head false
r=8: Lightweight, 2-4MB adaptersr=16: Standard, good qualityr=32: High capacity, use for complex tasksr=64: Maximum quality, 4× larger adapters litgpt finetune_lora \
microsoft/phi-2 \
--data JSON \
--data.json_path data/my_dataset.json \
--lora_r 16 \
--train.epochs 3 \
--train.learning_rate 1e-4 \
--train.micro_batch_size 4 \
--train.global_batch_size 32 \
--out_dir out/phi2-lora
Step 4: Merge LoRA weights (optional)
Merge LoRA adapters into base model for deployment:
litgpt merge_lora \
out/phi2-lora/final \
--out_dir out/phi2-merged
from litgpt import LLM
llm = LLM.load("out/phi2-merged" )
Workflow 3: Pretrain from scratch Train new model on your domain data.
Pretraining:
- [ ] Step 1: Prepare pretraining dataset
- [ ] Step 2: Configure model architecture
- [ ] Step 3: Set up multi-GPU training
- [ ] Step 4: Launch pretraining
Step 1: Prepare pretraining dataset
LitGPT expects tokenized data. Use prepare_dataset.py:
python scripts/prepare_dataset.py \
--source_path data/my_corpus.txt \
--checkpoint_dir checkpoints/tokenizer \
--destination_path data/pretrain \
--split train,val
Step 2: Configure model architecture
Edit config file or use existing:
model_name: pythia-160m
block_size: 2048
vocab_size: 50304
n_layer: 12
n_head: 12
n_embd: 768
rotary_percentage: 0.25
parallel_residual: true
bias: true
Step 3: Set up multi-GPU training
litgpt pretrain \
--config config/pythia-160m.yaml \
--data.data_dir data/pretrain \
--train.max_tokens 10_000_000_000
litgpt pretrain \
--config config/pythia-1b.yaml \
--data.data_dir data/pretrain \
--devices 8 \
--train.max_tokens 100_000_000_000
Step 4: Launch pretraining
For large-scale pretraining on cluster:
sbatch --nodes=8 --gpus-per-node=8 \
pretrain_script.sh
litgpt pretrain \
--config config/pythia-1b.yaml \
--data.data_dir /shared/data/pretrain \
--devices 8 \
--num_nodes 8 \
--train.global_batch_size 512 \
--train.max_tokens 300_000_000_000
Workflow 4: Convert and deploy model Export LitGPT models for production.
Model Deployment:
- [ ] Step 1: Test inference locally
- [ ] Step 2: Quantize model (optional)
- [ ] Step 3: Convert to GGUF (for llama.cpp)
- [ ] Step 4: Deploy with API
Step 1: Test inference locally
from litgpt import LLM
llm = LLM.load("out/phi2-lora/final" )
print (llm.generate("What is machine learning?" ))
for token in llm.generate("Explain quantum computing" , stream=True ):
print (token, end="" , flush=True )
prompts = ["Hello" , "Goodbye" , "Thank you" ]
results = [llm.generate(p) for p in prompts]
Step 2: Quantize model (optional)
Reduce model size with minimal quality loss:
litgpt convert_lit_checkpoint \
out/phi2-lora/final \
--dtype bfloat16 \
--quantize bnb.nf4
litgpt convert_lit_checkpoint \
out/phi2-lora/final \
--quantize bnb.nf4-dq
Step 3: Convert to GGUF (for llama.cpp)
python scripts/convert_lit_checkpoint.py \
--checkpoint_path out/phi2-lora/final \
--output_path models/phi2.gguf \
--model_name microsoft/phi-2
from fastapi import FastAPI
from litgpt import LLM
app = FastAPI()
llm = LLM.load("out/phi2-lora/final" )
@app.post("/generate"
def generate (prompt: str , max_tokens: int = 100 ):
result = llm.generate(
prompt,
max_new_tokens=max_tokens,
temperature=0.7
)
return {"response" : result}
When to use vs alternatives
Want to understand LLM architectures (clean, readable code)
Need production-ready training recipes
Educational purposes or research
Prototyping new model ideas
Lightning ecosystem user
Use alternatives instead:
Axolotl/TRL : More fine-tuning features, YAML configsMegatron-Core : Maximum performance for >70B modelsHuggingFace Transformers : Broadest model supportvLLM : Inference-only (no training)
Common issues Issue: Out of memory during fine-tuning
Use LoRA instead of full fine-tuning:
Or enable gradient checkpointing:
litgpt finetune_lora \
... \
--train.gradient_accumulation_iters 4
Enable Flash Attention (built-in, automatic on compatible hardware):
Use smaller micro-batch and accumulate:
--train.micro_batch_size 1 \
--train.global_batch_size 32 \
--train.gradient_accumulation_iters 32
litgpt download list
litgpt download meta-llama/Meta-Llama-3-8B
Verify checkpoints directory:
Issue: LoRA adapters too large
Apply LoRA to fewer layers:
--lora_query true \
--lora_value true \
--lora_projection false \
--lora_mlp false
Advanced topics
Hardware requirements
GPU : NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)Memory :
Inference (Phi-2): 6GB
LoRA fine-tuning (7B): 16GB
Full fine-tuning (7B): 40GB+
Pretraining (1B): 24GB
Storage : 5-50GB per model (depending on size)
Resources 