Skip to main content
implementing-llms-litgpt Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.
الانتقال إلى التثبيت سوق المهارات اكتشف واستكشف مهارات الذكاء الاصطناعي التي بناها المجتمع.
المهن ذات الصلة SOC
استنادا إلى تصنيف SOC المهني
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
عرض تفاصيل Prompt يتجاوز الأمر المباشر Prompt المخصّص للمراجعة. افحص المصدر قبل تشغيله.
npx skills add https://github.com/davila7/claude-code-templates --skill implementing-llms-litgptيبقى الأمر في سطر واحد. مرّر أفقيًا لمراجعته كاملًا قبل النسخ.
تفضّل نسخة محلية؟ نزّل الملفات المتاحة حاليًا لدى SkillsMP.
تحميل Zip جاري التحميل... المزيد من هذا المستودع Build AI agents that interact with computers like humans do - viewing screens, moving cursors, clicking buttons, and typing text. Covers Anthropic's Computer Use, OpenAI's Operator/CUA, and open-source alternatives. Critical focus on sandboxing, security, and handling the unique challenges of vision-based control. Use when: computer use, desktop automation agent, screen control AI, vision-based agent, GUI automation.
Expert in building products that wrap AI APIs (OpenAI, Anthropic, etc.) into focused tools people will pay for. Not just 'ChatGPT but different' - products that solve specific problems with AI. Covers prompt engineering for products, cost management, rate limiting, and building defensible AI businesses. Use when: AI wrapper, GPT product, AI tool, wrap AI, AI SaaS.
github-workflow-automation Automate GitHub workflows with AI assistance. Includes PR reviews, issue triage, CI/CD integration, and Git operations. Use when automating GitHub workflows, setting up PR review automation, creating GitHub Actions, or triaging issues.
name implementing-llms-litgpt description Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers. version 1.0.0 author Orchestra Research license MIT tags ["Model Architecture","LitGPT","Lightning AI","LLM Implementation","LoRA","QLoRA","Fine-Tuning","Llama","Gemma","Phi","Mistral","Educational"] dependencies ["litgpt","torch","transformers"]
LitGPT - Clean LLM Implementations
Quick start
LitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.
pip install 'litgpt[extra]'
from litgpt import LLM
llm = LLM.load("microsoft/phi-2" )
result = llm.generate(
"What is the capital of France?" ,
max_new_tokens=50 ,
temperature=0.7
)
print (result)
Common workflows
Workflow 1: Fine-tune on custom dataset Fine-Tuning Setup:
- [ ] Step 1: Download pretrained model
- [ ] Step 2: Prepare dataset
- [ ] Step 3: Configure training
- [ ] Step 4: Run fine-tuning
Step 1: Download pretrained model
litgpt download meta-llama/Meta-Llama-3-8B
litgpt download microsoft/phi-2
litgpt download google/gemma-2b
Models are saved to checkpoints/ directory.
LitGPT supports multiple formats:
Alpaca format (instruction-response):
[
{
"instruction" : "What is the capital of France?" ,
"input" : "" ,
"output" : "The capital of France is Paris."
} ,
{
"instruction" : "Translate to Spanish: Hello, how are you?" ,
"input" : "" ,
"output" : "Hola, ¿cómo estás?"
}
]
Save as data/my_dataset.json.
Step 3: Configure training
litgpt finetune \
meta-llama/Meta-Llama-3-8B \
--data JSON \
--data.json_path data/my_dataset.json \
--train.max_steps 1000 \
--train.learning_rate 2e-5 \
--train.micro_batch_size 1 \
--train.global_batch_size 16
litgpt finetune_lora \
microsoft/phi-2 \
--data JSON \
--data.json_path data/my_dataset.json \
--lora_r 16 \
--lora_alpha 32 \
--lora_dropout 0.05 \
--train.max_steps 1000 \
--train.learning_rate 1e-4
Training saves checkpoints to out/finetune/ automatically.
tail -f out/finetune/logs.txt
tensorboard --logdir out/finetune/lightning_logs
Workflow 2: LoRA fine-tuning on single GPU Most memory-efficient option.
LoRA Training:
- [ ] Step 1: Choose base model
- [ ] Step 2: Configure LoRA parameters
- [ ] Step 3: Train with LoRA
- [ ] Step 4: Merge LoRA weights (optional)
Step 1: Choose base model
For limited GPU memory (12-16GB):
Phi-2 (2.7B) - Best quality/size tradeoffLlama 3 1B - Smallest, fastestGemma 2B - Good reasoning Step 2: Configure LoRA parameters
litgpt finetune_lora \
microsoft/phi-2 \
--data JSON \
--data.json_path data/my_dataset.json \
--lora_r 16 \
--lora_alpha 32 \
--lora_dropout 0.05 \
--lora_query true \
--lora_key false \
--lora_value true \
--lora_projection true \
--lora_mlp false \
--lora_head false
r=8: Lightweight, 2-4MB adaptersr=16: Standard, good qualityr=32: High capacity, use for complex tasksr=64: Maximum quality, 4× larger adapters litgpt finetune_lora \
microsoft/phi-2 \
--data JSON \
--data.json_path data/my_dataset.json \
--lora_r 16 \
--train.epochs 3 \
--train.learning_rate 1e-4 \
--train.micro_batch_size 4 \
--train.global_batch_size 32 \
--out_dir out/phi2-lora
Step 4: Merge LoRA weights (optional)
Merge LoRA adapters into base model for deployment:
litgpt merge_lora \
out/phi2-lora/final \
--out_dir out/phi2-merged
from litgpt import LLM
llm = LLM.load("out/phi2-merged" )
Workflow 3: Pretrain from scratch Train new model on your domain data.
Pretraining:
- [ ] Step 1: Prepare pretraining dataset
- [ ] Step 2: Configure model architecture
- [ ] Step 3: Set up multi-GPU training
- [ ] Step 4: Launch pretraining
Step 1: Prepare pretraining dataset
LitGPT expects tokenized data. Use prepare_dataset.py:
python scripts/prepare_dataset.py \
--source_path data/my_corpus.txt \
--checkpoint_dir checkpoints/tokenizer \
--destination_path data/pretrain \
--split train,val
Step 2: Configure model architecture
Edit config file or use existing:
model_name: pythia-160m
block_size: 2048
vocab_size: 50304
n_layer: 12
n_head: 12
n_embd: 768
rotary_percentage: 0.25
parallel_residual: true
bias: true
Step 3: Set up multi-GPU training
litgpt pretrain \
--config config/pythia-160m.yaml \
--data.data_dir data/pretrain \
--train.max_tokens 10_000_000_000
litgpt pretrain \
--config config/pythia-1b.yaml \
--data.data_dir data/pretrain \
--devices 8 \
--train.max_tokens 100_000_000_000
Step 4: Launch pretraining
For large-scale pretraining on cluster:
sbatch --nodes=8 --gpus-per-node=8 \
pretrain_script.sh
litgpt pretrain \
--config config/pythia-1b.yaml \
--data.data_dir /shared/data/pretrain \
--devices 8 \
--num_nodes 8 \
--train.global_batch_size 512 \
--train.max_tokens 300_000_000_000
Workflow 4: Convert and deploy model Export LitGPT models for production.
Model Deployment:
- [ ] Step 1: Test inference locally
- [ ] Step 2: Quantize model (optional)
- [ ] Step 3: Convert to GGUF (for llama.cpp)
- [ ] Step 4: Deploy with API
Step 1: Test inference locally
from litgpt import LLM
llm = LLM.load("out/phi2-lora/final" )
print (llm.generate("What is machine learning?" ))
for token in llm.generate("Explain quantum computing" , stream=True ):
print (token, end="" , flush=True )
prompts = ["Hello" , "Goodbye" , "Thank you" ]
results = [llm.generate(p) for p in prompts]
Step 2: Quantize model (optional)
Reduce model size with minimal quality loss:
litgpt convert_lit_checkpoint \
out/phi2-lora/final \
--dtype bfloat16 \
--quantize bnb.nf4
litgpt convert_lit_checkpoint \
out/phi2-lora/final \
--quantize bnb.nf4-dq
Step 3: Convert to GGUF (for llama.cpp)
python scripts/convert_lit_checkpoint.py \
--checkpoint_path out/phi2-lora/final \
--output_path models/phi2.gguf \
--model_name microsoft/phi-2
from fastapi import FastAPI
from litgpt import LLM
app = FastAPI()
llm = LLM.load("out/phi2-lora/final" )
@app.post("/generate"
def generate (prompt: str , max_tokens: int = 100 ):
result = llm.generate(
prompt,
max_new_tokens=max_tokens,
temperature=0.7
)
return {"response" : result}
When to use vs alternatives
Want to understand LLM architectures (clean, readable code)
Need production-ready training recipes
Educational purposes or research
Prototyping new model ideas
Lightning ecosystem user
Use alternatives instead:
Axolotl/TRL : More fine-tuning features, YAML configsMegatron-Core : Maximum performance for >70B modelsHuggingFace Transformers : Broadest model supportvLLM : Inference-only (no training)
Common issues Issue: Out of memory during fine-tuning
Use LoRA instead of full fine-tuning:
Or enable gradient checkpointing:
litgpt finetune_lora \
... \
--train.gradient_accumulation_iters 4
Enable Flash Attention (built-in, automatic on compatible hardware):
Use smaller micro-batch and accumulate:
--train.micro_batch_size 1 \
--train.global_batch_size 32 \
--train.gradient_accumulation_iters 32
litgpt download list
litgpt download meta-llama/Meta-Llama-3-8B
Verify checkpoints directory:
Issue: LoRA adapters too large
Apply LoRA to fewer layers:
--lora_query true \
--lora_value true \
--lora_projection false \
--lora_mlp false
Advanced topics
Hardware requirements
GPU : NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)Memory :
Inference (Phi-2): 6GB
LoRA fine-tuning (7B): 16GB
Full fine-tuning (7B): 40GB+
Pretraining (1B): 24GB
Storage : 5-50GB per model (depending on size)
Resources 