Fast LLM fine-tuning with Unsloth - 2-5x faster training, 50-80% less VRAM. Use for single-GPU LoRA/QLoRA SFT, GRPO/RL reasoning training, vision/TTS fine-tuning, and GGUF export to Ollama/vLLM/llama.cpp. Supports 300+ models including Llama, Qwen, Gemma, DeepSeek, Mistral, Phi, and gpt-oss.
Instrucciones de origen · Vista previa de solo lectura
name
unsloth-fine-tuning
description
Fast LLM fine-tuning with Unsloth - 2-5x faster training, 50-80% less VRAM. Use for single-GPU LoRA/QLoRA SFT, GRPO/RL reasoning training, vision/TTS fine-tuning, and GGUF export to Ollama/vLLM/llama.cpp. Supports 300+ models including Llama, Qwen, Gemma, DeepSeek, Mistral, Phi, and gpt-oss.
Requirements: Linux or Windows (WSL), NVIDIA GPU with CUDA Capability 7.0+ (V100, T4, RTX 20-50, A100, H100, L40). AMD and Intel GPUs also supported. Python 3.10-3.13.
Workflow 1: SFT (Supervised Fine-Tuning)
Use this for standard instruction tuning, chat fine-tuning, or domain adaptation.
Checklist
Prepare dataset in ShareGPT, ChatML, or Alpaca format
Choose base vs instruct model (see Model Selection below)
Select QLoRA (4-bit) or LoRA (16-bit) based on VRAM
Set hyperparameters (rank, alpha, LR, epochs)
Run training with SFTTrainer
Save and deploy (LoRA adapter, merged 16-bit, or GGUF)
Implementation
from unsloth import FastLanguageModel
from trl import SFTTrainer, SFTConfig
from datasets import load_dataset
# Step 1: Load model (QLoRA 4-bit)
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Qwen3-8B-bnb-4bit", # or any HF model
max_seq_length=2048,
load_in_4bit=True, # False for LoRA 16-bit
)
# Step 2: Add LoRA adapters
model = FastLanguageModel.get_peft_model(
model,
r=16, # Rank: 8-128 (16-32 recommended)
lora_alpha=16, # Alpha: equal to r or 2*r
lora_dropout=0, # 0 is default, use 0.05-0.1 for regularization
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
use_gradient_checkpointing="unsloth", # 30% less VRAM
use_rslora=False, # True for rank-stabilized LoRA
)
# Step 3: Prepare dataset
dataset = load_dataset("philschmid/dolly-15k-oai-style", split="train")
# Step 4: Train
trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=dataset,
args=SFTConfig(
output_dir="./sft-output",
per_device_train_batch_size=2,
gradient_accumulation_steps=4, # Effective batch = 2*4 = 8
num_train_epochs=3,
learning_rate=2e-4,
fp16=True, # or bf16=True
logging_steps=10,
optim="adamw_8bit",
max_seq_length=2048,
packing=True, # Uncontaminated packing (2-5x faster)
),
)
trainer.train()
# Step 5: Save
model.save_pretrained("lora_adapter") # LoRA only (~6MB)
tokenizer.save_pretrained("lora_adapter")
Data Formats
Format
Template
Use Case
ShareGPT
{"conversations": [{"from": "human", ...}]}
Multi-turn chat, instruct models
ChatML / OpenAI
{"messages": [{"role": "user", ...}]}
OpenAI-compatible, instruct models
Alpaca
{"instruction": ..., "input": ..., "output": ...}
Single-turn tasks, base models
Raw text
Plain text corpus
Continued pretraining
Use get_chat_template(tokenizer, chat_template="chatml") to apply templates. Use standardize_sharegpt(dataset) for ShareGPT-formatted data with non-standard keys.
Training on Completions Only
Mask user inputs so loss is only computed on assistant responses:
from unsloth.chat_templates import train_on_responses_only
trainer = train_on_responses_only(
trainer,
instruction_part="<|start_header_id|>user<|end_header_id|>\n\n", # Llama 3.x
response_part="<|start_header_id|>assistant<|end_header_id|>\n\n",
)
# For Gemma: instruction_part="<start_of_turn>user\n", response_part="<start_of_turn>model\n"
Use this for training reasoning models with reward functions — math, code, format compliance, verifiable tasks.
Checklist
Define reward function(s) returning float scores
Choose model and enable vLLM fast inference
Enable Unsloth Standby for memory-efficient RL
Configure GRPOConfig with num_generations, epsilon, loss_type
Monitor reward curves and KL divergence
Save and export model
Implementation
import os
os.environ["UNSLOTH_VLLM_STANDBY"] = "1"# Memory-efficient RLfrom unsloth import FastLanguageModel
import torch
import re
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Qwen3-8B",
max_seq_length=2048,
load_in_4bit=True, # False for LoRA 16-bit
fast_inference=True, # Enable vLLM for fast generation
max_lora_rank=32,
gpu_memory_utilization=0.9, # Reduce if OOM
)
model = FastLanguageModel.get_peft_model(
model, r=32, lora_alpha=64,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
use_gradient_checkpointing="unsloth",
)
# Define reward functionsdefcorrectness_reward(completions, answer, **kwargs):
scores = []
for completion in completions:
match = re.search(r"<answer>(.*?)</answer>", completion, re.DOTALL)
extracted = match.group(1).strip() ifmatchelse""
scores.append(1.0if extracted == answer else0.0)
return scores
defformat_reward(completions, **kwargs):
pattern = r"<reasoning>.*?</reasoning>\s*<answer>.*?</answer>"return [1.0if re.search(pattern, c, re.DOTALL) else0.0for c in completions]
# Trainfrom trl import GRPOConfig, GRPOTrainer
training_args = GRPOConfig(
output_dir="./grpo-output",
per_device_train_batch_size=2,
gradient_accumulation_steps=4,
learning_rate=5e-6,
num_generations=8, # Rollouts per prompt
max_completion_length=512,
max_prompt_length=512,
max_steps=250,
temperature=1.0,
# RL algorithm variants
loss_type="dapo", # "grpo", "dr_grpo", "dapo", "bnpo"
epsilon=0.2,
epsilon_high=0.28, # DAPO upper clipping
scale_rewards="none", # Dr. GRPO: no reward scaling
optim="adamw_8bit",
report_to="none",
)
trainer = GRPOTrainer(
model=model,
processing_class=tokenizer,
args=training_args,
train_dataset=dataset,
reward_funcs=[correctness_reward, format_reward],
)
trainer.train()
# Save
model.save_lora("grpo_saved_lora")
RL Algorithm Variants
Algorithm
loss_type
Key Setting
Notes
GRPO
"grpo"
Default
Standard group relative policy optimization
Dr. GRPO
"dr_grpo"
scale_rewards="none"
No reward normalization, more stable
DAPO
"dapo"
epsilon_high=0.28
Two-sided clipping, recommended default
BNPO
"bnpo"
—
Bounded negative policy optimization
GSPO
any
importance_sampling_level="sequence"
Sequence-level importance weighting (Qwen team)
Unsloth Standby (Memory-Efficient RL)
Set os.environ["UNSLOTH_VLLM_STANDBY"] = "1" before imports. This shares vLLM's weight space with training and repurposes KV cache memory during training — saving up to 60% VRAM. On H100 80GB: 16GB shared weights + 64GB multi-purpose space.
Do not install alongside flash-attention in the same environment. Unsloth bundles xformers which may conflict with flash-attn on attention kernels. Use separate environments.
