Fast LLM fine-tuning with Unsloth - 2-5x faster training, 50-80% less VRAM. Use for single-GPU LoRA/QLoRA SFT, GRPO/RL reasoning training, vision/TTS fine-tuning, and GGUF export to Ollama/vLLM/llama.cpp. Supports 300+ models including Llama, Qwen, Gemma, DeepSeek, Mistral, Phi, and gpt-oss.
Standardmäßig ist der Prompt ausgewählt, der zuerst die Quelle prüft. Sie können zu einem direkten Befehl wechseln oder eine lokale Kopie herunterladen.
Quelldateien prüfen
Lesen Sie SKILL.md und alle von SkillsMP angezeigten Begleitdateien, bevor Sie sich für eine Installation entscheiden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
Fast LLM fine-tuning with Unsloth - 2-5x faster training, 50-80% less VRAM. Use for single-GPU LoRA/QLoRA SFT, GRPO/RL reasoning training, vision/TTS fine-tuning, and GGUF export to Ollama/vLLM/llama.cpp. Supports 300+ models including Llama, Qwen, Gemma, DeepSeek, Mistral, Phi, and gpt-oss.
Requirements: Linux or Windows (WSL), NVIDIA GPU with CUDA Capability 7.0+ (V100, T4, RTX 20-50, A100, H100, L40). AMD and Intel GPUs also supported. Python 3.10-3.13.
Workflow 1: SFT (Supervised Fine-Tuning)
Use this for standard instruction tuning, chat fine-tuning, or domain adaptation.
Checklist
Prepare dataset in ShareGPT, ChatML, or Alpaca format
Choose base vs instruct model (see Model Selection below)
Select QLoRA (4-bit) or LoRA (16-bit) based on VRAM
Set hyperparameters (rank, alpha, LR, epochs)
Run training with SFTTrainer
Save and deploy (LoRA adapter, merged 16-bit, or GGUF)
Implementation
from unsloth import FastLanguageModel
from trl import SFTTrainer, SFTConfig
from datasets import load_dataset
# Step 1: Load model (QLoRA 4-bit)
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Qwen3-8B-bnb-4bit", # or any HF model
max_seq_length=2048,
load_in_4bit=True, # False for LoRA 16-bit
)
# Step 2: Add LoRA adapters
model = FastLanguageModel.get_peft_model(
model,
r=16, # Rank: 8-128 (16-32 recommended)
lora_alpha=16, # Alpha: equal to r or 2*r
lora_dropout=0, # 0 is default, use 0.05-0.1 for regularization
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
use_gradient_checkpointing="unsloth", # 30% less VRAM
use_rslora=False, # True for rank-stabilized LoRA
)
# Step 3: Prepare dataset
dataset = load_dataset("philschmid/dolly-15k-oai-style", split="train")
# Step 4: Train
trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=dataset,
args=SFTConfig(
output_dir="./sft-output",
per_device_train_batch_size=2,
gradient_accumulation_steps=4, # Effective batch = 2*4 = 8
num_train_epochs=3,
learning_rate=2e-4,
fp16=True, # or bf16=True
logging_steps=10,
optim="adamw_8bit",
max_seq_length=2048,
packing=True, # Uncontaminated packing (2-5x faster)
),
)
trainer.train()
# Step 5: Save
model.save_pretrained("lora_adapter") # LoRA only (~6MB)
tokenizer.save_pretrained("lora_adapter")
Data Formats
Format
Template
Use Case
ShareGPT
{"conversations": [{"from": "human", ...}]}
Multi-turn chat, instruct models
ChatML / OpenAI
{"messages": [{"role": "user", ...}]}
OpenAI-compatible, instruct models
Alpaca
{"instruction": ..., "input": ..., "output": ...}
Single-turn tasks, base models
Raw text
Plain text corpus
Continued pretraining
Use get_chat_template(tokenizer, chat_template="chatml") to apply templates. Use standardize_sharegpt(dataset) for ShareGPT-formatted data with non-standard keys.
Training on Completions Only
Mask user inputs so loss is only computed on assistant responses:
from unsloth.chat_templates import train_on_responses_only
trainer = train_on_responses_only(
trainer,
instruction_part="<|start_header_id|>user<|end_header_id|>\n\n", # Llama 3.x
response_part="<|start_header_id|>assistant<|end_header_id|>\n\n",
)
# For Gemma: instruction_part="<start_of_turn>user\n", response_part="<start_of_turn>model\n"
Use this for training reasoning models with reward functions — math, code, format compliance, verifiable tasks.
Checklist
Define reward function(s) returning float scores
Choose model and enable vLLM fast inference
Enable Unsloth Standby for memory-efficient RL
Configure GRPOConfig with num_generations, epsilon, loss_type
Monitor reward curves and KL divergence
Save and export model
Implementation
import os
os.environ["UNSLOTH_VLLM_STANDBY"] = "1"# Memory-efficient RLfrom unsloth import FastLanguageModel
import torch
import re
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Qwen3-8B",
max_seq_length=2048,
load_in_4bit=True, # False for LoRA 16-bit
fast_inference=True, # Enable vLLM for fast generation
max_lora_rank=32,
gpu_memory_utilization=0.9, # Reduce if OOM
)
model = FastLanguageModel.get_peft_model(
model, r=32, lora_alpha=64,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
use_gradient_checkpointing="unsloth",
)
# Define reward functionsdefcorrectness_reward(completions, answer, **kwargs):
scores = []
for completion in completions:
match = re.search(r"<answer>(.*?)</answer>", completion, re.DOTALL)
extracted = match.group(1).strip() ifmatchelse""
scores.append(1.0if extracted == answer else0.0)
return scores
defformat_reward(completions, **kwargs):
pattern = r"<reasoning>.*?</reasoning>\s*<answer>.*?</answer>"return [1.0if re.search(pattern, c, re.DOTALL) else0.0for c in completions]
# Trainfrom trl import GRPOConfig, GRPOTrainer
training_args = GRPOConfig(
output_dir="./grpo-output",
per_device_train_batch_size=2,
gradient_accumulation_steps=4,
learning_rate=5e-6,
num_generations=8, # Rollouts per prompt
max_completion_length=512,
max_prompt_length=512,
max_steps=250,
temperature=1.0,
# RL algorithm variants
loss_type="dapo", # "grpo", "dr_grpo", "dapo", "bnpo"
epsilon=0.2,
epsilon_high=0.28, # DAPO upper clipping
scale_rewards="none", # Dr. GRPO: no reward scaling
optim="adamw_8bit",
report_to="none",
)
trainer = GRPOTrainer(
model=model,
processing_class=tokenizer,
args=training_args,
train_dataset=dataset,
reward_funcs=[correctness_reward, format_reward],
)
trainer.train()
# Save
model.save_lora("grpo_saved_lora")
RL Algorithm Variants
Algorithm
loss_type
Key Setting
Notes
GRPO
"grpo"
Default
Standard group relative policy optimization
Dr. GRPO
"dr_grpo"
scale_rewards="none"
No reward normalization, more stable
DAPO
"dapo"
epsilon_high=0.28
Two-sided clipping, recommended default
BNPO
"bnpo"
—
Bounded negative policy optimization
GSPO
any
importance_sampling_level="sequence"
Sequence-level importance weighting (Qwen team)
Unsloth Standby (Memory-Efficient RL)
Set os.environ["UNSLOTH_VLLM_STANDBY"] = "1" before imports. This shares vLLM's weight space with training and repurposes KV cache memory during training — saving up to 60% VRAM. On H100 80GB: 16GB shared weights + 64GB multi-purpose space.
Do not install alongside flash-attention in the same environment. Unsloth bundles xformers which may conflict with flash-attn on attention kernels. Use separate environments.
