peft-fine-tuning

name	peft-fine-tuning
description	Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
version	1.0.0
author	Orchestra Research
license	MIT
dependencies	["peft>=0.13.0","transformers>=4.45.0","torch>=2.0.0","bitsandbytes>=0.43.0"]
platforms	["linux","macos","windows"]
metadata	{"sonic":{"tags":["Fine-Tuning","PEFT","LoRA","QLoRA","Parameter-Efficient","Adapters","Low-Rank","Memory Optimization","Multi-Adapter"]}}

PEFT (Parameter-Efficient Fine-Tuning)

Fine-tune LLMs by training <1% of parameters using LoRA, QLoRA, and 25+ adapter methods.

When to use PEFT

Use PEFT/LoRA when:

• Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100)
• Need to train <1% parameters (6MB adapters vs 14GB full model)
• Want fast iteration with multiple task-specific adapters
• Deploying multiple fine-tuned variants from one base model

Use QLoRA (PEFT + quantization) when:

• Fine-tuning 70B models on single 24GB GPU
• Memory is the primary constraint
• Can accept ~5% quality trade-off vs full fine-tuning

Use full fine-tuning instead when:

• Training small models (<1B parameters)
• Need maximum quality and have compute budget
• Significant domain shift requires updating all weights

Quick start

Installation

# Basic installation
pip install peft

# With quantization support (recommended)
pip install peft bitsandbytes

# Full stack
pip install peft transformers accelerate bitsandbytes datasets

LoRA fine-tuning (standard)

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from peft import get_peft_model, LoraConfig, TaskType
from datasets import load_dataset

# Load base model
model_name = "meta-llama/Llama-3.1-8B"
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token

# LoRA configuration
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,                          # Rank (8-64, higher = more capacity)
    lora_alpha=32,                 # Scaling factor (typically 2*r)
    lora_dropout=0.05,             
    target_modules=[, , , ],  
    bias=                    
)


model = get_peft_model(model, lora_config)
model.print_trainable_parameters()



dataset = load_dataset(, split=)

 ():
    text = 
     tokenizer(text, truncation=, max_length=, padding=)

tokenized = dataset.(tokenize, remove_columns=dataset.column_names)


training_args = TrainingArguments(
    output_dir=,
    num_train_epochs=,
    per_device_train_batch_size=,
    gradient_accumulation_steps=,
    learning_rate=,
    fp16=,
    logging_steps=,
    save_strategy=
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized,
    data_collator= data: {: torch.stack([f[]  f  data]),
                                 : torch.stack([f[]  f  data]),
                                 : torch.stack([f[]  f  data])}
)

trainer.train()


model.save_pretrained()