huggingface

name	huggingface
description	[Applies to: **/*.py] Definitive guidelines for writing robust, performant, and maintainable Python code using Hugging Face Transformers and the Hugging Face Hub.
source	cursor_mdc

Hugging Face Best Practices

This guide establishes the definitive coding standards for developing with Hugging Face libraries, particularly transformers, and interacting with the Hugging Face Hub. Adherence ensures high-quality, performant, and maintainable ML code.

1. Code Organization and Structure

Always structure your Hugging Face projects for clarity, modularity, and Hub compatibility.

1.1 Model and Tokenizer Loading

Use Auto classes and from_pretrained for standard model and tokenizer loading. This ensures compatibility and leverages the Hub's versioning.

❌ BAD (Hardcoding specific model classes, limits flexibility)

from transformers import BertForSequenceClassification, BertTokenizer

model = BertForSequenceClassification.from_pretrained("bert-base-uncased")
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")

✅ GOOD (Flexible, auto-detecting model types and versions)

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

1.2 Project Structure for Hub Repositories

Embrace the "Git-first" workflow. Every model, dataset, or Space lives in a Git repository on the Hub. Structure your local project to mirror this, ensuring README.md (Model Card) and .gitattributes are present.

✅ GOOD (Standard Hub repository structure)

my-awesome-model/
├── src/
│   ├── model.py            # Custom model definition (if any)
│   └── utils.py            # Helper functions
├── scripts/
│   └── train.py            # Training script
├── data/                   # Small datasets, or pointers to Hub datasets
├── config.json             # Model configuration
├── tokenizer.json          # Tokenizer configuration
├── model.safetensors       # Model weights (or .bin)
├── README.md               # Crucial Model Card with metadata
├── .gitattributes          # For large file handling (Xet)
└── requirements.txt        # Project dependencies

2. Common Patterns and Anti-patterns

2.1 Inference with pipeline

For quick, high-level inference tasks, pipeline is the most efficient and robust choice. It handles preprocessing and postprocessing automatically.

❌ BAD (Manual steps for common tasks, error-prone)

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

inputs = tokenizer("I love Hugging Face!", return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=-1)
print(model.config.id2label[predictions.item()])

✅ GOOD (Leverage pipeline for simplicity and robustness)

from transformers import pipeline

classifier = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
result = classifier("I love Hugging Face!")
print(result)

2.2 Saving and Loading Models

Always use save_pretrained() and from_pretrained() for models and tokenizers. This ensures all necessary files (config, weights, tokenizer files) are handled correctly for Hub compatibility.

❌ BAD (Framework-specific saving, loses crucial metadata)

import torch
from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased")
torch.save(model.state_dict(), "my_model_weights.pt")
# To load, you'd still need the config and potentially other files.

✅ GOOD (Hugging Face standard, Hub-ready)

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = "bert-base-uncased"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Save to a local directory
save_directory = "./my_fine_tuned_model"
model.save_pretrained(save_directory)
tokenizer.save_pretrained(save_directory)

# Load later
loaded_tokenizer = AutoTokenizer.from_pretrained(save_directory)
loaded_model = AutoModelForSequenceClassification.from_pretrained(save_directory)

3. Performance Considerations

Prioritize performance from the start, especially for training and large model inference.

3.1 Hardware-Aware Libraries

Leverage Optimum, PEFT, bitsandbytes, and accelerate for memory and speed optimizations.

✅ GOOD (Using accelerate for distributed training/inference)

# In train.py
from accelerate import Accelerator
from transformers import AutoModelForSequenceClassification, AutoTokenizer, AdamW
from torch.utils.data import DataLoader, Dataset
import torch

class MyDataset(Dataset):
    def __len__(self): return 100
    def __getitem__(self, idx): return {"input_ids": torch.randint(0, 1000, (128,)), "labels": torch.randint(0, 2, (1,))}

accelerator = Accelerator()
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased")
optimizer = AdamW(model.parameters(), lr=1e-5)
train_dataloader = DataLoader(MyDataset(), batch_size=8)

model, optimizer, train_dataloader = accelerator.prepare(
    model, optimizer, train_dataloader
)

# Training loop
for batch in train_dataloader:
    outputs = model(**batch)
    loss = outputs.loss
    accelerator.backward(loss)
    optimizer.step()
    optimizer.zero_grad()

✅ GOOD (Using bitsandbytes for 4-bit quantization with PEFT for LoRA)

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model
import torch

model_id = "meta-llama/Llama-2-7b-hf"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Apply PEFT LoRA
lora_config = LoraConfig(
    r=8, lora_alpha=16, target_modules=["q_proj", "v_proj"], lora_dropout=0.05, bias="none", task_type="CAUSAL_LM"
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

3.2 Mixed Precision Training

Always enable mixed precision (e.g., bfloat16 or float16) for GPU training to reduce memory footprint and speed up computation.

✅ GOOD (With Trainer)

from transformers import Trainer, TrainingArguments, AutoModelForSequenceClassification, AutoTokenizer
from datasets import load_dataset

model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
dataset = load_dataset("imdb")

def tokenize_function(examples):
    return tokenizer(examples["text"], padding="max_length", truncation=True)

tokenized_datasets = dataset.map(tokenize_function, batched=True)

training_args = TrainingArguments(
    output_dir="./results",
    fp16=True, # Enable mixed precision (or bf16=True for bfloat16)
    per_device_train_batch_size=16,
    num_train_epochs=3,
    logging_dir='./logs',
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets["train"],
)
trainer.train()

4. Common Pitfalls and Gotchas

4.1 Device Placement

Ensure your model and inputs are on the same device (CPU/GPU) to avoid runtime errors. Use model.to(device) and inputs.to(device).

❌ BAD (Mixing devices, leads to runtime errors)

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased").cuda() # Model on GPU
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
inputs = tokenizer("Hello", return_tensors="pt") # Inputs on CPU by default
outputs = model(**inputs) # ERROR: Inputs on CPU, model on GPU

✅ GOOD (Consistent device placement)

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased").to(device)
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
inputs = tokenizer("Hello", return_tensors="pt").to(device) # Move inputs to device
outputs = model(**inputs)

4.2 Tokenizer Mismatch

Always use the tokenizer associated with the specific pretrained model you are using. Different models often have different tokenization schemes.

❌ BAD (Using a generic tokenizer, leads to incorrect results)

from transformers import AutoTokenizer, AutoModelForCausalLM

# Using a tokenizer not designed for Llama-2
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModelForCausalLM.from