fine-tuning-expert

Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model.

跳到安装

仓库: Jeffallan/claude-skills
最近来源活动: 2026年4月29日 03:32
检测到的 SKILL.md 语言: 英语
星标: 11,000
分支: 1,031

安装方式

默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。

用 Codex 或 Claude 帮你安装复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。

直接命令不会经过审查 Prompt；运行前请先检查来源。

npx skills add https://github.com/Jeffallan/claude-skills --skill fine-tuning-expert

命令会保持在同一行。复制前请横向滚动并检查完整内容。

想先保存到本地？可下载 SkillsMP 当前能够提供的文件。

检查来源文件

决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。

文件资源管理器

6 个文件

SKILL.md

来源说明 · 只读预览

返回搜索返回安装

name	fine-tuning-expert
description	Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model.
license	MIT
metadata	{"author":"https://github.com/Jeffallan","version":"1.1.0","domain":"data-ml","triggers":"fine-tuning, fine tuning, finetuning, LoRA, QLoRA, PEFT, adapter tuning, transfer learning, model training, custom model, LLM training, instruction tuning, RLHF, model optimization, quantization","role":"expert","scope":"implementation","output-format":"code","related-skills":"devops-engineer"}

Fine-Tuning Expert

Senior ML engineer specializing in LLM fine-tuning, parameter-efficient methods, and production model optimization.

Core Workflow

Dataset preparation — Validate and format data; run quality checks before training starts
- Checkpoint: python validate_dataset.py --input data.jsonl — fix all errors before proceeding
Method selection — Choose PEFT technique based on GPU memory and task requirements
- Use LoRA for most tasks; QLoRA (4-bit) when GPU memory is constrained; full fine-tune only for small models

from datasets import load_dataset from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments from peft import LoraConfig, get_peft_model, TaskType from trl import SFTTrainer import torch # 1. Load base model and tokenizer model_id = "meta-llama/Llama-3-8B" tokenizer = AutoTokenizer.from_pretrained(model_id) tokenizer.pad_token = tokenizer.eos_token model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto", ) # 2. Configure LoRA adapter lora_config = LoraConfig( task_type=TaskType.CAUSAL_LM, r=16, # rank — increase for more capacity, decrease to save memory lora_alpha=32, # scaling factor; typically 2× rank target_modules=["q_proj", "v_proj"], lora_dropout=0.05, bias="none", ) model = get_peft_model(model, lora_config) model.print_trainable_parameters() # verify: should be ~0.1–1% of total params # 3. Load and format dataset (Alpaca-style JSONL) dataset = load_dataset("json", data_files={"train": "train.jsonl", "test": "test.jsonl"}) def format_prompt(example): return {"text": f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"} dataset = dataset.map(format_prompt) # 4. Training arguments training_args = TrainingArguments( output_dir="./checkpoints", num_train_epochs=3, per_device_train_batch_size=4, gradient_accumulation_steps=4, # effective batch size = 16 learning_rate=2e-4, lr_scheduler_type="cosine", warmup_ratio=0.03, # always use warmup fp16=False, bf16=True, logging_steps=10, eval_strategy="steps", eval_steps=100, save_steps=200, load_best_model_at_end=True, ) # 5. Train trainer = SFTTrainer( model=model, args=training_args, train_dataset=dataset["train"], eval_dataset=dataset["test"], dataset_text_field="text", max_seq_length=2048, ) trainer.train() # 6. Save adapter weights only model.save_pretrained("./lora-adapter") tokenizer.save_pretrained("./lora-adapter")

Topic	Reference	Load When
LoRA/PEFT	`references/lora-peft.md`	Parameter-efficient fine-tuning, adapters
Dataset Prep	`references/dataset-preparation.md`	Training data formatting, quality checks
Hyperparameters	`references/hyperparameter-tuning.md`	Learning rates, batch sizes, schedulers
Evaluation	`references/evaluation-metrics.md`	Benchmarking, metrics, model comparison
Deployment	`references/deployment-optimization.md`	Model merging, quantization, serving

fine-tuning-expert

安装方式

检查来源文件

Fine-Tuning Expert

Core Workflow

Fine-Tuning Expert

Core Workflow

Reference Guide

Minimal Working Example — LoRA Fine-Tuning with Hugging Face PEFT

Constraints

MUST DO

MUST NOT DO

Output Templates

fine-tuning-expert

来源信息

安装方式

检查来源文件

同仓库更多 Skills

同仓库更多 Skills

Fine-Tuning Expert

Core Workflow

Fine-Tuning Expert

Core Workflow

Reference Guide

Minimal Working Example — LoRA Fine-Tuning with Hugging Face PEFT

Constraints

MUST DO

MUST NOT DO

Output Templates