Exécutez n'importe quel Skill dans Manus
en un clic

Exécutez n'importe quel Skill dans Manus en un clic

Commencer

llm-fine-tuning

Étoiles6

Forks1

Mis à jour11 juin 2026 à 12:51

LLM fine-tuning: Axolotl (YAML configs), TRL (RLHF/DPO/GRPO), and Unsloth (2-5x faster fine-tuning).

Installation

Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.

Exécuter dans Manus

Source

qiushile

qiushile/AgentTeam

Ouvrir le dépôt GitHub Voir les dépôts du créateur

Téléchargement

Exécuter dans Manus

Métiers associésSOC

Basé sur la classification professionnelle SOC

Développeurs de logicielsProfessions informatiques et mathématiques·SOC 15-1252

SKILL.md

readonly

name	llm-fine-tuning
description	LLM fine-tuning: Axolotl (YAML configs), TRL (RLHF/DPO/GRPO), and Unsloth (2-5x faster fine-tuning).
version	1.0.0
author	Hermes Agent
license	MIT
platforms	["linux","macos"]
metadata	{"hermes":{"tags":["LLM","fine-tuning","Axolotl","TRL","Unsloth","RLHF","DPO","GRPO"]}}

LLM Fine-Tuning Suite

Unified skill for fine-tuning large language models. Three frameworks: Axolotl (YAML-driven), TRL (RLHF/DPO/GRPO), and Unsloth (speed-optimized).

Section A: Axolotl (YAML-Driven Fine-Tuning)

Expert guidance for fine-tuning LLMs with Axolotl. Uses YAML configuration files to define training runs.

Best for: Standard supervised fine-tuning (SFT) and instruction tuning with clean YAML configs.

Typical workflow:

Prepare dataset (JSON/Parquet with instruction/input/output format)
Write Axolotl YAML config (model, dataset, training params)
Run training: accelerate launch -m axolotl.cli.train config.yaml
Merge LoRA weights if applicable
Test fine-tuned model

Section B: TRL — RLHF/DPO/GRPO Training

Fine-tune LLMs using reinforcement learning with the TRL library.

Best for: Alignment training (RLHF, DPO, GRPO, ORPO, KTO) after supervised fine-tuning.

Methods:

SFT: Supervised Fine-Tuning
DPO: Direct Preference Optimization (simpler than RLHF)
GRPO: Group Relative Preference Optimization
ORPO: Odds Ratio Preference Optimization
KTO: Kahneman-Tversky Optimization
RLHF: Full RL from Human Feedback (PPO-based)

Workflow: Start with SFT model → prepare preference data → run DPO/ORPO/KTO or PPO → evaluate.

Section C: Unsloth (2-5x Faster Fine-Tuning)

Optimized fine-tuning: 2-5x faster, 60-80% less VRAM. Compatible with TRL and HuggingFace trainers.

Best for: Limited VRAM (70B on 24GB with QLoRA), fast iteration, training many variants.

Shared Pitfalls

Dataset quality matters most — inspect before training
Start small — test with data subset first
LoRA vs full FT — LoRA is faster; full FT gives better results
Always validate — test on held-out examples
Mixed precision — use bf16 or fp16 for speed

Plus depuis ce dépôt

même dépôt

macos-automation

qiushile/AgentTeam

macOS-specific automation: iMessage/SMS messaging and desktop control (screenshots, clicks, keyboard input).

2026-07-036

communication-platforms

qiushile/AgentTeam

Messaging platform integrations: email (IMAP/SMTP via Himalaya), X/Twitter (xurl CLI), and Yuanbao groups (@mentions, DMs).

2026-07-036

creative-software-control

qiushile/AgentTeam

Control creative software programmatically: ComfyUI (AI image/video generation via REST/WebSocket API) and TouchDesigner (real-time visual programming via MCP).

2026-07-036

engineering-methodologies

qiushile/AgentTeam

Systematic engineering disciplines: debugging (root cause investigation), TDD (test-first development), and exploratory QA (web app testing).

2026-07-036

creative-coding

qiushile/AgentTeam

Creative coding: p5.js sketches, Manim animations, Pretext text layouts.

2026-06-266

powerpoint

qiushile/AgentTeam

Create, read, edit .pptx decks, slides, notes, templates.

2026-06-266

name	llm-fine-tuning
description	LLM fine-tuning: Axolotl (YAML configs), TRL (RLHF/DPO/GRPO), and Unsloth (2-5x faster fine-tuning).
version	1.0.0
author	Hermes Agent
license	MIT
platforms	["linux","macos"]
metadata	{"hermes":{"tags":["LLM","fine-tuning","Axolotl","TRL","Unsloth","RLHF","DPO","GRPO"]}}

LLM Fine-Tuning Suite

Unified skill for fine-tuning large language models. Three frameworks: Axolotl (YAML-driven), TRL (RLHF/DPO/GRPO), and Unsloth (speed-optimized).

Section A: Axolotl (YAML-Driven Fine-Tuning)

Expert guidance for fine-tuning LLMs with Axolotl. Uses YAML configuration files to define training runs.

Best for: Standard supervised fine-tuning (SFT) and instruction tuning with clean YAML configs.

Typical workflow:

Prepare dataset (JSON/Parquet with instruction/input/output format)
Write Axolotl YAML config (model, dataset, training params)
Run training: accelerate launch -m axolotl.cli.train config.yaml
Merge LoRA weights if applicable
Test fine-tuned model

Section B: TRL — RLHF/DPO/GRPO Training

Fine-tune LLMs using reinforcement learning with the TRL library.

Best for: Alignment training (RLHF, DPO, GRPO, ORPO, KTO) after supervised fine-tuning.

Methods:

SFT: Supervised Fine-Tuning
DPO: Direct Preference Optimization (simpler than RLHF)
GRPO: Group Relative Preference Optimization
ORPO: Odds Ratio Preference Optimization
KTO: Kahneman-Tversky Optimization
RLHF: Full RL from Human Feedback (PPO-based)

Workflow: Start with SFT model → prepare preference data → run DPO/ORPO/KTO or PPO → evaluate.

Section C: Unsloth (2-5x Faster Fine-Tuning)

Optimized fine-tuning: 2-5x faster, 60-80% less VRAM. Compatible with TRL and HuggingFace trainers.

Best for: Limited VRAM (70B on 24GB with QLoRA), fast iteration, training many variants.

Shared Pitfalls

Dataset quality matters most — inspect before training
Start small — test with data subset first
LoRA vs full FT — LoRA is faster; full FT gives better results
Always validate — test on held-out examples
Mixed precision — use bf16 or fp16 for speed