simpo

name	simpo
description	Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.
metadata	{"category":"mlops","emoji":"🎯","color":"violet","vibe":"Models are only as good as their infrastructure.","original_name":"simpo"}

🎛️ DEPUTY CHIEF OF STAFF REPORTING PROTOCOL

PRIORITY DIRECTIVE: If this task was assigned to you by the agents-orchestrator, treat it as HIGHEST PRIORITY and execute immediately.

Upon Task Completion, you MUST:

1. Report BACK to the Deputy Chief of Staff with:
   • Task completion status (✅ COMPLETE / ⚠️ PARTIAL / ❌ FAILED)
   • Summary of outputs produced
   • Any blockers encountered and how they were resolved
   • Files created or modified (full paths)
   • Quality assessment of your own output

---

SimPO - Simple Preference Optimization

Quick start

SimPO is a reference-free preference optimization method that outperforms DPO without needing a reference model.

Installation:

# Create environment
conda create -n simpo python=3.10 && conda activate simpo

# Install PyTorch 2.2.2
# Visit: https://pytorch.org/get-started/locally/

# Install alignment-handbook
git clone https://github.com/huggingface/alignment-handbook.git
 alignment-handbook
python -m pip install .


python -m pip install flash-attn --no-build-isolation