slime-rl-training

name	slime-rl-training
description	Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.
version	1.0.0
author	Orchestra Research
license	MIT
dependencies	["sglang-router>=0.2.3","ray","torch>=2.0.0","transformers>=4.40.0"]
platforms	["linux","macos"]
metadata	{"sonic":{"tags":["Reinforcement Learning","Megatron-LM","SGLang","GRPO","Post-Training","GLM"]}}

slime: LLM Post-Training Framework for RL Scaling

slime is an LLM post-training framework from Tsinghua's THUDM team, powering GLM-4.5, GLM-4.6, and GLM-4.7. It connects Megatron-LM for training with SGLang for high-throughput rollout generation.

When to Use slime

Choose slime when you need:

• Megatron-LM native training with SGLang inference
• Custom data generation workflows with flexible data buffers
• Training GLM, Qwen3, DeepSeek V3, or Llama 3 models
• Research-grade framework with production backing (Z.ai)

Consider alternatives when:

• You need enterprise-grade stability features → use miles
• You want flexible backend swapping → use verl
• You need PyTorch-native abstractions → use torchforge

Key Features

• Training: Megatron-LM with full parallelism support (TP, PP, DP, SP)
• Rollout: SGLang-based high-throughput generation with router
• Data Buffer: Flexible prompt management and sample storage
• Models: GLM-4.x, Qwen3, DeepSeek V3/R1, Llama 3

Architecture Overview

┌─────────────────────────────────────────────────────────┐
│                    Data Buffer                          │
│ - Prompt initialization and management                  │
│ - Custom data generation and filtering                  │
│ - Rollout sample storage                                │
└─────────────┬───────────────────────────┬───────────────┘
              │                           │
┌─────────────▼───────────┐ ┌─────────────▼───────────────┐
│ Training (Megatron-LM)  │ │ Rollout (SGLang + Router)   │
│ - Actor model training  │ │ - Response generation       │
│ - Critic (optional)     │ │ - Reward/verifier output    │
│ - Weight sync to rollout│ │ - Multi-turn support        │
└─────────────────────────┘ └─────────────────────────────┘

Installation

# Recommended: Docker
docker pull slimerl/slime:latest
docker run --rm --gpus all --ipc=host --shm-size=16g \
  -it slimerl/slime:latest /bin/bash

# Inside container
cd /root/slime && pip install -e . --no-deps

From Source

git clone https://github.com/THUDM/slime.git
cd slime
pip install -r requirements.txt
pip install -e .