Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
The command stays on one line. Scroll horizontally to inspect it before copying.
Prefer a local copy? Download the files currently available to SkillsMP.
File Explorer
4 files
Showing SKILL.md
SKILL.md
Source instructions · Read-only preview
name
model-merging
description
Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.
Improve performance beyond single models (often +5-10% on benchmarks)
Reduce training costs - no GPUs needed, merges run on CPU
Experiment rapidly - create new model variants in minutes, not days
Preserve multiple skills - merge without catastrophic forgetting
Success Stories: Marcoro14-7B-slerp (best on Open LLM Leaderboard 02/2024), many top HuggingFace models use merging
Tools: mergekit (Arcee AI), LazyMergekit, Model Soup
Installation
# Install mergekit
git clone https://github.com/arcee-ai/mergekit.git
cd mergekit
pip install -e .
# Or via pip
pip install mergekit
# Optional: Transformer library
pip install transformers torch
Quick Start
Simple Linear Merge
# config.yml - Merge two models with equal weightsmerge_method:linearmodels:-model:mistralai/Mistral-7B-v0.1parameters:weight:0.5-model:teknium/OpenHermes-2.5-Mistral-7Bparameters:weight:0.5dtype:bfloat16
# Run merge
mergekit-yaml config.yml ./merged-model --cuda
# Use merged model
python -m transformers.models.auto --model_name_or_path ./merged-model
merge_method:tiesbase_model:mistralai/Mistral-7B-v0.1models:-model:WizardLM/WizardMath-7B-V1.1parameters:density:0.5# Keep top 50% of parametersweight:1.0-model:teknium/OpenHermes-2.5-Mistral-7Bparameters:density:0.5weight:1.0-model:NousResearch/Nous-Hermes-2-Mistral-7B-DPOparameters:density:0.5weight:1.0parameters:normalize:truedtype:bfloat16
DARE Merge
Best for: Reducing redundancy
merge_method:dare_tiesbase_model:mistralai/Mistral-7B-v0.1models:-model:WizardLM/WizardMath-7B-V1.1parameters:density:0.5# Drop 50% of deltasweight:0.6-model:teknium/OpenHermes-2.5-Mistral-7Bparameters:density:0.5weight:0.4parameters:int8_mask:true# Use int8 for masks (saves memory)dtype:bfloat16
Advanced Patterns
Layer-wise Merging
# Different models for different layersmerge_method:passthroughslices:-sources:-model:mistralai/Mistral-7B-v0.1layer_range: [0, 16] # First half-sources:-model:teknium/OpenHermes-2.5-Mistral-7Blayer_range: [16, 32] # Second halfdtype:bfloat16
MoE from Merged Models
# Create Mixture of Expertsmerge_method:moebase_model:mistralai/Mistral-7B-v0.1experts:-source_model:WizardLM/WizardMath-7B-V1.1positive_prompts:-"math"-"calculate"-source_model:teknium/OpenHermes-2.5-Mistral-7Bpositive_prompts:-"chat"-"conversation"-source_model:ajibawa-2023/Code-Mistral-7Bpositive_prompts:-"code"-"python"dtype:bfloat16
Tokenizer Merging
merge_method:linearmodels:-model:mistralai/Mistral-7B-v0.1-model:custom/specialized-modeltokenizer:source:"union"# Combine vocabularies from both modelstokens:<|special_token|>:source:"custom/specialized-model"
Best Practices
1. Model Compatibility
# ✅ Good: Same architecture
models = [
"mistralai/Mistral-7B-v0.1",
"teknium/OpenHermes-2.5-Mistral-7B", # Both Mistral 7B
]
# ❌ Bad: Different architectures
models = [
"meta-llama/Llama-2-7b-hf", # Llama"mistralai/Mistral-7B-v0.1", # Mistral (incompatible!)
]
2. Weight Selection
# ✅ Good: Weights sum to 1.0models:-model:model_aparameters:weight:0.6-model:model_bparameters:weight:0.4# 0.6 + 0.4 = 1.0# ⚠️ Acceptable: Weights don't sum to 1 (for task arithmetic)models:-model:model_aparameters:weight:0.8-model:model_bparameters:weight:0.8# May boost performance
3. Method Selection
# Choose merge method based on use case:# 2 models, smooth blend → SLERP
merge_method = "slerp"# 3+ models, simple average → Linear
merge_method = "linear"# Multiple task-specific models → Task Arithmetic or TIES
merge_method = "ties"# Want to reduce redundancy → DARE
merge_method = "dare_ties"
4. Density Tuning (TIES/DARE)
# Start conservative (keep more parameters)parameters:density:0.8# Keep 80%# If performance good, increase sparsityparameters:density:0.5# Keep 50%# If performance degrades, reduce sparsityparameters:density:0.9# Keep 90%
5. Layer-specific Merging
# Preserve base model's beginning and endmerge_method:passthroughslices:-sources:-model:base_modellayer_range: [0, 2] # Keep first layers-sources:-model:merged_middle# Merge middle layerslayer_range: [2, 30]
-sources:-model:base_modellayer_range: [30, 32] # Keep last layers
Evaluation & Testing
Benchmark Merged Models
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged model
model = AutoModelForCausalLM.from_pretrained("./merged-model")
tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Test on various tasks
test_prompts = {
"math": "Calculate: 25 * 17 =",
"code": "Write a Python function to reverse a string:",
"chat": "What is the capital of France?",
}
for task, prompt in test_prompts.items():
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
print(f"{task}: {tokenizer.decode(outputs[0])}")
Common Benchmarks
Open LLM Leaderboard: General capabilities
MT-Bench: Multi-turn conversation
MMLU: Multitask accuracy
HumanEval: Code generation
GSM8K: Math reasoning
Production Deployment
Save and Upload
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged model
model = AutoModelForCausalLM.from_pretrained("./merged-model")
tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Upload to HuggingFace Hub
model.push_to_hub("username/my-merged-model")
tokenizer.push_to_hub("username/my-merged-model")
