Skip to main content Inicio Creadores adu2021 skillxiv mm-helix-reasoning
mm-helix-reasoning Train multimodal models for long-chain reflective reasoning (iterative thinking, backtracking) using Adaptive Hybrid Policy Optimization. Trigger: improve VLM performance on tasks requiring iterative refinement and error correction.
Ir a la instalación Skills Marketplace Descubre y explora habilidades de IA creadas por la comunidad.
Ocupaciones relacionadas SOC
Basado en la clasificación ocupacional SOC
Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.
Mostrar detalles del prompt Un comando directo omite el prompt de revisión. Revisa el origen antes de ejecutarlo.
npx skills add https://github.com/ADu2021/skillXiv --skill mm-helix-reasoningEl comando permanece en una sola línea. Desplázate horizontalmente para revisarlo antes de copiarlo.
¿Prefieres una copia local? Descarga los archivos que SkillsMP tiene disponibles ahora.
Descargar Zip Descargando... Más de este repositorio meaningful-kebab-case-name Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
action-quantization-behavior-cloning Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
adaptive-lora-personalized-ranks Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
name mm-helix-reasoning title MM-HELIX: Multimodal Long-Chain Reflective Reasoning via AHPO version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2510.08540 keywords ["multimodal-reasoning","reflective-thinking","reinforcement-learning","backtracking","adaptive-training"] description Train multimodal models for long-chain reflective reasoning (iterative thinking, backtracking) using Adaptive Hybrid Policy Optimization. Trigger: improve VLM performance on tasks requiring iterative refinement and error correction.
MM-HELIX: Multimodal Reflective Reasoning with AHPO
Core Concept
MM-HELIX addresses a critical gap in multimodal LLMs: they struggle with long-chain reflective reasoning that requires iterative thinking and backtracking. The paper introduces three components: a benchmark (MM-HELIX) with 1,260 samples requiring iterative thinking, a large-scale training dataset (MM-HELIX-100K), and Adaptive Hybrid Policy Optimization (AHPO) that dynamically unifies offline supervision with online RL in a single training stage, achieving 18.6% accuracy improvement.
The key insight: Reflective reasoning requires dynamic training that switches between supervised learning on correct traces and online RL refinement—not static offline or online training alone.
Architecture Overview
Long-Chain Reasoning : Multi-step thinking with iterative refinementBacktracking Capability : Ability to detect and correct reasoning errorsMM-HELIX-100K Dataset : 100K high-quality reasoning traces with iterative patternsAdaptive Hybrid Policy Optimization : Unified offline+online training in single stageCatastrophic Forgetting Prevention : Dynamic balancing prevents mode collapse
Implementation Steps
1. Understand Long-Chain Reflective Reasoning Structure
Define what constitutes reflective reasoning in multimodal contexts.
class ReflectiveReasoningStructure :
"""
Represent multi-step reasoning with backtracking.
"""
def __init__ (self ):
pass
@staticmethod
def extract_reasoning_steps (trace ):
"""
Parse reasoning into steps with branch points.
"""
steps = []
current_path = []
lines = trace.split('\n' )
for line in lines:
if 'THOUGHT:' in line:
current_path.append({
"type" : "thought" ,
"content"
'THOUGHT:'
1
elif
'BACKTRACK:'
in
"path"
"backtrack_reason"
'BACKTRACK:'
1
elif
'CONCLUSION:'
in
"path"
"conclusion"
'CONCLUSION:'
1
return
@staticmethod
def
score_reasoning_quality
trace, ground_truth
"""
Evaluate reasoning quality: correctness, efficiency, iterativeness.
"""
1.0
if
else
0.0
sum
1
for
in
if
"backtrack_reason"
in
min
5.0
1.0
len
1.0
1.0
500
0.7
0.2
0.1
return
"quality_score"
"correctness"
"iterativeness"
"efficiency"
2. Implement Step-Elicited Response Generation (SERG) Create high-quality reasoning traces with explicit step-by-step generation.
class StepElicitedResponseGenerator :
"""
Systematically generate reasoning traces with explicit steps.
"""
def __init__ (self, model ):
self .model = model
def generate_with_steps (self, problem, image=None , max_iterations=5 ):
"""
Generate reasoning by eliciting explicit steps.
Args:
problem: Problem statement
image: Multimodal input (optional)
max_iterations: Max refinement iterations
Returns:
Multi-step reasoning trace
"""
trace = f"Problem: {problem} \n\n"
if image:
trace += f"[Image analysis]\n"
iteration = 0
current_solution = None
while iteration < max_iterations:
thought_prompt = (
trace +
f"\n[Step {iteration + 1 } ]\n"
f"Thinking: "
)
thought = self .model.generate(
thought_prompt,
max_tokens=150 ,
temperature=0.7
)
trace += f"THOUGHT: {thought} \n"
conclusion_prompt = (
trace +
f"Conclusion from this step: "
)
conclusion = self .model.generate(
conclusion_prompt,
max_tokens=100 ,
temperature=0.3
)
trace += f"CONCLUSION: {conclusion} \n"
current_solution = conclusion
evaluation_prompt = (
trace +
f"Is this solution correct and complete? "
f"[Yes/No]. If No, what needs to be reconsidered? "
)
evaluation = self .model.generate(
evaluation_prompt,
max_tokens=50 ,
temperature=0.5
)
if "yes" in evaluation.lower():
trace += "FINAL: Solution is correct.\n"
break
else :
backtrack_reason = evaluation.split("No" )[-1 ].strip()
trace += f"BACKTRACK: {backtrack_reason} \n"
trace += "[Reconsidering...]\n"
iteration += 1
return trace
def generate_dataset_with_serg (self, problems, images, num_samples=100000 ):
"""
Systematically generate 100K reasoning traces.
"""
dataset = []
for idx, (problem, image) in enumerate (zip (problems, images)):
trace = self .generate_with_steps(problem, image)
quality = ReflectiveReasoningStructure.score_reasoning_quality(
trace,
ground_truth=extract_answer(problem)
)
example = {
"problem" : problem,
"image" : image,
"reasoning" : trace,
"quality_score" : quality["quality_score" ]
}
dataset.append(example)
if (idx + 1 ) % 10000 == 0 :
print (f"Generated {idx + 1 } examples" )
if len (dataset) >= num_samples:
break
return dataset
3. Implement Adaptive Hybrid Policy Optimization (AHPO) Dynamically blend offline supervision with online RL in a single unified stage.
class AdaptiveHybridPolicyOptimization :
"""
Unified training that adapts between supervised and online RL.
"""
def __init__ (self, model ):
self .model = model
def compute_ahpo_loss (self, batch, step, total_steps ):
"""
Compute loss that adapts between offline and online optimization.
Args:
batch: Training batch with problems, images, reasoning traces
step: Current training step
total_steps: Total steps in training
Returns:
Scalar loss value
"""
total_loss = 0
batch_size = len (batch)
offline_weight = max (0.7 - (step / total_steps) * 0.5 , 0.2 )
online_weight = 1.0 - offline_weight
for example in batch:
problem = example["problem" ]
image = example["image" ]
target_trace = example["reasoning" ]
generated = self .model.generate(
f"Problem: {problem} " ,
image=image,
max_tokens=2000
)
offline_loss = compute_token_loss(
generated,
target_trace
)
quality = ReflectiveReasoningStructure.score_reasoning_quality(
generated,
extract_answer(problem)
)
reward = quality["quality_score" ]
log_prob = self .model.compute_log_prob(generated)
online_loss = -reward * log_prob
combined_loss = (
offline_weight * offline_loss +
online_weight * online_loss
)
total_loss += combined_loss
entropy = compute_entropy_bonus(
self .model.get_logits(generated)
)
total_loss -= 0.01 * entropy
return total_loss / batch_size, offline_weight, online_weight
def train_ahpo (self, model, dataset, config ):
"""
Execute AHPO training loop.
"""
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5 )
total_steps = config.num_epochs * len (dataset) // config.batch_size
for epoch in range (config.num_epochs):
epoch_loss = 0
for batch_idx in range (0 , len (dataset), config.batch_size):
batch = dataset[batch_idx:batch_idx + config.batch_size]
step = epoch * len (dataset) // config.batch_size + batch_idx // config.batch_size
loss, offline_w, online_w = self .compute_ahpo_loss(
batch,
step,
total_steps
)
optimizer.zero_grad()
loss.backward()
torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0 )
optimizer.step()
epoch_loss += loss.item()
if (batch_idx // config.batch_size) % 100 == 0 :
print (f"Epoch {epoch} , Batch {batch_idx} : "
f"loss={loss:.4 f} , "
f"offline_w={offline_w:.2 f} , online_w={online_w:.2 f} " )
print (f"Epoch {epoch} avg loss: {epoch_loss / len (dataset) * config.batch_size:.4 f} " )
return model
4. Full MM-HELIX Training Pipeline Combine SERG data generation with AHPO training.
def train_mm_helix (base_model, problem_dataset, image_dataset, config ):
"""
Complete MM-HELIX training pipeline.
"""
print ("Step 1: Generating reasoning dataset with SERG" )
serg_generator = StepElicitedResponseGenerator(base_model)
training_dataset = serg_generator.generate_dataset_with_serg(
problem_dataset,
image_dataset,
num_samples=100000
)
quality_filtered = [
ex for ex in training_dataset
if ex["quality_score" ] > 0.5
]
print (f"Kept {len (quality_filtered)} high-quality examples" )
print ("\nStep 2: Training with Adaptive Hybrid Policy Optimization" )
ahpo_trainer = AdaptiveHybridPolicyOptimization(base_model)
trained_model = ahpo_trainer.train_ahpo(
base_model,
quality_filtered,
config
)
return trained_model
5. Evaluation on Reflective Reasoning Benchmarks Assess model performance on iterative reasoning tasks.
def evaluate_mm_helix (model, benchmark_dataset ):
"""
Evaluate model on long-chain reflective reasoning tasks.
"""
results = {
"accuracy" : 0 ,
"avg_iterations" : 0 ,
"avg_reasoning_length" : 0
}
correct = 0
total_iterations = []
reasoning_lengths = []
for example in benchmark_dataset:
problem = example["problem" ]
image = example["image" ]
ground_truth = example["ground_truth" ]
trace = model.generate(
f"Problem: {problem} " ,
image=image,
max_tokens=2000
)
final_answer = extract_final_answer(trace)
if final_answer == ground_truth:
correct += 1
iterations = trace.count("BACKTRACK" )
total_iterations.append(iterations)
reasoning_lengths.append(len (trace.split()))
results["accuracy" ] = correct / len (benchmark_dataset) * 100
results["avg_iterations" ] = np.mean(total_iterations)
results["avg_reasoning_length" ] = np.mean(reasoning_lengths)
print (f"Accuracy: {results['accuracy' ]:.1 f} %" )
print (f"Avg iterations: {results['avg_iterations' ]:.1 f} " )
print (f"Avg reasoning length: {results['avg_reasoning_length' ]:.0 f} tokens" )
return results
Practical Guidance
Offline weight decay : Start at 0.7, decay to 0.2 over trainingOnline weight growth : Start at 0.3, grow to 0.8 over trainingEntropy bonus : 0.01 (prevent mode collapse)Learning rate : 1e-5 (conservative for multimodal models)Batch size : 32-64 (depends on model size)
Training multimodal models for reasoning tasks
Want iterative refinement and backtracking capability
Have access to image+text paired data
Downstream applications require error correction
Single-turn visual question answering
Real-time inference (reflective reasoning adds latency)
Limited compute for large-scale SERG generation
Tasks without multiple solution paths
Reference 