Skip to main content الرئيسية المنشئون adu2021 skillxiv synthrl-visual-reasoning
synthrl-visual-reasoning Scale visual reasoning via automated synthesis of challenging questions from seed samples, using verification mechanisms to ensure correctness and verify RL training gains on out-of-domain visual math tasks.
الانتقال إلى التثبيت سوق المهارات اكتشف واستكشف مهارات الذكاء الاصطناعي التي بناها المجتمع.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
عرض تفاصيل Prompt يتجاوز الأمر المباشر Prompt المخصّص للمراجعة. افحص المصدر قبل تشغيله.
npx skills add https://github.com/ADu2021/skillXiv --skill synthrl-visual-reasoningيبقى الأمر في سطر واحد. مرّر أفقيًا لمراجعته كاملًا قبل النسخ.
تفضّل نسخة محلية؟ نزّل الملفات المتاحة حاليًا لدى SkillsMP.
تحميل Zip جاري التحميل... المهن ذات الصلة SOC
استنادا إلى تصنيف SOC المهني
name synthrl-visual-reasoning title SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2506.02096 keywords ["data synthesis","visual reasoning","reinforcement learning","vision-language models","mathematical reasoning"] description Scale visual reasoning via automated synthesis of challenging questions from seed samples, using verification mechanisms to ensure correctness and verify RL training gains on out-of-domain visual math tasks.
SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis
Core Concept
SynthRL addresses data scarcity in visual reasoning tasks by automatically synthesizing challenging training questions from a seed dataset. Rather than manually creating thousands of visual math problems, the framework programmatically generates harder variants and verifies their correctness through automatic checkers.
The approach synthesizes 3,300+ additional challenging questions from approximately 8,000 seed samples, then trains vision-language models using reinforcement learning with verifiable reward signals. The key advantage is that synthesized data gains compound across test-time compute scaling, with improvements most pronounced on the hardest evaluation samples.
Architecture Overview
Seed Question Selection : Identify representative problems from original datasetAugmentation Pipeline : Automatically generate harder variants through question transformationVerification Mechanism : Validate correctness of synthesized questions via automated checkingRLVR Framework : Reinforcement learning with verifiable rewards (not learned rewards)Out-of-Domain Testing : Evaluate on benchmarks different from training distributionScalable Data Generation : Generate thousands of new questions with minimal manual effort
Implementation
The following steps outline how to implement verifiable data synthesis for visual reasoning:
Prepare seed dataset - Collect initial visual reasoning problems with verified answersSelect candidates for augmentation - Identify problems suitable for difficulty increaseGenerate harder variants - Apply transformations to create more challenging versionsVerify correctness - Automatically check that generated questions are correct and harderPrepare RL training - Create question-answer pairs with verifiable reward signalsTrain with RL - Optimize model using reinforcement learning on synthetic dataEvaluate on benchmarks - Test on out-of-domain visual reasoning tasks
from typing import List , Dict , Tuple , Optional
import torch
class
QuestionSynthesizer
"""Generate harder variants of visual reasoning questions."""
def
__init__
self, base_model, transformer_templates: List [str ]
self
self
def
select_candidates
self, seed_questions: List [Dict ], selection_ratio: float = 0.3
List
Dict
"""Select representative questions for augmentation."""
int
len
sorted
lambda
'difficulty'
0.5
2
2
return
def
augment_question
self, question: Dict , transformation: str
Optional
Dict
"""Apply transformation to make question harder."""
"question"
"image"
"answer"
"multi_step"
f"First calculate intermediate result. {original_text} "
"constraints"
f"With the constraint that all values are positive: {original_text} "
"complexity"
f"This problem involves more complex relationships: {original_text} "
"precision"
f"Provide answer accurate to 3 decimal places: {original_text} "
return
"question"
"image"
"original_answer"
"transformation"
def
generate_synthetic_dataset
self, seed_data: List [Dict ],
augmentations_per_question: int = 4
List
Dict
"""Generate synthetic questions from seed data."""
self
for
in
for
in
enumerate
self
self
if
return
class
QuestionVerifier
"""Verify correctness and difficulty of synthetic questions."""
def
__init__
self, reference_solver
self
def
verify_correctness
self, question: Dict
Tuple
bool
Optional
float
"""Check if synthesized question has correct answer."""
try
self
"image"
"question"
"original_answer"
self
self
return
except
as
return
False
None
def
verify_difficulty
self, original_question: Dict , synthetic_question: Dict
float
"""Assess if synthetic question is harder than original."""
"question"
"question"
len
len
return
min
1.0
def
_compare_answers
self, predicted, expected
bool
"""Compare answers with tolerance for numerical problems."""
if
isinstance
int
float
and
isinstance
int
float
return
abs
0.01
return
str
str
def
_estimate_confidence
self, answer
float
"""Estimate confidence in answer (simplified)."""
return
0.95
if
else
0.1
def
filter_synthetic_data
self, synthetic_questions: List [Dict ],
quality_threshold: float = 0.9
Tuple
List
Dict
Dict
"""Filter synthetic questions by quality and difficulty."""
"total"
len
"valid"
0
"rejected"
0
"avg_difficulty"
0
for
in
self
if
not
"rejected"
1
continue
"_original"
self
if
0.0
"valid"
1
"avg_difficulty"
sum
len
if
else
0
return
class
VisualReasoningRLTrainer
"""Train vision-language model with reinforcement learning on synthetic data."""
def
__init__
self, model, verifier: QuestionVerifier
self
self
def
compute_reward
self, question: Dict , predicted_answer: str
float
"""Compute verifiable reward signal."""
self
if
return
if
else
0.95
else
return
0.0
def
rl_train_step
self, batch_questions: List [Dict ], optimizer,
discount_factor: float = 0.99
float
"""Execute one RL training step."""
0.0
for
in
self
"image"
"question"
self
return
len
def
train
self, synthetic_dataset: List [Dict ], num_epochs: int = 3 ,
batch_size: int = 32 , learning_rate: float = 1e-4
Dict
"""Train model on synthetic dataset."""
self
"epoch_losses"
"total_correct"
0
"total_samples"
0
for
in
range
0.0
for
in
range
0
len
self
len
"epoch_losses"
print
f"Epoch {epoch} : Loss = {avg_loss:.4 f} "
return
class
SynthRLPipeline
"""End-to-end pipeline for synthetic data generation and RL training."""
def
__init__
self, base_model, reference_solver, templates: List [str ]
self
self
self
self
def
run_pipeline
self, seed_data: List [Dict ], num_epochs: int = 3
Dict
"""Run full synthesis and training pipeline."""
print
"Step 1: Generating synthetic questions..."
self
print
"Step 2: Verifying synthetic data quality..."
self
print
f" Validation: {stats['valid' ]} /{stats['total' ]} valid ({100 *stats['valid' ]/stats['total' ]:.1 f} %)"
print
"Step 3: Training with RL..."
self
return
"synthesis_stats"
"training_metrics"
"synthetic_dataset_size"
len
Practical Guidance Data synthesis configuration:
Augmentations per question : 3-5 variants per seed question; balance diversity with verification costSelection ratio : 30-50% of seed data; focus on representative examplesQuality threshold : 0.85-0.95; higher threshold ensures clean training data
Reference solver : Use symbolic solver for math, multiple models for consensus checkingDifficulty assessment : Measure by problem complexity metrics, not length aloneRejection rate : Expect 20-40% of initial synthetic questions to fail verification
Batch size : 32-64 depending on dataset size and GPU memoryLearning rate : 1e-4 to 1e-5 for stable training on noisy RL signalsEpochs : 2-5 epochs typically sufficient; monitor for overfitting
Scaling training data for visual reasoning without manual labeling
Tasks with verifiable correct answers (math, logic, code)
Improving out-of-domain generalization through diverse synthetic data
Research on automated curriculum learning and data augmentation
Tasks without automated verification (open-ended reasoning, creative tasks)
Domains where synthetic data distribution differs significantly from real data
Systems requiring exact distribution matching (GANs, etc.)
Real-time applications where verification overhead is prohibitive
Distribution shift : Synthetic data may not match test distribution; validate on original benchmarksVerification bias : Incorrect reference solver validates wrong answers; use multiple verifiersDifficulty plateau : Transformations may not consistently increase difficulty; use adaptive samplingMode collapse : RL training may overfit to verification artifacts; regularize with original dataComputational cost : Verification adds significant overhead; batch verification to amortize cost
Reference SynthRL synthesizes 3,300+ additional challenging questions from approximately 8,000 seed samples, demonstrating consistent improvements across five visual math reasoning benchmarks. Gains are most pronounced on the hardest evaluation samples, suggesting the approach effectively elicits deeper reasoning patterns.
Original paper: "SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis" (arxiv.org/abs/2506.02096)
