Skip to main content Home Creators adu2021 skillxiv internvl3-5-multimodal-cascade-rl
internvl3-5-multimodal-cascade-rl Enhance multimodal models through cascade RL for reasoning improvement and visual resolution routing for inference efficiency, achieving 16% reasoning gains and 4.05x speedup.
Jump to install Skills Marketplace Discover and explore AI skills built by the community.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Show prompt details A direct command skips the review prompt. Inspect the source before running it.
npx skills add https://github.com/ADu2021/skillXiv --skill internvl3-5-multimodal-cascade-rlThe command stays on one line. Scroll horizontally to inspect it before copying.
Prefer a local copy? Download the files currently available to SkillsMP.
Download Zip Downloading... More from this repository meaningful-kebab-case-name Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
action-quantization-behavior-cloning Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
adaptive-lora-personalized-ranks Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
Related occupations SOC
Based on SOC occupation classification
name internvl3-5-multimodal-cascade-rl title InternVL3.5: Cascade RL and Visual Resolution Router for Multimodal Efficiency version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2508.18265 keywords ["multimodal-models","cascade-reinforcement-learning","visual-efficiency","reasoning","inference-optimization"] description Enhance multimodal models through cascade RL for reasoning improvement and visual resolution routing for inference efficiency, achieving 16% reasoning gains and 4.05x speedup.
InternVL3.5: Cascade RL and Visual Resolution Router
Core Concept
InternVL3.5 advances multimodal AI through two complementary techniques: Cascade RL for enhanced reasoning and Visual Resolution Router (ViR) for adaptive efficiency. Cascade RL combines offline RL for stable convergence with online RL for refined alignment, improving performance on visual reasoning benchmarks. ViR dynamically adjusts visual token resolution based on image complexity, maintaining performance while reducing computation. Together, these achieve 16% reasoning improvement and 4.05x inference speedup.
Architecture Overview
Cascade RL Framework : Two-stage training (offline → online)Visual Resolution Router : Dynamic resolution selection per imageDecoupled Vision-Language Deployment : Distributed inferenceReasoning Enhancement : Improved visual understandingEfficiency Optimization : Token reduction without quality loss
Implementation Steps
1. Implement Cascade RL Framework
Two-stage training for stable and effective policy learning:
import torch
import torch.nn as nn
from typing import Dict , List , Tuple
class CascadeRLTrainer :
"""Two-stage RL: offline stable convergence + online refinement."""
def __init__ (
self,
model: "MultimodalLLM" ,
offline_steps: int = 1000 ,
online_steps: int = 500 ,
offline_lr: float = 1e-5 ,
online_lr: float = 5e-6
):
self .model = model
self .offline_steps = offline_steps
self .online_steps = online_steps
self
self
self
"offline"
"online"
def
offline_rl_phase
self,
dataset: List [Dict ],
reference_model: "MultimodalLLM" = None
Dict
str
float
"""
Offline RL: learn from fixed dataset without environment interaction.
Prioritizes stability over exploration.
"""
print
"Starting Offline RL Phase..."
"loss"
"reward"
for
in
range
self
self
8
self
"images"
"questions"
self
"answers"
self
self
self
1.0
self
"loss"
"reward"
if
1
100
0
sum
"loss"
100
100
sum
"reward"
100
100
print
f"Offline Step {step+1 } : Loss={avg_loss:.4 f} , Reward={avg_reward:.4 f} "
self
"offline"
return
"avg_loss"
sum
"loss"
len
"loss"
"avg_reward"
sum
"reward"
len
"reward"
def
online_rl_phase
self,
env: "VisualReasoning Env" ,
num_episodes: int = 100
Dict
str
float
"""
Online RL: interact with environment for refinement.
Fine-tunes policy on real interactions.
"""
print
"Starting Online RL Phase..."
"episode_reward"
"episode_length"
for
in
range
0
0
for
in
range
self
self
"image"
"question"
100
self
self
self
self
1
if
break
"episode_reward"
"episode_length"
if
1
10
0
sum
"episode_reward"
10
10
print
f"Online Episode {episode+1 } : Reward={avg_reward:.2 f} "
self
"online"
return
"avg_episode_reward"
sum
"episode_reward"
len
"episode_reward"
"avg_episode_length"
sum
"episode_length"
len
"episode_length"
def
_compute_offline_loss
self,
logits: torch.Tensor,
rewards: torch.Tensor
"""Offline RL loss: conservative policy update."""
1
1
0.0
return
def
_compute_online_loss
self,
state: Dict ,
action: str ,
reward: float
"""Online RL loss: standard policy gradient."""
self
"image"
"question"
self
return
def
_compute_correctness_reward
self,
predictions: List [str ],
targets: List [str ]
"""Compute reward based on answer correctness."""
for
in
zip
1.0
if
else
0.1
return
def
_sample_batch
self, dataset: List [Dict ], batch_size: int
"""Sample batch from dataset."""
import
min
len
return
"images"
"image"
for
in
"questions"
"question"
for
in
"answers"
"answer"
for
in
def
_store_transition
self, state, action, reward, next_state, done
pass
def
_get_log_prob_of_action
self, action: str , logits: torch.Tensor
pass
2. Implement Visual Resolution Router Dynamically adjust image resolution based on complexity:
class VisualResolutionRouter (nn.Module):
"""Adaptively select image resolution for efficiency."""
def __init__ (self, model: "MultimodalLLM" , resolution_options: List [int ] = [224 , 336 , 448 , 672 ] ):
super ().__init__()
self .model = model
self .resolution_options = sorted (resolution_options)
self .complexity_encoder = nn.Sequential(
nn.Conv2d(3 , 64 , kernel_size=3 , padding=1 ),
nn.ReLU(),
nn.AdaptiveAvgPool2d((1 , 1 )),
nn.Flatten(),
nn.Linear(64 , 32 ),
nn.ReLU(),
nn.Linear(32 , len (resolution_options))
)
def forward (
self,
images: torch.Tensor,
return_complexity: bool = False
) -> Tuple [torch.Tensor, List [int ]]:
"""
Select resolution for each image based on complexity.
"""
batch_size = images.shape[0 ]
normalized_images = torch.nn.functional.interpolate(
images,
size=(224 , 224 ),
mode='bilinear' ,
align_corners=False
)
complexity_logits = self .complexity_encoder(normalized_images)
complexity_probs = torch.softmax(complexity_logits, dim=-1 )
complexity_scores = torch.max (complexity_probs, dim=-1 )[1 ]
selected_resolutions = [
self .resolution_options[score.item()] for score in complexity_scores
]
resized_images = []
for img, res in zip (images, selected_resolutions):
resized = torch.nn.functional.interpolate(
img.unsqueeze(0 ),
size=(res, res),
mode='bilinear' ,
align_corners=False
).squeeze(0 )
resized_images.append(resized)
return resized_images, selected_resolutions
def train_router (
self,
train_images: List [torch.Tensor],
train_quality: List [float ],
num_epochs: int = 10
):
"""
Train router to balance quality and efficiency.
"""
optimizer = torch.optim.Adam(self .complexity_encoder.parameters(), lr=1e-4 )
for epoch in range (num_epochs):
total_loss = 0.0
for image, target_quality in zip (train_images, train_quality):
img_normalized = torch.nn.functional.interpolate(
image.unsqueeze(0 ),
size=(224 , 224 )
)
logits = self .complexity_encoder(img_normalized)
complexity_score = torch.softmax(logits, dim=-1 )
expected_quality = complexity_score * torch.tensor([0.7 , 0.8 , 0.9 , 1.0 ])
loss = torch.nn.functional.mse_loss(expected_quality.sum (), torch.tensor(target_quality))
optimizer.zero_grad()
loss.backward()
optimizer.step()
total_loss += loss.item()
if (epoch + 1 ) % 2 == 0 :
print (f"Router Training Epoch {epoch+1 } : Loss={total_loss/len (train_images):.4 f} " )
3. Implement Decoupled Deployment Separate vision and language models for distributed inference:
class DecoupledInference :
"""Vision and language models on separate GPUs for efficiency."""
def __init__ (
self,
vision_model: nn.Module,
language_model: nn.Module,
vision_device: str = "cuda:0" ,
language_device: str = "cuda:1"
):
self .vision_model = vision_model.to(vision_device)
self .language_model = language_model.to(language_device)
self .vision_device = vision_device
self .language_device = language_device
def forward (
self,
images: torch.Tensor,
questions: List [str ]
) -> List [str ]:
"""
Distributed forward pass: vision on GPU0, language on GPU1.
"""
images = images.to(self .vision_device)
with torch.no_grad():
vision_embeddings = self .vision_model(images)
vision_embeddings = vision_embeddings.to(self .language_device)
language_inputs = self .language_model.encode_multimodal(
vision_embeddings,
questions
)
responses = self .language_model.generate(language_inputs, max_length=100 )
return responses
def measure_throughput (
self,
num_examples: int = 100 ,
batch_size: int = 8
) -> Dict [str , float ]:
"""
Measure inference throughput with decoupled deployment.
"""
import time
start_time = time.time()
for _ in range (num_examples // batch_size):
dummy_images = torch.randn(batch_size, 3 , 224 , 224 )
dummy_questions = ["What is in the image?" ] * batch_size
_ = self .forward(dummy_images, dummy_questions)
elapsed = time.time() - start_time
throughput = num_examples / elapsed
return {
"throughput_examples_per_second" : throughput,
"latency_per_example_ms" : (elapsed / num_examples) * 1000 ,
"total_time_seconds" : elapsed
}
Practical Guidance
When to Use InternVL3.5 Techniques
Visual reasoning tasks (MMMU, MathVista)
Multimodal applications with efficiency constraints
GUI or document analysis with variable complexity
Production systems requiring inference optimization
Models needing both reasoning and speed
When NOT to Use
Single-image simple classification
Models without GPU distribution capability
Real-time ultra-low-latency systems (<50ms)
Tasks where resolution is critical and fixed
Key Hyperparameters
offline_rl_steps : 500-2000online_rl_episodes : 50-200resolution_options : [224, 336, 448, 672]cascade_rl_weight : 0.5 for each phasetemperature (offline) : 0.7-1.0
Performance Expectations
Reasoning Improvement: +16.0%
Inference Speedup: 4.05x
Quality Preservation: 95%+ accuracy maintained
GPU Memory: Distributed across devices
Reference Researchers. (2024). InternVL3.5: Advancing Open-Source Multimodal Models. arXiv preprint arXiv:2508.18265.
