think-router-hybrid-reasoning

name	think-router-hybrid-reasoning
title	ThinkRouter: Efficient Reasoning via Routing between Latent and Discrete
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.11683
keywords	["Reasoning","Latent Space","Discrete Tokens","Confidence-Based Routing","Efficiency"]
description	Route reasoning between discrete token space (when uncertain) and latent soft embeddings (when confident). Use maximum next-token probability as a routing threshold to dynamically select the reasoning space, improving accuracy under latent reasoning while reducing computational cost through selective discrete sampling.

ThinkRouter: Efficient Reasoning via Routing between Latent and Discrete

Problem Context

Extended reasoning models can operate in two spaces: discrete token space (one token sampled at a time) or latent space (soft embeddings aggregated from distributions). Latent reasoning is more efficient but accumulates noise from low-confidence steps, leading to spurious high confidence in wrong answers. Discrete reasoning commits to single tokens, avoiding aggregation noise but is slower. ThinkRouter solves this by routing between spaces based on model confidence.

Core Concept

At each reasoning step, ThinkRouter measures the maximum next-token probability. If probability is below a threshold τ, reasoning operates in discrete token space (commitment under uncertainty). If probability meets or exceeds τ, reasoning uses latent soft embeddings (efficient exploration when confident).

This hybrid approach leverages discrete reasoning's robustness when uncertain and latent reasoning's efficiency when confident.

Architecture Overview

• Confidence detector: Compute maximum next-token probability at each step
• Routing threshold: Learned or grid-searched threshold τ per model-dataset pair
• Discrete branch: Sample single token when max_prob < τ
• Latent branch: Aggregate soft embeddings when max_prob ≥ τ
• Combined reasoning: Seamlessly switch spaces within single reasoning trajectory

Implementation

Step 1: Compute confidence scores and routing decision

Measure model confidence and determine reasoning space.

import torch
import torch.nn.functional as F
from typing import Tuple

class ConfidenceRouter:
    """Route reasoning between latent and discrete spaces based on confidence."""

    def __init__(self, routing_threshold: float = 0.8):
        """
        Args:
            routing_threshold: Probability threshold for routing decision.
                             P(next_token_max) < threshold -> discrete
                             P(next_token_max) >= threshold -> latent
        """
        .routing_threshold = routing_threshold

     () -> [torch.Tensor, torch.Tensor, torch.Tensor]:
        
        
        probs = F.softmax(logits / temperature, dim=-)
        max_prob, max_token_idx = torch.(probs, dim=-)

        
        route_to_latent = max_prob >= .routing_threshold

         max_prob, route_to_latent, max_token_idx

     () -> :
        
        max_probs_tensor = torch.tensor(max_probs)
        route_to_latent = (route_decisions)
        route_to_discrete = (route_decisions) - route_to_latent

         {
            : max_probs_tensor.mean().item(),
            : max_probs_tensor.std().item(),
            : route_to_latent / (route_decisions),
            : route_to_discrete / (route_decisions)
        }