reasoning-palette

name	reasoning-palette
title	Reasoning Palette: Modulating Reasoning via Latent Contextualization
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.17206
keywords	["reinforcement-learning","reasoning","exploration","latent-space","vae"]
description	Overcome token-level randomness limitations in RL by shifting exploration to latent reasoning strategies. Train a VAE encoding diverse reasoning patterns, sample latents during RL, decode to prefix embeddings steering internal reasoning—enabling structured exploration across math, coding, and QA with interpretable, controllable behavior.

Overview

Reasoning Palette addresses a fundamental inefficiency in LLM reinforcement learning: standard sampling schemes create limited trajectory diversity because token-level randomness doesn't generate sufficiently different reasoning approaches. This framework shifts exploration to a learned latent space of reasoning strategies.

Core Technique

The key insight is that reasoning diversity should come from high-level strategy changes, not token-level noise.

VAE-Based Latent Strategy Space: Learn a continuous space encoding diverse reasoning patterns from multi-domain data.

# VAE for reasoning strategy encoding
class ReasoningStrategyVAE:
    def __init__(self, latent_dim=16):
        self.encoder = Encoder()  # Question-answer → latent
        self.decoder = Decoder()  # Latent → prefix embedding
        self.latent_dim = latent_dim

    def encode_strategies(self, qa_pairs):
        """
        Encode diverse reasoning patterns from mathematical,
        coding, and QA domains into shared latent space.
        """
        strategies = []
        for question, answer in qa_pairs:
            # Embed both question and answer
            q_embed = embedding_model(question)
            a_embed = embedding_model(answer)

            # Mean-pool to get strategy representation
            strategy_rep = (q_embed + a_embed) / 2

            # Encode into latent space
            latent = self.encoder(strategy_rep)
            strategies.append(latent)

        return torch.stack(strategies)

    def sample_and_decode(self):
        """
        Sample from Gaussian latent space and decode to
        prefix embeddings that steer reasoning.
        """
        
        z = torch.randn(, .latent_dim)

        
        prefix_embedding = .decoder(z)

         prefix_embedding