reasoning-via-sampling

name	reasoning-via-sampling
title	Reasoning with Sampling: Your Base Model is Smarter Than You Think
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.14901
keywords	["Sampling","MCMC","Inference","Reasoning","Base Models"]
description	Improves base model reasoning through iterative sampling without training or fine-tuning. Uses MCMC-inspired sampling to extract latent reasoning from pretrained models, achieving RL-comparable gains on math, coding, and QA tasks while preserving diversity.

Reasoning via Sampling: Unlocking Latent Model Capabilities

Base language models possess latent reasoning capabilities that remain untapped by standard greedy decoding. Fine-tuning and RL are expensive; sampling offers a simpler alternative that leverages model likelihoods directly.

This technique applies MCMC-inspired iterative sampling to extract superior reasoning from any pretrained LLM at inference time, with no training requirements or curated verification datasets.

Core Concept

The approach iteratively refines samples by:

• Generating multiple candidate solutions through sampling
• Using the model's own likelihood as a quality signal
• Selecting highest-likelihood sequences that solve the problem
• Repeating to find better solutions within the model's capability envelope

Unlike RL, this preserves sample diversity because it doesn't collapse reward toward mode-seeking behavior.

Architecture Overview

• Multiple sampling rounds with early stopping when solutions are found
• Likelihood-based ranking of valid solutions (no external verifier needed)
• Resampling around promising regions based on conditional probabilities
• Terminal condition when acceptable solution found or iteration limit reached

Implementation Steps

Initialize the sampling loop to generate multiple rollouts from the base model. The core insight is that higher model likelihood correlates with quality:

def iterative_sample(model, prompt, num_rounds=10, samples_per_round=8):
    best_solution = None
    best_likelihood = float('-inf')

    for round_idx in range(num_rounds):
        # Generate candidates from current prompt state
        rollouts = []
        for _ in range(samples_per_round):
            sequence, likelihood = model.sample_with_likelihood(prompt)
            rollouts.append((sequence, likelihood))

        # Find best valid solution this round
        for sequence, likelihood in rollouts:
             is_valid_solution(sequence)  likelihood > best_likelihood:
                best_likelihood = likelihood
                best_solution = sequence

        
         best_solution  best_likelihood > threshold:
            

     best_solution