markovian-thinker

name	markovian-thinker
title	The Markovian Thinker: Streaming Language Models with Decoupled Thinking
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.06557
keywords	["reasoning","efficient-inference","streaming","rl-training","context-management"]
description	Enable LLMs to scale reasoning length from O(n²) to O(n) by structuring thinking into fixed-size chunks with learnable cross-chunk summaries. Trigger: train reasoning models with unbounded or expensive chain-of-thought sequences.

The Markovian Thinker: Decoupled Chunk-Based Reasoning

Core Concept

Traditional RL-based reasoning in LLMs suffers from quadratic compute scaling as thinking length grows. The Markovian Thinker solves this by decomposing reasoning into fixed-size chunks where the model learns to write concise summaries at chunk boundaries. This enables the context to reset between chunks while maintaining reasoning continuity—achieving linear scaling instead of quadratic.

The key insight: Markovian structure allows "chunked thinking" where state transitions depend only on the previous chunk summary, not the entire prior history.

Architecture Overview

• Chunk-Based Decomposition: Divide reasoning into fixed windows (e.g., 8K tokens)
• Boundary Summaries: Model learns to write compressed summaries at chunk edges
• Context Reset Strategy: Environment resets context window after each chunk, reinitializing with prompt + summary
• Delethink RL Environment: Custom RL environment that enforces chunk boundaries
• Linear Recurrence: Only the previous summary state carries forward, enabling O(n) compute

Implementation Steps

1. Design Chunk Structure and Boundaries

Define your chunk size based on available context window and thinking complexity. Typical configurations use 8K-token chunks for a 16K context window.

class ChunkThinkingConfig:
    chunk_size = 8192  # tokens per thinking segment
    context_window = 16384
    summary_max_tokens = 256  # compressed carryover state
    model_size = "1.5B"  # parameter count

    def validate(self):
        assert self.summary_max_tokens < self.chunk_size // 32
        assert self.context_window >= 2 * self.chunk_size

2. Implement Delethink Environment