| name | markovian-thinker |
| title | The Markovian Thinker: Streaming Language Models with Decoupled Thinking |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2510.06557 |
| keywords | ["reasoning","efficient-inference","streaming","rl-training","context-management"] |
| description | Enable LLMs to scale reasoning length from O(n²) to O(n) by structuring thinking into fixed-size chunks with learnable cross-chunk summaries. Trigger: train reasoning models with unbounded or expensive chain-of-thought sequences. |
The Markovian Thinker: Decoupled Chunk-Based Reasoning
Core Concept
Traditional RL-based reasoning in LLMs suffers from quadratic compute scaling as thinking length grows. The Markovian Thinker solves this by decomposing reasoning into fixed-size chunks where the model learns to write concise summaries at chunk boundaries. This enables the context to reset between chunks while maintaining reasoning continuity—achieving linear scaling instead of quadratic.
The key insight: Markovian structure allows "chunked thinking" where state transitions depend only on the previous chunk summary, not the entire prior history.
Architecture Overview
- Chunk-Based Decomposition: Divide reasoning into fixed windows (e.g., 8K tokens)
- Boundary Summaries: Model learns to write compressed summaries at chunk edges
- Context Reset Strategy: Environment resets context window after each chunk, reinitializing with prompt + summary
- Delethink RL Environment: Custom RL environment that enforces chunk boundaries
- Linear Recurrence: Only the previous summary state carries forward, enabling O(n) compute
Implementation Steps
1. Design Chunk Structure and Boundaries
Define your chunk size based on available context window and thinking complexity. Typical configurations use 8K-token chunks for a 16K context window.
class ChunkThinkingConfig:
chunk_size = 8192
context_window = 16384
summary_max_tokens = 256
model_size = "1.5B"
def validate(self):
assert self.summary_max_tokens < self.chunk_size // 32
assert self.context_window >= 2 * self.chunk_size
2. Implement Delethink Environment
