sim-cot-supervised-implicit-chain-of-thought

name	sim-cot-supervised-implicit-chain-of-thought
title	SIM-CoT: Supervised Implicit Chain-of-Thought
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2509.20317
keywords	["implicit reasoning","chain-of-thought","latent representations","step-level supervision","training stability","efficiency","auxiliary decoder","semantic alignment"]
description	Train LLMs to reason implicitly with step-level supervision, stabilizing latent representations while preserving 2.3× inference speedup over explicit chain-of-thought. Addresses training collapse in implicit reasoning by aligning intermediate latent states with explicit reasoning steps through an auxiliary decoder during training, then removing the decoder for efficient inference.

Stabilize Implicit Reasoning with Step-Level Supervision

Build chain-of-thought reasoning into latent token representations rather than explicit text tokens, maintaining inference efficiency while achieving explicit CoT accuracy.

The Problem: Latent Instability in Implicit CoT

Implicit chain-of-thought methods compress reasoning into hidden states rather than generated text tokens, achieving 2.3× speedup over explicit CoT. However, this efficiency comes at a cost: training becomes unstable as you scale reasoning complexity. When you increase implicit reasoning tokens from 3 to 5, models experience training collapse where latent representations become nearly identical, information dissolves, and the model fails to learn meaningful reasoning.

The root cause: current implicit CoT methods provide only answer-level supervision. The model receives feedback only on whether the final answer matches, but nothing guides the intermediate latent states. Without step-level guidance, these representations drift into semantic homogeneity. The latent space geometrically deteriorates—distances between reasoning steps shrink while representations drift from the vocabulary embedding space.

Core Concept: Supervised Implicit Chain-of-Thought

SIM-CoT solves latent instability through auxiliary step-level supervision during training. An auxiliary decoder aligns each implicit latent state with its corresponding explicit reasoning step. This dual-objective training distributes learning signals across the entire reasoning chain rather than concentrating them only at the final answer.

The key insight: you can supervise intermediate representations during training without paying an inference cost. The auxiliary decoder exists only during training—at inference time, you remove it and get pure latent reasoning with all the efficiency benefits.

This enables:

• Stable training with 8-16 implicit tokens (prior methods collapsed at 5)
• Per-step interpretability by projecting latent tokens onto reasoning vocabulary
• Performance gains of +3 to +8 percentage points over prior implicit methods
• 2.3× token efficiency maintained at inference

Architecture Overview

Two-Phase Inference Structure:

• Implicit phase: Model generates K fixed latent reasoning steps, extracting the hidden state at each step as a latent vector z_k
• Explicit phase: After K implicit steps, model switches to standard vocabulary decoding for the final answer

Training-Time Architecture:

• Main model: Transformer LLM with embedded implicit reasoning capability
• Auxiliary decoder: Lightweight module that maps each latent z_k to vocabulary logits for reasoning step s_k