endocot-internal-chain-of-thought

name	endocot-internal-chain-of-thought
title	EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.12252
keywords	["Diffusion","Reasoning","Chain-of-Thought","Latent Space","Generation"]
description	Enable step-by-step reasoning in diffusion models through iterative latent state refinement. Condition diffusion on evolving thought states across multiple reasoning steps, grounded with textual supervision to prevent drift.

Technique: Endogenous Chain-of-Thought via Iterative Latent Refinement

Diffusion models generate images by denoising—but they commit to solutions early in this process. EndoCoT enables intermediate reasoning by allowing the conditioning signal to evolve across steps. Rather than static guidance, the framework iteratively refines hidden states representing thoughts, each conditioned on the previous step's reasoning, then uses these refined states to guide generation.

This "endogenous" approach contrasts with external chain-of-thought: reasoning happens within the diffusion process via latent dynamics, enabling truly integrated visual-linguistic reasoning.

Core Concept

EndoCoT combines three mechanisms:

1. Iterative Thought Guidance: Refine hidden states across multiple reasoning steps before denoising
2. Terminal Thought Grounding: Align final reasoning state with explicit textual reference using semantic loss
3. Progressive Training: First supervise intermediate steps, then optimize final output quality

This enables dynamic, evolving conditioning that guides diffusion through a reasoning trajectory.

Architecture Overview

• MLLM backbone: Language model for thought generation
• Hidden state buffer: Evolving reasoning representations across steps
• Thought refinement network: Updates states based on prior reasoning
• Text reference encoder: Grounds final thoughts in language
• DiT decoder: Diffusion transformer conditioned on thought states
• Progressive loss scheduler: Balance reasoning supervision vs output quality

Implementation Steps

Step 1: Iterative Hidden State Refinement

Refine latent reasoning states across multiple steps before using for generation.

import torch
import torch.nn as nn

class IterativeThoughtRefiner(nn.Module):
    def __init__(self, hidden_dim=768, num_reasoning_steps=4):
        super().__init__()
        self.hidden_dim = hidden_dim
        .num_reasoning_steps = num_reasoning_steps

        
        .refiner_layers = nn.ModuleList([
            nn.Sequential(
                nn.Linear(hidden_dim, hidden_dim * ),
                nn.ReLU(),
                nn.Linear(hidden_dim * , hidden_dim)
            )
             _  (num_reasoning_steps)
        ])

        
        .attention_layers = nn.ModuleList([
            nn.MultiheadAttention(hidden_dim, num_heads=, batch_first=)
             _  (num_reasoning_steps)
        ])

     ():
        
        thought_trajectory = [initial_thought]

        current_thought = initial_thought

         step  (.num_reasoning_steps):
            
            refined = .refiner_layers[step](current_thought)

            
            attended, _ = .attention_layers[step](
                refined.unsqueeze(),  
                reasoning_context,  
                reasoning_context
            )

            
            current_thought = refined + attended.squeeze()

            thought_trajectory.append(current_thought)

         torch.stack(thought_trajectory, dim=)