terminator-early-stopping-cot

name	terminator-early-stopping-cot
title	TERMINATOR: Learning Optimal Exit Points for Early Stopping in CoT Reasoning
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.12529
keywords	["Early Stopping","Chain-of-Thought","Reasoning Efficiency","Token Reduction","Exit Prediction"]
description	Learn when to terminate chain-of-thought reasoning by predicting answer arrival patterns. Train an early-exit mechanism on empirical first-answer positions to reduce reasoning length by 14-55% while maintaining or improving accuracy.

TERMINATOR: Learning Optimal Exit Points for Early Stopping in CoT Reasoning

Large Reasoning Models (LRMs) generate excessive intermediate tokens during chain-of-thought reasoning, continuing to compute and output reasoning steps long after arriving at the correct answer. TERMINATOR solves this by treating the problem not as hand-tuned stopping rules, but as a learnable prediction task. By observing where answers first appear in successful reasoning traces, you can train an early-exit mechanism that terminates generation at those optimal points—reducing reasoning token length by 14-55% across benchmarks while maintaining or improving accuracy.

The core insight: answer arrival patterns are predictable from problem characteristics and early reasoning steps. Rather than arbitrary length limits, you can learn task-specific stopping points empirically from data.

Core Concept

TERMINATOR operates through three key phases:

1. Data Collection — Generate reasoning traces with full CoT output, identify first answer occurrence position
2. Pattern Learning — Train predictor on early reasoning context to forecast optimal exit point
3. Inference — Use predictor to signal early termination, preventing wasteful over-generation

The technique treats reasoning length not as a fixed budget, but as a learned function of problem difficulty and generation progress.

Architecture Overview

• Reference Dataset Generation — Run full CoT on target tasks, mark first answer arrival position
• Early-Context Encoder — Processes partial reasoning (first N% of generated tokens) to predict optimal stopping point
• Exit Signal Predictor — Outputs confidence that answer has arrived; triggers stopping when confidence exceeds threshold
• Length Estimation Model — Predicts task-specific optimal reasoning length distribution
• Inference Pipeline — Monitor generation stream, apply early exit when signal triggers
• Validation Framework — Test on held-out problems, measure accuracy preservation and token reduction

Implementation Steps

Begin by collecting reference data showing where answers first appear in successful reasoning traces.

import json
from collections import defaultdict

def collect_answer_arrival_data(model, tasks, output_dir=):
    
    arrivals = []

     task_id, task  (tasks):
        
        prompt = 
        full_trace = model.generate(prompt, max_tokens=, return_token_log=)

        
        answer_token_start = 
        answer_text = task.get(, )

        
         token_idx, token_data  (full_trace[]):
             answer_text.lower()  token_data[].lower():
                answer_token_start = token_idx
                

         answer_token_start  :
              

        arrivals.append({
            : task_id,
            : task[],
            : (full_trace[]),
            : answer_token_start,
            : answer_token_start / (full_trace[]),
            : full_trace[],
            : full_trace[][:answer_token_start]
        })

    
    by_domain = defaultdict()
     arrival  arrivals:
        by_domain[arrival[]].append(
            arrival[])

    ()
     domain, ratios  by_domain.items():
        avg_ratio = (ratios) / (ratios)
        ()

     arrivals