orion-language-of-thought

name	orion-language-of-thought
title	ORION: Teaching LMs to Reason Efficiently via Language of Thought
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2511.22891
keywords	["reasoning-efficiency","language-models","mentalese","reinforcement-learning","structured-reasoning"]
description	Compact Mentalese symbolic format trained via SFT, then refined with SLPO (Shorter Length Preference Optimization) to reward concise correct solutions without over-penalizing necessarily long reasoning. Compress reasoning while maintaining accuracy.

Summary

ORION introduces Mentalese, a compact symbolic reasoning format inspired by cognitive science, paired with Shorter Length Preference Optimization (SLPO), an RL method that adaptively rewards concise solutions. The two-stage approach first aligns models to structured reasoning traces via supervised fine-tuning, then uses RLVR with SLPO to recover accuracy while maintaining compression.

Core Technique

Mentalese Format: A symbolic reasoning language more compact than natural language. For example, instead of reasoning in English prose, express reasoning as a series of symbolic operations:

[assign:x=5] [op:multiply:x:2:->y] [verify:y==10] [conclude:true]

This compact format reduces token count while maintaining reasoning structure.

Shorter Length Preference Optimization (SLPO): Standard reward models might over-penalize long but necessary solutions. SLPO uses an adaptive formulation:

reward = accuracy_bonus - λ(length) * penalty

where λ varies based on solution validity. Only penalize length for correct solutions.

Two-Stage Training:

1. SFT: Teach models to output Mentalese reasoning traces
2. RLVR + SLPO: Apply verifier rewards + length penalties, adapting penalties to avoid failing valid long solutions

Implementation

Mentalese template design: Define symbolic operators for your domain:

• Assignment/computation: [op:action:args:->var]
• Verification: [verify:condition]
• Reasoning: [assume:statement]
• Conclusion: [conclude:result]

SFT alignment: Collect or generate reasoning traces in Mentalese format. Train with standard language modeling loss on these traces.

SLPO formulation: Define reward:

def slpo_reward(mentalese_trace, correctness, length):
    acc_bonus = 1.0 if correctness else -0.5
    if correctness:
        length_penalty = -0.1 * log(length)  # Gentle penalty for correct long solutions
    else:
        length_penalty = 0  # No penalty for incorrect (already penalized)
     acc_bonus + length_penalty