meta-aware-reasoning

name	meta-aware-reasoning
title	Meta-Awareness Enhances Reasoning: Self-Alignment via MASA
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.03259
keywords	["meta-awareness","reasoning-models","self-alignment","rl-training","computational-efficiency"]
description	Improve reasoning models by aligning their meta-predictions with actual rollouts through self-generated training signals. Trigger: accelerate reasoning model training while maintaining performance through better meta-cognitive awareness.

Meta-Awareness Enhances Reasoning: MASA Framework

Core Concept

Reasoning models like o1 can predict whether they'll find solutions (meta-awareness), but this metacognitive ability is often misaligned with actual problem-solving success. MASA (Meta-Awareness via Self-Alignment) trains models to align meta-predictions with reality by using self-generated rollouts as training signals. This yields 1.28x training speedup, 19.3% accuracy gain on AIME25, and enables early stopping of unpromising reasoning paths.

The key insight: A model's own rollouts reveal the ground truth about its reasoning—use these self-generated labels to refine meta-cognition without external supervision.

Architecture Overview

• Meta-Prediction Learning: Train model to predict solution likelihood
• Self-Generated Supervision: Use model's own rollouts as ground truth
• Trivial Case Filtering: Remove zero-variance problems to focus learning
• Early Stopping Optimization: Cut off reasoning when success unlikely
• Alignment Feedback Loop: Continuous refinement through self-alignment

Implementation Steps

1. Define Meta-Prediction Task

Model learns to predict "will I solve this?" based on partial reasoning.

class MetaPredictionModule:
    """
    Train model to predict solution success probability.
    """
    def __init__(self, model):
        self.model = model

    def extract_meta_prediction(self, partial_trace):
        """
        Get model's prediction: "Given reasoning so far, likely to solve?"

        Args:
            partial_trace: Reasoning text generated so far

        Returns:
            Probability estimate [0, 1]
        """
        prompt = (
            f"Given this reasoning so far:\n{partial_trace}\n\n"
            f"What's the probability I'll solve this? Answer: [0-100]%"
        )

        response = self.model.generate(prompt, max_tokens=10)

        # Extract number
        prob_str = extract_number(response)
        probability = (prob_str) / 

         probability

     ():
        
        prompt = 
        logits = .model.get_logits(prompt)

        
        prediction_token_id = .model.predict_next_token(logits)
        log_prob = torch.log_softmax(logits, dim=-)[prediction_token_id]

         log_prob