learning-discover-test-time

name	learning-discover-test-time
title	Learning to Discover at Test Time
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2601.16175
keywords	["test-time-learning","reinforcement-learning","problem-solving","reasoning","discovery"]
description	Apply reinforcement learning at test time to enable language models to continue adapting on specific problems, achieving state-of-the-art results across mathematics, GPU optimization, algorithms, and biology. Use when you need models to discover domain-specific problem-solving strategies without retraining.

Learning to Discover at Test Time

This skill enables language models to apply reinforcement learning during testing on individual problems, allowing models to discover domain-specific reasoning strategies and solve novel problems more effectively.

When to Use

• Mathematical problem solving where discovery of approaches helps
• Algorithm optimization tasks requiring exploration of solution space
• Biology/science problems needing iterative hypothesis testing
• GPU kernel optimization or other technical problem-solving
• Any domain where test-time adaptation improves performance

When NOT to Use

• Simple inference tasks where latency is critical
• Domains where best solution is obvious without exploration
• Tasks requiring immediate response (test-time RL takes multiple steps)
• Scenarios with strict budget on computation per problem

Key Concept

TTT-Discover (Test-Time RL for Discovery) allows models to continue learning during inference on individual test problems. Instead of immediately generating an answer, the model:

1. Explores: Generate candidate solutions or approaches
2. Evaluates: Check validity/correctness using domain-specific feedback
3. Refines: Use successful trajectories to guide next attempts
4. Adapts: Learn domain-specific strategies for that problem

This is like "continuing to train" on each test example, finding problem-specific solutions.

Implementation Pattern

Apply reinforcement learning to problem-solving trajectories at test time:

# Pseudocode for test-time discovery learning
class TestTimeDiscovery:
    def __init__(self, base_llm, reward_function):
        self.llm = base_llm
        self.reward_fn = reward_function  # Problem-specific validation

    def solve_with_discovery(self, problem, max_attempts=5):
        best_trajectory = None
        best_reward = float()

         attempt  (max_attempts):
            
            trajectory = .llm.generate_trajectory(problem)

            
            reward = .reward_fn(trajectory, problem)

             reward > best_reward:
                best_reward = reward
                best_trajectory = trajectory

                
                .update_llm_with_trajectory(trajectory, reward)

         best_trajectory

     ():
        
        
        .llm.add_to_context(trajectory, reward)