pets-trajectory-allocation-framework

name	pets-trajectory-allocation-framework
title	PETS: Principled Framework for Optimal Trajectory Allocation
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.16745
keywords	["self-consistency","trajectory sampling","budget allocation","Bayesian optimization","reasoning"]
description	Optimize sampling budget allocation for self-consistency inference by treating trajectory allocation as a crowdsourcing problem. Introduce self-consistency rate (agreement with infinite-budget consensus) as optimization target. Offline setting uses Optimistic Knowledge Gradient for Bayesian optimization; online setting uses difficulty grid discretization and greedy allocation. Achieves up to 75% budget reduction (offline) and 55% (online) while maintaining accuracy.

PETS: Efficient Sampling via Principled Trajectory Allocation

Self-consistency inference—aggregating outputs from multiple sampled trajectories—improves LLM reasoning accuracy substantially. However, it requires sampling many trajectories per question, creating a trade-off: allocating fixed budgets uniformly wastes resources on easy questions while starving hard ones.

The challenge is discovering which questions need more samples. Easy questions often achieve consensus quickly; hard questions require more samples to reach the same confidence threshold. Allocating more samples to hard questions improves final accuracy without increasing total budget.

Core Concept

PETS formulates trajectory allocation as a budget optimization problem. The key metric is "self-consistency rate"—the agreement between a finite-budget majority vote and the theoretical consensus from unlimited sampling. This metric measures when a question has been sampled "enough."

PETS provides two algorithms: an offline setting (all questions known upfront) using Bayesian optimization with Knowledge Gradient, and an online setting (sequential question arrivals) using difficulty grid discretization and greedy allocation.

Architecture Overview

• Self-Consistency Rate Metric: Measure agreement between k-sample majority vote and infinite-budget consensus
• Offline Bayesian Optimization: Model uncertainty over per-question difficulty and select questions for additional sampling using Optimistic Knowledge Gradient
• Online Greedy Allocation: Estimate question difficulty from training distribution; allocate budgets greedily upon arrival
• Difficulty Estimator: Predict question difficulty from features (problem length, token count, answer distribution entropy)
• Budget Solver: Given difficulty distribution and total budget, compute per-question allocations

Implementation

Compute self-consistency rate and identify under-sampled questions:

def compute_self_consistency_rate(trajectories_by_question):
    """
    Measure agreement between sampled consensus and infinite-budget consensus.
    trajectories_by_question: dict mapping question_id -> list of (answer, score)
    Returns: dict mapping question_id -> consistency_rate (0.0-1.0)
    """
    consistency_rates = {}

    for qid, trajectories in trajectories_by_question.items():
        answers, scores = zip(*trajectories)

        
         collections  Counter
        answer_counts = Counter(answers)
        infinite_consensus = answer_counts.most_common()[][]

        
        consistency_by_k = {}

         k  [, , , , (trajectories)]:
             k > (trajectories):
                k = (trajectories)

            top_k_answers = answers[:k]
            top_k_majority = Counter(top_k_answers).most_common()[][]

            is_consistent = (top_k_majority == infinite_consensus)
            consistency_by_k[k] = (is_consistent)

        
        consistency_rates[qid] = (consistency_by_k.values()) / (consistency_by_k)

     consistency_rates