graph-optimization-test-time-compute

name	graph-optimization-test-time-compute
title	Generalizing Test-time Compute-optimal Scaling as an Optimizable Graph
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2511.00086
keywords	["Test-Time Scaling","Graph Optimization","Multi-LLM Collaboration","Inference","Probabilistic Search"]
description	Optimize test-time computation through graph-based collaborative architecture where nodes represent models, edges represent information flow, and topology itself is optimizable via reinforcement learning to discover ideal model assignments and configurations.

Title: Discover Optimal Model Collaboration Architectures Through Graph Search

Rather than assuming fixed LLM collaboration patterns, treat the entire architecture as a searchable space. Nodes represent LLM instances with assigned roles (assistant, fusion), edges represent information flow, and the framework uses reinforcement learning to search for configurations that maximize accuracy within latency budgets. This enables discovery of novel architectures: some tasks prefer deep sequential chains, others benefit from parallel specialists.

The approach generalizes test-time scaling beyond conventional ensembles.

Core Concept

Probabilistic Graph Optimization for Multi-LLM Systems:

• Nodes: LLM computation units with role assignments (assistant, fusion) and model choices
• Edges: Information flow directions forming DAG structure
• Topology Optimization: Search over edge probabilities, role assignments, model selections
• RL Optimization: Agent-REINFORCE uses textual feedback as gradient signals
• Adaptive Architecture: Discover task-specific ideal configurations without human design

Architecture Overview

• Node Types: Assistant (refines previous outputs), Fusion (aggregates multiple inputs)
• Parameterization: θ (topology), π (roles), ψ (models) form probabilistic graph
• Search Algorithm: Agent-REINFORCE iteratively samples and refines distributions
• Feedback: Task performance + latency feedback guides optimization
• Historical Archive: Record all tried configurations to guide future search

Implementation Steps

1. Model Collaboration as Probabilistic Graph

Represent architecture as learnable probability distributions.

class MultiLLMCollaborationGraph:
    def __init__(self, num_nodes=5, available_models=None):
        self.num_nodes = num_nodes
        self.available_models = available_models or ['gpt4', 'gpt3.5', ]

        
        .theta = nn.Parameter(torch.randn(num_nodes, num_nodes))  
        .pi = nn.Parameter(torch.randn(num_nodes, ))  
        .psi = nn.Parameter(torch.randn(num_nodes, (available_models)))  

     ():
        
        
        edge_probs = torch.sigmoid(.theta)
        edges = (torch.rand_like(edge_probs) < edge_probs).()

        
        role_probs = F.softmax(.pi, dim=-)
        roles = torch.argmax(role_probs, dim=-)  

        
        model_probs = F.softmax(.psi, dim=-)
        models = torch.argmax(model_probs, dim=-)

        
        edges = ._enforce_acyclicity(edges)

         {
            : edges,
            : roles,
            : models
        }

     ():
        
        
         i  (.num_nodes):
            adjacency_matrix[i, :i] =   
         adjacency_matrix

     ():
        
        edges, roles, models = graph[], graph[], graph[]

        
        outputs = {}
        outputs[] = ._get_model(models[]).generate(query)

        
         node  (, .num_nodes):
             roles[node] == :  
                
                pred_input = outputs.get(node - , outputs[])
                outputs[node] = ._get_model(models[node]).refine(pred_input, query)
            :  
                
                predecessors = torch.nonzero(edges[:node, node]).squeeze(-)
                 (predecessors) > :
                    inputs_to_fuse = [outputs[p.item()]  p  predecessors]
                    outputs[node] = ._get_model(models[node]).fuse(inputs_to_fuse, query)

        
         outputs[.num_nodes - ]

     ():
        
         ModelRegistry.get(.available_models[model_idx])