| name | graph-optimization-test-time-compute |
| title | Generalizing Test-time Compute-optimal Scaling as an Optimizable Graph |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2511.00086 |
| keywords | ["Test-Time Scaling","Graph Optimization","Multi-LLM Collaboration","Inference","Probabilistic Search"] |
| description | Optimize test-time computation through graph-based collaborative architecture where nodes represent models, edges represent information flow, and topology itself is optimizable via reinforcement learning to discover ideal model assignments and configurations. |
Title: Discover Optimal Model Collaboration Architectures Through Graph Search
Rather than assuming fixed LLM collaboration patterns, treat the entire architecture as a searchable space. Nodes represent LLM instances with assigned roles (assistant, fusion), edges represent information flow, and the framework uses reinforcement learning to search for configurations that maximize accuracy within latency budgets. This enables discovery of novel architectures: some tasks prefer deep sequential chains, others benefit from parallel specialists.
The approach generalizes test-time scaling beyond conventional ensembles.
Core Concept
Probabilistic Graph Optimization for Multi-LLM Systems:
- Nodes: LLM computation units with role assignments (assistant, fusion) and model choices
- Edges: Information flow directions forming DAG structure
- Topology Optimization: Search over edge probabilities, role assignments, model selections
- RL Optimization: Agent-REINFORCE uses textual feedback as gradient signals
- Adaptive Architecture: Discover task-specific ideal configurations without human design
Architecture Overview
- Node Types: Assistant (refines previous outputs), Fusion (aggregates multiple inputs)
- Parameterization: θ (topology), π (roles), ψ (models) form probabilistic graph
- Search Algorithm: Agent-REINFORCE iteratively samples and refines distributions
- Feedback: Task performance + latency feedback guides optimization
- Historical Archive: Record all tried configurations to guide future search
Implementation Steps
1. Model Collaboration as Probabilistic Graph
Represent architecture as learnable probability distributions.
class MultiLLMCollaborationGraph:
def __init__(self, num_nodes=5, available_models=None):
self.num_nodes = num_nodes
self.available_models = available_models or ['gpt4', 'gpt3.5', ]
.theta = nn.Parameter(torch.randn(num_nodes, num_nodes))
.pi = nn.Parameter(torch.randn(num_nodes, ))
.psi = nn.Parameter(torch.randn(num_nodes, (available_models)))
():
edge_probs = torch.sigmoid(.theta)
edges = (torch.rand_like(edge_probs) < edge_probs).()
role_probs = F.softmax(.pi, dim=-)
roles = torch.argmax(role_probs, dim=-)
model_probs = F.softmax(.psi, dim=-)
models = torch.argmax(model_probs, dim=-)
edges = ._enforce_acyclicity(edges)
{
: edges,
: roles,
: models
}
():
i (.num_nodes):
adjacency_matrix[i, :i] =
adjacency_matrix
():
edges, roles, models = graph[], graph[], graph[]
outputs = {}
outputs[] = ._get_model(models[]).generate(query)
node (, .num_nodes):
roles[node] == :
pred_input = outputs.get(node - , outputs[])
outputs[node] = ._get_model(models[node]).refine(pred_input, query)
:
predecessors = torch.nonzero(edges[:node, node]).squeeze(-)
(predecessors) > :
inputs_to_fuse = [outputs[p.item()] p predecessors]
outputs[node] = ._get_model(models[node]).fuse(inputs_to_fuse, query)
outputs[.num_nodes - ]
():
ModelRegistry.get(.available_models[model_idx])
