lopa-lookahead

name	lopa-lookahead
title	LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.16229
keywords	["diffusion-language-model","inference","parallel-decoding","throughput"]
description	Maximize parallelism in diffusion language models by intelligently exploring token filling orders. Spawn multiple candidate branches predicting high-confidence positions, evaluate all branches in one forward pass, and select the branch enabling most future parallelism—increasing tokens-per-forward-pass 4.4× without accuracy loss.

Overview

LoPA addresses a critical bottleneck in diffusion language model (dLLM) inference: limited parallelism due to dependence on Token Filling Order (TFO). Which positions are filled first dramatically affects future prediction confidence and parallelism opportunities. This framework discovers optimal TFOs dynamically.

Core Technique

The key insight is that token filling order significantly impacts future parallelism potential.

Multi-Branch Lookahead Exploration: Instead of committing to a single filling order, explore multiple candidates concurrently.

# Lookahead branch exploration pattern
class LoopaheadDecoding:
    def __init__(self, k_branches=5):
        self.k_branches = k_branches

    def decode_iteration(self, model, unfilled_positions):
        """
        In single forward pass, explore k+1 candidate TFOs:
        - 1 anchor (confidence-driven)
        - k lookahead (top-k alternatives)
        """
        # Phase 1: Anchor branch (standard sampling)
        anchor_confidence = model(anchor_seed)
        anchor_sampled = sample_positions(anchor_confidence)

        # Phase 2: Lookahead branches (concurrent exploration)
        lookahead_samples = []
        for i in range(self.k_branches):
            # Sample from top-k highest-confidence unfilled positions
            top_k_positions = get_top_k_confidence(unfilled_positions, k=i+1)
            lookahead = sample_from(top_k_positions)
            lookahead_samples.append(lookahead)

        # Phase 3: Branch evaluation (single forward pass)
        all_branches = [anchor_sampled] + lookahead_samples
        branch_evaluations = model.evaluate_branches(all_branches)

        return branch_evaluations

Branch Confidence Metric: Evaluate each branch by its average prediction confidence over remaining positions.

def ():
    
    best_branch = 
    best_score = -()

     branch  branches:
        
        remaining_positions = get_unfilled(branch)

         (remaining_positions) == :
            
             branch

        
        future_confidences = model.predict_confidence(branch, remaining_positions)
        branch_score = np.mean(future_confidences)

         branch_score > best_score:
            best_score = branch_score
            best_branch = branch

     best_branch