diffucoder-diffusion-code

name	diffucoder-diffusion-code
title	DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2506.20639
keywords	["Diffusion Models","Code Generation","Non-autoregressive","Reinforcement Learning","Masked Denoising"]
description	Train masked diffusion models for code generation using coupled-GRPO to optimize non-autoregressive generation. Achieves 4.4% improvement on code benchmarks while reducing autoregressive bias.

DiffuCoder: Optimizing Masked Diffusion for Code Generation

Standard autoregressive code models generate left-to-right, committing to each token before seeing what comes next. This is efficient at inference but limits the model's ability to plan globally—if the first tokens are wrong, the model propagates errors forward. Masked diffusion models offer an alternative: denoise all tokens simultaneously across multiple iterations, enabling global planning and potentially better code quality.

However, diffusion models are harder to train and optimize than autoregressive models. DiffuCoder demonstrates that coupled-GRPO, a reinforcement learning method designed specifically for non-autoregressive generation, significantly improves diffusion model code quality. The approach introduces "autoregressive-ness" metrics that measure how much a diffusion model exploits sequential structure, revealing that models can learn to adjust their causality dynamically.

Core Concept

The key insights are:

1. Diffusion models can learn to be semi-autoregressive: Rather than always denoising randomly, models learn when left-to-right sequential decoding is beneficial
2. Coupled sampling improves RL training: Standard policy gradient methods have high variance for non-autoregressive generation; coupled sampling (paired complementary masks) reduces variance without requiring semi-autoregressive decoding
3. Temperature controls both selection and order: In diffusion models, temperature affects not just which token is chosen but also the order tokens are denoised—unlike autoregressive models where it only affects token selection

These insights lead to better training and more efficient sampling.

Architecture Overview

DiffuCoder modifies a masked diffusion model for code with:

• Masked Language Model: Corrupts code tokens randomly; model predicts all simultaneously
• Iterative Refinement: Multiple denoising steps gradually clean up predictions
• Coupled-GRPO Training: Uses complementary mask pairs to reduce RL gradient variance
• Autoregressive-ness Tracking: Monitors when the model transitions between parallel and sequential generation
• Temperature Adjustment: Dynamically set during sampling to balance quality and diversity

Implementation

Step 1: Implement masked diffusion denoising process

Create a model that predicts all tokens simultaneously from corrupted input.

import torch
import torch.nn  nn
 torch.nn.functional  F

 (nn.Module):
    

     ():
        ().__init__()
        .vocab_size = vocab_size
        .hidden_dim = hidden_dim

        
        .token_embed = nn.Embedding(vocab_size, hidden_dim)
        .pos_embed = nn.Embedding(, hidden_dim)  

        
        .layers = nn.ModuleList([
            nn.TransformerEncoderLayer(
                d_model=hidden_dim,
                nhead=,
                dim_feedforward=,
                batch_first=
            )
             _  (num_layers)
        ])

        
        .output_head = nn.Linear(hidden_dim, vocab_size)

     ():
        
        batch_size, seq_length = input_ids.shape

        
        embeddings = .token_embed(input_ids)

        
        positions = torch.arange(seq_length, device=input_ids.device).unsqueeze()
        embeddings = embeddings + .pos_embed(positions)

        
        hidden_states = embeddings
         layer  .layers:
            hidden_states = layer(hidden_states)

        
        logits = .output_head(hidden_states)  

         logits

     ():
        
        current_ids = input_ids.clone()
        confidence_history = []

         iteration  (num_iterations):
            
            logits = .forward(current_ids)
            probs = torch.softmax(logits, dim=-)

            
            predicted_tokens = logits.argmax(dim=-)
            max_probs = probs.(dim=-)[]  

            confidence_history.append(max_probs)

            
            
            uncertainty =  - max_probs
            threshold = uncertainty.quantile(, dim=, keepdim=)  

            mask_next = uncertainty > threshold
            current_ids = current_ids.clone()
            current_ids[mask_next] =   

            
             (max_probs > ).().mean() > :
                

         current_ids, confidence_history