weak-driven-learning-logit-mixing

name	weak-driven-learning-logit-mixing
title	Weak-Driven Learning: How Weak Agents Make Strong Agents Stronger
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.08222
keywords	["Knowledge Distillation","Logit Mixing","Fine-Tuning","Hard Negatives","Training Saturation"]
description	Break through supervised fine-tuning saturation by mixing logits from weaker model checkpoints into strong model training targets. Amplifies hard negatives that strong models have already suppressed, enabling continued learning after standard training plateaus.

Weak-Driven Learning: Leveraging Weak Models to Improve Strong Models

Standard supervised fine-tuning for language models hits saturation: target logits plateau while non-target logits stop declining. The strong model has already learned simple patterns and no longer benefits from standard training signals. Weak-driven learning inverts traditional knowledge distillation—instead of learning from a superior teacher, the strong model learns from its own earlier, weaker checkpoints.

Weak models assign non-negligible probability to plausible-but-incorrect alternatives that strong models have already suppressed. By mixing weak model logits into training targets, you reintroduce uncertainty and force the strong model to maintain harder distinctions even after standard training stagnates.

Core Concept

Standard fine-tuning: min ||logits(x) - logits_correct||^2. This saturates because target logits converge to large values, non-target gradients vanish.

Weak-driven learning: mix weak and strong logits as target:

logits_mix(x) = λ·logits_strong(x) + (1-λ)·logits_weak(x)

Train strong model to match this mixture. The weak model's high probability on incorrect alternatives creates "hard negatives" the strong model must actively suppress.

Architecture Overview

• Weak Reference Model: Earlier strong checkpoint or smaller variant
• Logit Mixing: Combine weak and strong logits with tunable weight λ
• Curriculum Learning: Select harder examples (where weak and strong disagree most)
• Joint Training: Update strong model with mixed-logit targets
• Calibration: Mixture weight λ balances signal strength and training stability

Implementation

Implement logit mixing for training:

import torch
import torch.nn.functional as F

def compute_mixed_logits(strong_logits, weak_logits, lambda_weight=0.3):
    """Mix weak and strong logits to create training targets."""
    mixed = lambda_weight * strong_logits + (1 - lambda_weight) * weak_logits
    return mixed

def compute_weak_driven_loss(strong_logits, weak_logits, labels, lambda_weight=, temperature=):
    
    
    mixed_logits = compute_mixed_logits(strong_logits, weak_logits, lambda_weight)

    
    strong_logits_scaled = strong_logits / temperature
    mixed_logits_scaled = mixed_logits.detach() / temperature

    
    loss = F.kl_div(
        F.log_softmax(strong_logits_scaled, dim=-),
        F.softmax(mixed_logits_scaled, dim=-),
        reduction=
    )

     loss


 ():
    
    inputs, labels = batch

    
     torch.no_grad():
        weak_logits = weak_model(inputs).logits

    strong_logits = strong_model(inputs).logits

    
    loss = compute_weak_driven_loss(strong_logits, weak_logits, labels, lambda_weight)

    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

     loss.item()