signround-v2-low-bit-quant

name	signround-v2-low-bit-quant
title	SignRoundV2: Extremely Low-Bit Post-Training Quantization via DeltaLoss Sensitivity
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.04746
keywords	["quantization","model-compression","low-bit","post-training","layer-wise-bit-allocation"]
description	DeltaLoss sensitivity metric combining gradient and quantization-induced parameter deviation for adaptive bit-width allocation, with lightweight pre-tuning search for scale initialization, enabling competitive accuracy at 4-5 bits in 2.5-6 hours.

Summary

SignRoundV2 presents two main technical innovations for extremely low-bit post-training quantization. The primary contribution is DeltaLoss, a sensitivity metric capturing both local parameter distortions and global impact on task loss for reliable layer-wise bit allocation. The secondary contribution is lightweight pre-tuning search for initialization, enabling stable and accurate quantization at extremely low bit-widths (2-5 bits) in minimal time.

Core Technique

DeltaLoss Sensitivity Metric: Measures importance of each layer by combining:

1. Gradient information (how sensitive loss is to weight changes)
2. Quantization-induced deviation (how much quantization distorts parameters)

DeltaLoss_layer = ||∇_W L|| · ||W - Q(W)||

This captures both local and global impact—sensitive layers get more bits.

Pre-Tuning Scale Search: Before main quantization, perform lightweight search for optimal quantization scale (step size) per layer:

scale_opt = argmin_scale MSE(Q_scale(W), W)

Adaptive Bit-Width Allocation: Use DeltaLoss to allocate bits from a budget:

bits_layer = base_bits + extra_bits[DeltaLoss_rank(layer)]

Critical layers get extra bits; less important layers get fewer.

Implementation

DeltaLoss computation:

def compute_deltaloss_sensitivity(weight, gradient, quant_weight):
    # Gradient magnitude (local sensitivity)
    gradient_norm = torch.norm(gradient.flatten())

    # Quantization deviation (distortion magnitude)
    deviation = torch.norm(weight - quant_weight)

    # Combined sensitivity
    deltaloss = gradient_norm * deviation
    return deltaloss

Pre-tuning scale search:

def search_optimal_scale(weight, bits=4):
    scales = torch.linspace(0.001, 1.0, 100)
    best_scale = None
    best_mse = float()

     scale  scales:
        
        quant_weight = quantize_symmetric(weight / scale, bits) * scale
        mse = torch.mean((weight - quant_weight) ** )

         mse < best_mse:
            best_mse = mse
            best_scale = scale

     best_scale