grinqh-adaptive-quantization-hierarchy

name	grinqh-adaptive-quantization-hierarchy
description	Graded Input-based Quantization Hierarchy for efficient LLM generation. Dynamic precision assignment based on activation magnitudes as computational importance proxy. Unifies quantization and sparsification.
trigger_words	["adaptive quantization","graded precision","LLM efficiency","memory bandwidth","dynamic bit-width"]
version	1
arxiv	2606.23419v1
authors	Jette Oberländer, Jan Finkbeiner, Catherine M. Schöfde Schöfmann, Emre Neftci
date	2026-06-22T00:00:00.000Z
categories	cs.LG, cs.AI

GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation

Core Insight: Use activation magnitudes as computational importance proxy to dynamically assign weight channels to different precision levels during decoding.

Problem Addressed

• Memory bandwidth bottleneck in autoregressive decoding
• Prefill vs. decoding asymmetry - compute-bound vs. memory-bound stages
• Uniform quantization ignores stage-specific needs
• Fixed bit-width limits flexibility

Key Methodology

Graded Quantization Framework

1. Activation Magnitude Proxy: Use activations to estimate computational importance
2. Dynamic Precision Assignment: Assign weight channels to precision levels based on importance
3. Hierarchical Memory Layout: Nested layout for multi-precision storage
4. Unified Quantization + Sparsification: Combine both for decoding acceleration

Implementation Pattern

def grinqh_quantize(weights, activations_history):
    # Compute importance from activation magnitudes
    importance_scores = compute_channel_importance(activations_history)
    
    # Assign graded precision levels
    precision_assignment = assign_precision_hierarchy(
        importance_scores,
        avg_bit_width_target
    )
    
    # Apply channel-wise quantization
    quantized_weights = hierarchical_quantize(
        weights,
        precision_assignment
    )
    
    # Sparsification integration
    sparse_weights = apply_sparsification(quantized_weights)
    
    return sparse_weights, precision_assignment

def assign_precision_hierarchy(importance, target_bits):
    # High-importance → higher precision
    # Low-importance → lower precision / sparsity
    precision_map = {
        'high': 4-bit,    # Critical channels
        'medium': 3-bit, 
        : -bit      
    }
     precision_map