criticality-constrained-snn-pruning

name	criticality-constrained-snn-pruning
category	ai_collection
trigger_words	["SNN pruning","criticality-constrained pruning","CQP pruning","surrogate-gradient criticality","neuromorphic deployment","energy-efficient SNN","continuous-relaxation trap","zombie-weight","criticality cliff","临界性约束剪枝","脉冲神经网络剪枝","神经形态部署"]
description	Criticality-Constrained Quadratic Pruning (CQP) methodology for energy-efficient SNN deployment on neuromorphic hardware. Combines weight magnitude with surrogate-gradient criticality into analytically exact importance metric. Identifies continuous-relaxation trap, zombie-weight failure mode, and criticality cliff phenomenon. Achieves 95.6% accuracy at 90% sparsity on MNIST; 73% energy reduction at 70% sparsity.
arxiv_id	2606.30676
authors	["Muhammad Hamza"]
affiliation	IIT Kharagpur
date	2026-06-26

Criticality-Constrained Iterative Pruning for Energy-Efficient SNNs (CQP)

Overview

CQP is a native PyTorch pipeline that fuses weight magnitude with surrogate-gradient criticality into an analytically exact importance metric for SNN pruning. It addresses three critical failure modes in existing SNN pruning approaches:

1. Continuous-Relaxation Trap: OSQP-solver fractional masks overshoot intended sparsity by up to 12 percentage points, causing 44pp accuracy collapse upon binarization
2. Zombie-Weight Failure Mode: Adam's first-moment tensors resurrect pruned synapses, violating binary sparsity guarantee
3. Gradient Staleness at High Sparsity: Criticality scores become outdated as network operates at extreme sparsity levels

Core Methodology

Importance Metric

The CQP importance score combines two signals:

• Weight magnitude |w|: Standard magnitude-based pruning signal
• Surrogate-gradient criticality: Measures how much each synapse contributes to the surrogate gradient flow during backpropagation-through-time (BPTT)

Combined importance: I(w) = α|w| + β·criticality(w)

This analytically exact metric avoids the rounding artifacts endemic to QP-based approaches.

Continuous-Relaxation Trap

Problem: When casting pruning as a Quadratic Program (QP) with continuous relaxation (e.g., via CVXPY/OSQP), the solver produces fractional masks. Binarizing these masks causes them to overshoot the target sparsity level by up to 12pp.

Consequence: This overshoot precipitates a 44pp accuracy collapse at moderate-to-high sparsity levels.

Solution: CQP bypasses QP relaxation entirely, using the analytically exact combined importance metric instead.

Zombie-Weight Remediation

Problem: After pruning synapses to zero, Adam optimizer's first-moment estimate retains momentum from pre-pruning gradients. This "resurrects" pruned weights during fine-tuning.

Solution: Reset Adam's first-moment tensors to zero for pruned synapses, combined with gradient masking to enforce hard sparsity constraints.

Iterative Schedule

The CQP pipeline follows an iterative schedule:

1. Prune: Remove synapses below importance threshold
2. Fine-tune: Train with gradient masking to enforce sparsity
3. Recompute criticality: Update importance scores with current weights
4. Repeat: Continue until target sparsity achieved