| name | criticality-constrained-snn-pruning |
| category | ai_collection |
| trigger_words | ["SNN pruning","criticality-constrained pruning","CQP pruning","surrogate-gradient criticality","neuromorphic deployment","energy-efficient SNN","continuous-relaxation trap","zombie-weight","criticality cliff","临界性约束剪枝","脉冲神经网络剪枝","神经形态部署"] |
| description | Criticality-Constrained Quadratic Pruning (CQP) methodology for energy-efficient SNN deployment on neuromorphic hardware. Combines weight magnitude with surrogate-gradient criticality into analytically exact importance metric. Identifies continuous-relaxation trap, zombie-weight failure mode, and criticality cliff phenomenon. Achieves 95.6% accuracy at 90% sparsity on MNIST; 73% energy reduction at 70% sparsity.
|
| arxiv_id | 2606.30676 |
| authors | ["Muhammad Hamza"] |
| affiliation | IIT Kharagpur |
| date | 2026-06-26 |
Criticality-Constrained Iterative Pruning for Energy-Efficient SNNs (CQP)
Overview
CQP is a native PyTorch pipeline that fuses weight magnitude with surrogate-gradient criticality into an analytically exact importance metric for SNN pruning. It addresses three critical failure modes in existing SNN pruning approaches:
- Continuous-Relaxation Trap: OSQP-solver fractional masks overshoot intended sparsity by up to 12 percentage points, causing 44pp accuracy collapse upon binarization
- Zombie-Weight Failure Mode: Adam's first-moment tensors resurrect pruned synapses, violating binary sparsity guarantee
- Gradient Staleness at High Sparsity: Criticality scores become outdated as network operates at extreme sparsity levels
Core Methodology
Importance Metric
The CQP importance score combines two signals:
- Weight magnitude |w|: Standard magnitude-based pruning signal
- Surrogate-gradient criticality: Measures how much each synapse contributes to the surrogate gradient flow during backpropagation-through-time (BPTT)
Combined importance: I(w) = α|w| + β·criticality(w)
This analytically exact metric avoids the rounding artifacts endemic to QP-based approaches.
Continuous-Relaxation Trap
Problem: When casting pruning as a Quadratic Program (QP) with continuous relaxation (e.g., via CVXPY/OSQP), the solver produces fractional masks. Binarizing these masks causes them to overshoot the target sparsity level by up to 12pp.
Consequence: This overshoot precipitates a 44pp accuracy collapse at moderate-to-high sparsity levels.
Solution: CQP bypasses QP relaxation entirely, using the analytically exact combined importance metric instead.
Zombie-Weight Remediation
Problem: After pruning synapses to zero, Adam optimizer's first-moment estimate retains momentum from pre-pruning gradients. This "resurrects" pruned weights during fine-tuning.
Solution: Reset Adam's first-moment tensors to zero for pruned synapses, combined with gradient masking to enforce hard sparsity constraints.
Iterative Schedule
The CQP pipeline follows an iterative schedule:
- Prune: Remove synapses below importance threshold
- Fine-tune: Train with gradient masking to enforce sparsity
- Recompute criticality: Update importance scores with current weights
- Repeat: Continue until target sparsity achieved
