STARS (Spike Tail-Aware Relational Synthesis) - plug-and-play method for ANN-to-SNN Data-Free Knowledge Distillation (DFKD). Augments BN-guided synthesis with Relational Consistency Alignment and Tail-Aware Regularization. Achieves up to 4.6% improvement on CIFAR-10 and 6.7% on CIFAR-100. Activation: SNN knowledge distillation, data-free distillation, ANN-to-SNN conversion, tail-aware regularization, relational consistency, spike threshold dynamics, 无数据蒸馏, 跨模态蒸馏.
license
Complete terms in LICENSE.txt
metadata
{"arxiv_id":"2605.27409","published":"2026-05-28","authors":"Shuhan Ye, Yi Yu, Qixin Zhang, Hui Lu, Jiaming He, Qinggang Zhang, Li Shen, Xudong Jiang","tags":["snn","knowledge-distillation","data-free","ann-to-snn","threshold-crossing","tail-probability","bn-matching","relational-consistency"]}
STARS: Spike Tail-Aware Relational Synthesis for ANN-to-SNN DFKD
Plug-and-play method for ANN-to-SNN data-free knowledge distillation that addresses the fundamental mismatch between ANN-oriented BN matching and SNN threshold-crossing dynamics.
Problem Statement
ANN-to-SNN knowledge distillation helps narrow the performance gap between ANNs and SNNs. However, in practical deployment settings, the original training data are often unavailable. Existing Data-Free Knowledge Distillation (DFKD) methods face a critical limitation:
ANN-oriented constraints (BN statistics matching) primarily regularize mean and variance, which remain under-constrained for SNN students whose responses depend on , not just statistical moments.
threshold-crossing dynamics
Core Innovation: Two Complementary Objectives
STARS augments standard BN-guided synthesis with two key objectives:
1. Relational Consistency Alignment (RCA)
Preserves cross-sample relational consistency between teacher (ANN) and student (SNN):
Ensures relative ordering of samples is maintained
Captures manifold structure that BN matching ignores
Prevents collapse to generic synthetic distributions
2. Tail-Aware Regularization (TAR)
Regularizes threshold-relevant tail probabilities through soft exceedance over teacher-derived thresholds:
Directly targets the dynamics that govern SNN spike generation
Focuses on the distribution tails where threshold crossing occurs
Bridges the gap between continuous ANN outputs and discrete SNN spikes
Key Insight
SNN students require threshold-relevant information that BN statistics (mean/variance) do not capture. The tail probabilities of the distribution determine:
Whether a neuron fires (crosses threshold)
Spike timing (when threshold is crossed)
Firing rate (frequency of threshold crossing)
Standard DFKD focuses on the bulk of the distribution; SNNs care about the tails.
Performance Results
Dataset
Baseline DFKD
STARS
Improvement
CIFAR-10
~85%
~89.6%
+4.6%
CIFAR-100
~60%
~66.7%
+6.7%
Tiny-ImageNet
~45%
~48-50%
+3-5%
STARS consistently improves conventional DFKD baselines and even surpasses several KD methods that use real data.
Scenario: Deploy SNN on neuromorphic hardware without access to original training data
Benefit: Generate synthetic data locally, distill ANN knowledge to SNN
Use case: Medical devices, autonomous drones, IoT sensors
2. Privacy-Preserving Distillation
Scenario: Distill proprietary ANN models to public SNN architectures
Benefit: Transfer knowledge without sharing original datasets
Use case: Model licensing, cross-organization deployment
3. Architecture Search Support
Scenario: Evaluate candidate SNN architectures without training on real data
Benefit: Rapid architecture screening via synthetic data distillation
Use case: SNN design optimization, hardware-aware search
Pitfalls and Solutions
Pitfall 1: Threshold Misalignment
Problem: Teacher-derived thresholds may not match SNN firing thresholds.
Solution: Adaptive threshold estimation using layer-wise membrane potential statistics.
Pitfall 2: Synthetic Data Collapse
Problem: Synthetic batches converge to generic distributions, losing diversity.
Solution: RCA prevents collapse by enforcing relational structure preservation.
Pitfall 3: Scale Mismatch
Problem: ANN output scale differs from SNN membrane potential scale.
Solution: Normalize outputs before threshold estimation, use relative thresholds.
Research Connections
STARS bridges two key domains:
Data-Free Knowledge Distillation: Transfer knowledge without access to original data
SNN Threshold Dynamics: Discrete spike generation governed by threshold crossing
The intersection reveals that SNN-specific distillation requires tail-aware optimization, not just mean/variance matching.
Related Skills
circulate-firing-snn-training - Direct SNN training with enhanced neurons
