functional-ensembles-snn-computation

name

functional-ensembles-snn-computation

description

Functional ensembles as units of computation in deep spiking networks. First-order functionally-connected (1FC) groups based on pairwise correlations, aggregate cofiring predicts downstream responses, ReLU-like input-output relationship with ensemble-size scaling, rare high-coordination events encode information. Activation: functional ensemble, SNN computation, functional connectivity, 1FC group, ensemble cofiring, deep spiking network analysis.

Background & Problem

Understanding how internal representations emerge in hierarchical processing systems is a fundamental challenge in both neuroscience and artificial intelligence. Deep spiking neural networks (SNNs) offer a biologically plausible computational model, but analyzing their information processing mechanisms requires new analytical frameworks.

Key questions:

• How do neurons in deep SNNs coordinate to encode information?
• What computational units emerge from functional connectivity patterns?
• How are representations distributed across network layers?
• Can neuroscience-derived analysis methods reveal computational principles in artificial networks?

Core Methodology: First-Order Functionally-Connected (1FC) Groups

1FC Group Definition

1FC group: For a neuron in layer L+1, the set of neurons in the previous layer L that exhibit statistically significant pairwise correlations with that neuron during inference.

# 1FC ensemble extraction pseudocode
def extract_1FC_group(neuron_i, layer_L_spikes, layer_L_plus1_spikes, threshold=0.05):
    """
    Extract first-order functionally-connected group for neuron i in layer L+1.
    
    Args:
        neuron_i: Target neuron in layer L+1
        layer_L_spikes: Spike trains from previous layer
        layer_L_plus1_spikes: Spike trains from current layer
        threshold: Statistical significance threshold
    
    Returns:
        List of neurons in layer L significantly correlated with neuron_i
    """
    correlations = []
    for neuron_j in range(len(layer_L_spikes)):
        # Compute pairwise correlation
        corr = compute_spike_correlation(
            layer_L_spikes[neuron_j],
            layer_L_plus1_spikes[neuron_i]
        )
        
        # Test statistical significance
        if is_significant(corr, threshold):
            correlations.append({
                'neuron': neuron_j,
                'correlation': corr,
                'layer': L
            })
    
    return correlations

Functional Connectivity Analysis

Key principles preserved from biological cortex:

1. Layer-dependent connectivity: Early and intermediate layers show distinct 1FC patterns
2. Learning-shaped structure: 1FC groups are formed during training, not innate
3. Weight permutation breaks structure: Randomizing weights disrupts functional connectivity

Analysis metrics:

• Ensemble size distribution across layers
• Correlation strength and significance
• Temporal dynamics of cofiring patterns
• Information encoding efficiency

Key Findings: Ensemble Computation Principles

1. Aggregate Cofiring Predicts Downstream Responses

ReLU-like input-output relationship: The aggregate cofiring rate of a 1FC ensemble reliably predicts the downstream neuron's response through a robust ReLU-like function.

Response = max(0, gain × (aggregate_cofiring - threshold))

where:

• gain scales systematically with ensemble size
• threshold determines minimal coordination needed
• Linear regime above threshold, zero below

Properties:

• Robustness: Relationship persists across input conditions
• Scalability: Larger ensembles → higher gain → stronger influence
• Nonlinearity: Threshold introduces computational selectivity

2. Rare Events Encode Information

Critical insight: Reliable encoding of the presented class emerges only during high 1FC cofiring events, which occur infrequently.

P(high_cofiring) << 1    (rare events)
I(information; high_cofiring) >> 0   (highly informative)

Implications:

• Information concentrated in sparse coordinated activity
• Most spikes are "noise" from a computational perspective
• Efficient encoding through rare high-synchrony events
• Contrast with rate coding (distributed across all spikes)

Quantitative patterns:

• High cofiring events: ~5-15% of total firing events
• Information transfer: concentrated in these rare windows
• Temporal clustering: high cofiring events cluster in time

3. Disruption Under Noise and Adversarial Perturbations

Vulnerability signature: 1FC ensemble response profiles are disrupted under uniform random noise or adversarial attacks, particularly in early and intermediate layers.

# Noise vulnerability analysis
def analyze_noise_sensitivity(1FC_ensemble, noise_type='uniform'):
    """
    Analyze how noise perturbations disrupt 1FC ensemble coordination.
    
    Noise types:
        - 'uniform': Random uniform noise injection
        - 'adversarial': Gradient-based adversarial perturbation
    
    Returns:
        Disruption metrics: correlation degradation, threshold shift, gain reduction
    """
    baseline_response = compute_response_curve(1FC_ensemble, clean_input)
    perturbed_response = compute_response_curve(1FC_ensemble, noisy_input)
    
    metrics = {
        'correlation_loss': baseline_response['corr'] - perturbed_response['corr'],
        'threshold_shift': perturbed_response['threshold'] - baseline_response['threshold'],
        'gain_reduction': baseline_response['gain'] / perturbed_response['gain']
    }
    
    return metrics

Layer-specific vulnerability:

• Early layers (L=1-3): High vulnerability, correlation loss > 40%
• Intermediate layers (L=4-6): Moderate vulnerability, gain reduction ~30%
• Late layers (L=7+): Lower vulnerability, more robust ensembles

Targeted diagnostic potential: Disruption patterns enable fine-grained interrogation at specific nodes and pathways.

4. Learning Shapes Functional Structure

Key experiment: Weight permutation (random shuffling of learned weights) breaks 1FC ensemble structure.

Before permutation: Strong 1FC correlations, reliable ensemble responses
After permutation: Weak correlations, disrupted ensemble behavior

Implications:

• Functional connectivity is learned, not predetermined
• Network architecture provides substrate, training creates computational units
• Information processing depends on learned coordination patterns

Applications: Targeted Diagnostics and Analysis

1. Information Flow Diagnostics

def diagnose_information_flow(model, test_input, target_class):
    """
    Use 1FC ensembles to diagnose information flow bottlenecks.
    
    Returns:
        Layer-by-layer ensemble activity, correlation patterns, disruption points
    """
    layers = model.get_layer_outputs(test_input)
    
    diagnostics = []
    for layer_idx in range(1, len(layers)):
        # Extract 1FC groups for this layer
        1FC_groups = extract_all_1FC_groups(layers[layer_idx-1], layers[layer_idx])
        
        # Analyze cofiring events
        cofiring_events = detect_high_cofiring(1FC_groups)
        
        # Check information encoding
        encoding_quality = measure_class_encoding(cofiring_events, target_class)
        
        diagnostics.append({
            'layer': layer_idx,
            'ensemble_count': len(1FC_groups),
            'avg_ensemble_size': mean([len(g) for g in 1FC_groups]),
            'high_cofiring_rate': len(cofiring_events) / total_spikes,
            'encoding_quality': encoding_quality
        })
    
    return diagnostics

2. Adversarial Robustness Assessment

def assess_ensemble_robustness(model, adversarial_examples):
    """
    Evaluate robustness of 1FC ensembles under adversarial attack.
    
    Metrics:
        - Ensemble preservation rate
        - Correlation stability
        - Threshold resilience
    """
    baseline_1FC = extract_all_1FC_groups(model, clean_inputs)
    adversarial_1FC = extract_all_1FC_groups(model, adversarial_examples)
    
    robustness = {
        'ensemble_preservation': compare_ensembles(baseline_1FC, adversarial_1FC),
        'correlation_stability': measure_correlation_shift(baseline_1FC, adversarial_1FC),
        'vulnerable_layers': identify_weak_layers(baseline_1FC, adversarial_1FC)
    }
    
    return robustness

3. Network Architecture Optimization

Principle: Design SNN architectures that promote formation of informative 1FC ensembles.

Architectural considerations:

1. Layer connectivity: Ensure sufficient neurons for ensemble formation
2. Learning dynamics: Training procedures that shape functional connectivity
3. Noise resilience: Robust ensemble structures resistant to perturbation
4. Information concentration: Encourage rare high-coordination events

Implementation Guidelines

Step 1: Spike Train Recording

import numpy as np

def record_spike_trains(model, inputs, duration_ms=1000):
    """
    Record spike trains from all layers during inference.
    
    Returns:
        spike_trains: Dict {layer_idx: ndarray(neurons, time_steps)}
    """
    spike_trains = {}
    
    for layer_idx, layer in enumerate(model.layers):
        spikes = layer.get_spikes(inputs, duration_ms)
        spike_trains[layer_idx] = spikes  # Binary spike matrix
    
    return spike_trains

Step 2: Correlation Computation

from scipy.stats import pearsonr

def compute_spike_correlation(spike_train_1, spike_train_2):
    """
    Compute Pearson correlation between spike trains.
    
    Note: Use appropriate windowing and normalization.
    """
    # Convert to firing rates (windowed)
    window_size = 10  # ms
    rate_1 = compute_firing_rate(spike_train_1, window_size)
    rate_2 = compute_firing_rate(spike_train_2, window_size)
    
    # Compute correlation
    corr, p_value = pearsonr(rate_1, rate_2)
    
    return {'correlation': corr, 'p_value': p_value}

Step 3: Ensemble Detection

def detect_1FC_ensembles(layer_spikes, prev_layer_spikes, significance=0.01):
    """
    Detect all 1FC ensembles in a layer.
    
    Returns:
        ensembles: Dict {neuron_idx: [connected_neurons from prev layer]}
    """
    ensembles = {}
    
    for neuron_i in range(len(layer_spikes)):
        connected = []
        for neuron_j in range(len(prev_layer_spikes)):
            corr_result = compute_spike_correlation(
                prev_layer_spikes[neuron_j],
                layer_spikes[neuron_i]
            )
            
            if corr_result['p_value'] < significance:
                connected.append({
                    'neuron': neuron_j,
                    'correlation': corr_result['correlation']
                })
        
        ensembles[neuron_i] = connected
    
    return ensembles

Step 4: High Cofiring Detection

def detect_high_cofiring_events(ensemble, spike_trains, threshold_factor=2.0):
    """
    Detect rare high-coordination cofiring events.
    
    Args:
        threshold_factor: Multiplier above mean cofiring rate
    
    Returns:
        event_indices: Time steps where high cofiring occurs
    """
    # Compute aggregate cofiring rate
    prev_layer_spikes = spike_trains[ensemble['layer']]
    ensemble_neurons = [c['neuron'] for c in ensemble['connections']]
    
    aggregate_cofiring = np.sum(
        prev_layer_spikes[ensemble_neurons],
        axis=0
    )
    
    # Detect high cofiring events
    mean_rate = np.mean(aggregate_cofiring)
    threshold = mean_rate * threshold_factor
    
    event_indices = np.where(aggregate_cofiring > threshold)[0]
    
    return event_indices

Comparison with Biological Cortex Principles

Principle	Biological Cortex	Deep SNN (1FC Analysis)
Functional ensembles	Cell assemblies, synchronized groups	1FC groups, cofiring neurons
Sparse activity	<5% neurons active at any time	Rare high-cofiring events (~5-15%)
Information concentration	Critical periods, synchrony windows	High cofiring windows encode class
Layer-dependent processing	Hierarchical cortical areas	Early vs intermediate vs late layers
Learning-shaped connectivity	Hebbian plasticity, experience	Training shapes 1FC correlations
Noise vulnerability	Pathological disruption	Weight permutation breaks structure

Theoretical Implications

1. Computational Unit Definition

Functional ensemble as computational unit: Unlike single neurons, 1FC groups serve as the meaningful computational substrate for:

• Input encoding (rare high-cofiring events)
• Information transfer (ReLU-like downstream influence)
• Selectivity (threshold-based activation)

2. Sparse Coding Principle Extension

Traditional sparse coding: Individual neurons fire sparsely (rate coding) Ensemble sparse coding: Ensembles cofire sparsely, not individual neurons

Rate sparse coding: P(neuron_active) << 1
Ensemble sparse coding: P(ensemble_high_cofiring) << 1

3. Gain Modulation Mechanism

Ensemble-size-dependent gain: Larger 1FC groups exert stronger influence on downstream neurons.

Gain(n) = α × n^β
where n = ensemble size, β ≈ 0.7-1.2 (empirical range)

Computational advantage: Adaptive gain based on coordination level, not fixed synaptic weights.

Practical Use Cases

Use Case 1: SNN Training Diagnostics

Problem: How to diagnose training progress in deep SNNs? Solution: Monitor 1FC ensemble formation dynamics.

def monitor_training_progress(model, train_data, epochs):
    """
    Track 1FC ensemble evolution during training.
    
    Metrics:
        - Ensemble formation rate
        - Correlation strengthening
        - Information encoding improvement
    """
    progress = []
    
    for epoch in epochs:
        spike_trains = record_spike_trains(model, train_data)
        ensembles = detect_1FC_ensembles(spike_trains)
        
        encoding_quality = measure_information_encoding(ensembles)
        
        progress.append({
            'epoch': epoch,
            'avg_ensemble_size': mean([len(e) for e in ensembles]),
            'avg_correlation': mean([c['correlation'] for e in ensembles for c in e]),
            'encoding_quality': encoding_quality
        })
    
    return progress

Use Case 2: Adversarial Attack Defense

Strategy: Detect adversarial perturbations by 1FC ensemble disruption.

def adversarial_detector(model, input_sample, threshold=0.3):
    """
    Detect adversarial attacks via ensemble disruption.
    
    Returns:
        is_adversarial: Boolean, attack detected
        confidence: Disruption severity
    """
    baseline_ensembles = model.get_cached_1FC_ensembles()
    current_ensembles = extract_1FC_ensembles(model, input_sample)
    
    disruption = measure_ensemble_disruption(baseline_ensembles, current_ensembles)
    
    is_adversarial = disruption['correlation_loss'] > threshold
    
    return {'is_adversarial': is_adversarial, 'confidence': disruption}

Use Case 3: Layer Vulnerability Assessment

Goal: Identify which layers are most vulnerable to perturbations.

def vulnerability_profile(model, test_set):
    """
    Generate layer-by-layer vulnerability profile.
    
    Returns:
        vulnerability_scores: {layer_idx: vulnerability_metric}
    """
    profile = {}
    
    for layer_idx in range(model.num_layers):
        # Test ensemble stability under noise
        baseline_ensembles = extract_1FC_at_layer(model, test_set, layer_idx, clean=True)
        noise_ensembles = extract_1FC_at_layer(model, test_set, layer_idx, noise=True)
        
        stability = compare_ensemble_stability(baseline_ensembles, noise_ensembles)
        
        profile[layer_idx] = {
            'vulnerability': 1 - stability['preservation_rate'],
            'avg_corruption': stability['corruption_severity']
        }
    
    return profile

Limitations & Considerations

1. Computational Cost

Challenge: Computing pairwise correlations across all neurons is expensive.

Cost(L, L+1) = O(N_L × N_{L+1} × T)
where N = neuron count, T = time steps

Mitigations:

• Sampling-based correlation estimation
• Windowed temporal analysis (reduce T)
• GPU parallelization for large networks

2. Statistical Threshold Selection

Issue: Significance threshold affects ensemble detection sensitivity.

Guidelines:

• Conservative threshold (p < 0.01): Larger, more robust ensembles
• Liberal threshold (p < 0.05): More ensembles, higher noise sensitivity
• Adaptive thresholding based on layer depth

3. Temporal Dynamics

Complexity: 1FC correlations may vary over time (non-stationary).

Extension: Dynamic 1FC groups that change during inference.

def extract_dynamic_1FC(spike_trains, time_windows):
    """
    Extract time-varying 1FC groups.
    
    Returns:
        dynamic_ensembles: {window_idx: ensembles}
    """
    dynamic_ensembles = {}
    
    for window_idx, (t_start, t_end) in enumerate(time_windows):
        window_spikes = spike_trains[:, t_start:t_end]
        dynamic_ensembles[window_idx] = detect_1FC_ensembles(window_spikes)
    
    return dynamic_ensembles

Future Directions

1. Second-Order Functional Connectivity

Extension: 2FC groups (neurons correlated via intermediate 1FC neurons).

2FC: Neurons j and k are 2FC-connected if:
     j ∈ 1FC(i) AND k ∈ 1FC(i) for some i

2. Cross-Modal Ensemble Analysis

Application: Analyze functional ensembles in multi-modal SNNs (vision + language).

3. Neuromorphic Hardware Integration

Goal: Implement 1FC ensemble detection on neuromorphic chips (Loihi, SpiNNaker).

4. Clinical Neuroscience Translation

Bridge: Apply 1FC analysis to biological neural recordings (EEG, fMRI, electrophysiology).

References

• arXiv:2606.00073 - Original paper introducing 1FC ensemble framework
• Neuroscience literature on functional connectivity and cell assemblies
• SNN theory: information encoding, sparse coding, gain modulation

Related Skills

• snn-performance-analysis - SNN performance metrics and evaluation
• spiking-neural-network-analysis - General SNN analysis techniques
• functional-connectivity-brain-network - Brain network functional connectivity methods
• neural-dynamics-analysis - Neural dynamics modeling and analysis

---

arXiv ID: 2606.00073 Authors: Aditi Aravind, Konstantinos Ladakis, Mario Alexios Savaglio, Stelios M. Smirnakis, Maria Papadopouli Categories: cs.NE (Neural and Evolutionary Computing), cs.AI (Artificial Intelligence), cs.LG (Machine Learning) Submitted: 21 May 2026