ember-autonomous-cognitive-behaviour-learned-spiking

name	ember-autonomous-cognitive-behaviour-learned-spiking
description	EMBER: Autonomous cognitive behaviour from learned spiking neural network dynamics. Self-organizing SNN agents with intrinsic motivation, curiosity, and goal-directed behavior emerging from plastic recurrent connectivity without external reward shaping. Keywords: autonomous cognition, intrinsic motivation, SNN agents, emergent behavior, self-organization, curiosity-driven learning.

EMBER: Autonomous Cognitive Behaviour from Learned Spiking Neural Network Dynamics

EMBER (Emergent Mind Through Biological Event-driven Responses) - self-organizing autonomous agents where complex cognitive behaviors emerge from learned spiking neural network dynamics through intrinsic motivation, curiosity, and plastic recurrent connectivity without external reward engineering.

Metadata

• Source: arXiv:2604.12167v1
• Authors: [Authors from paper]
• Published: 2026-04-14
• Category: Neural and Evolutionary Computing (cs.NE), Artificial Intelligence (cs.AI)

Core Methodology

Key Innovation

EMBER presents a paradigm shift from reward-engineered to self-organizing autonomous agents. Rather than designing reward functions for specific behaviors, EMBER agents develop cognitive capabilities through:

• Intrinsic Motivation: Curiosity, novelty-seeking, and competence drives
• Plastic Recurrent Connectivity: Self-organizing RNN/SNN with local learning rules
• Emergent Goal-Directedness: Goals arise from internal state rather than external specification

Technical Framework

1. Intrinsic Motivation Architecture

Novelty-Based Motivation:

R_novelty(s) = -log p(s | history)

High novelty reward for unexpected states.

Competence-Based Motivation:

R_competence(s, a) = ||s_target - s_actual|| - ||s_target - s_predicted||

Reward for successfully controlling outcomes.

Information Gain Motivation:

R_info = H(s_future) - H(s_future | a)

Reward for actions that reduce uncertainty.

2. Spiking Recurrent Network with Plasticity

Recurrent SNN Structure:

• Input neurons: Sensory encoding
• Recurrent excitatory pool: Working memory, sequence processing
• Inhibitory interneurons: Gain control, competition
• Motor output: Action selection

Local Learning Rules:

• Hebbian plasticity: "Neurons that fire together wire together"
• Homeostatic regulation: Maintains firing rate stability
• Meta-plasticity: Learning rate modulation by success

3. Self-Organized Behavior Generation

Goal Emergence:

• Goals as attractor states in recurrent dynamics
• No explicit goal encoding - goals are high-value internal states
• Goal switching through attractor basin hopping

Action Selection:

• Intrinsic motivation guides exploration
• Learned predictions guide exploitation
• Balance through uncertainty-weighted sampling

Key Findings

1. Emergent Cognitive Behaviors

• Foraging: Agents self-organize efficient resource gathering
• Tool Use: Spontaneous discovery of environmental affordances
• Social Behaviors: Communication and coordination in multi-agent scenarios
• Planning: Multi-step action sequences emerge from predictive dynamics

2. Robustness to Environment Changes

• Adapts to novel environments without retraining
• Generalizes across task domains
• Resilient to sensor/actuator failures

3. Scalability

• Behavior complexity scales with network size
• Modular architecture enables hierarchical cognition
• Parallel exploration in multi-agent settings

Implementation Guide

Prerequisites

• Python 3.8+
• snnTorch or Norse for SNN simulation
• Gymnasium/OpenAI Gym for environments
• PyTorch for network components

Step-by-Step Implementation

Step 1: Intrinsic Motivation Module

import torch
import torch.nn as nn
import numpy as np

class IntrinsicMotivationModule:
    """
    Compute intrinsic rewards for autonomous exploration
    """
    def __init__(self, state_dim, memory_size=10000, novelty_decay=0.99):
        self.state_dim = state_dim
        self.memory = []  # Buffer of experienced states
        self.memory_size = memory_size
        self.novelty_decay = novelty_decay
        
        # Forward model for prediction errors
        self.forward_model = self._build_forward_model()
        
        # State distribution model (for novelty)
        self.state_density = OnlineKernelDensity(state_dim)
    
    def _build_forward_model(self):
        """Build forward dynamics model"""
        return nn.Sequential(
            nn.Linear(self.state_dim + self.action_dim, 256),
            nn.ReLU(),
            nn.Linear(256, self.state_dim)
        )
    
    def compute_novelty(self, state):
        """
        Compute novelty reward as negative log likelihood
        
        Args:
            state: Current state (batch, state_dim)
        
        Returns:
            novelty: Scalar novelty score
        """
        # Estimate probability density
        prob = .state_density.estimate(state)
        novelty = -torch.log(prob + )
        
         novelty
    
     ():
        
        
        input_sa = torch.cat([state, action], dim=-)
        predicted_next = .forward_model(input_sa)
        
        
        error = torch.mean((next_state - predicted_next) ** , dim=-)
        
         error
    
     ():
        
        
        n_samples = 
        entropies = []
        
         _  (n_samples):
            
            next_state = .forward_model(torch.cat([state, action]))
            next_state += torch.randn_like(next_state) *   
            
            
            prob = .state_density.estimate(next_state)
            entropy = -torch.log(prob + )
            entropies.append(entropy)
        
        
        info_gain = torch.mean(torch.stack(entropies))
        
         info_gain
    
     ():
        
        
        novelty = .compute_novelty(next_state)
        
        
        pred_error = .compute_prediction_error(state, action, next_state)
        
        
        info_gain = .compute_information_gain(state, action)
        
        
        total_reward = novelty +  * pred_error +  * info_gain
        
        components = {
            : novelty.item(),
            : pred_error.item(),
            : info_gain.item()
        }
        
        
        .update_memory(next_state)
        
         total_reward, components
    
     ():
        
        .memory.append(state.detach().cpu())
         (.memory) > .memory_size:
            .memory.pop()
        
        
        .state_density.update(state)

 :
    
     ():
        .dim = dim
        .bandwidth = bandwidth
        .samples = []
        .max_samples = 
    
     ():
        
        .samples.append(sample.detach().cpu().numpy())
         (.samples) > .max_samples:
            .samples = .samples[-.max_samples:]
    
     ():
        
         (.samples) < :
             torch.ones(query.shape[], device=query.device) * 
        
        query_np = query.detach().cpu().numpy()
        samples_np = np.array(.samples)
        
        
        probs = []
         q  query_np:
            distances = np.linalg.norm(samples_np - q, axis=)
            kernel_vals = np.exp(-distances** / ( * .bandwidth**))
            prob = np.mean(kernel_vals)
            probs.append(prob)
        
         torch.tensor(probs, device=query.device, dtype=torch.float32)

Step 2: Recurrent SNN with Plastic Connectivity

import snntorch as snn
from snntorch import surrogate

class PlasticRecurrentSNN(nn.Module):
    """
    Recurrent SNN with biologically plausible plasticity
    """
    def __init__(self, input_size, hidden_size, output_size, 
                 recurrent_size=256, beta=0.9):
        super().__init__()
        
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.recurrent_size = recurrent_size
        
        # Input encoding
        self.input_encoder = nn.Linear(input_size, recurrent_size)
        
        # Recurrent layer with excitatory and inhibitory populations
        self.rec_exc = snn.Leaky(beta=beta, init_hidden=True)
        self.rec_inh = snn.Leaky(beta=beta, init_hidden=True)
        
        # Recurrent weights (plastic)
        self.w_ee = nn.Parameter(torch.randn(recurrent_size, recurrent_size) * 0.01)
        self.w_ei = nn.Parameter(torch.randn(recurrent_size, recurrent_size) * 0.01)
        self.w_ie = nn.Parameter(torch.randn(recurrent_size, recurrent_size) * 0.01)
        
        # Output readout
        self.readout = nn.Linear(recurrent_size, output_size)
        
        # Surrogate gradient
        self.surrogate = surrogate.fast_sigmoid(slope=25)
        
        # Plasticity parameters
        self.A_plus = 
        .A_minus = 
        .tau_stdp = 
    
     ():
        
        batch_size = x.shape[]
        
        
        input_current = .input_encoder(x)
        
        
        mem_exc = .rec_exc.init_leaky()
        mem_inh = .rec_inh.init_leaky()
        
        
        spikes_exc = []
        spikes_inh = []
        
         t  (time_steps):
            
             t == :
                rec_input_exc = input_current
            :
                prev_exc = spikes_exc[-]  spikes_exc  torch.zeros_like(mem_exc)
                prev_inh = spikes_inh[-]  spikes_inh  torch.zeros_like(mem_inh)
                
                rec_input_exc = (torch.matmul(prev_exc, .w_ee.t()) - 
                                torch.matmul(prev_inh, .w_ie.t()) +
                                input_current)
            
            
            spk_exc, mem_exc = .rec_exc(rec_input_exc, mem_exc)
            
            
            inh_input = torch.matmul(spk_exc, .w_ei.t())
            spk_inh, mem_inh = .rec_inh(inh_input, mem_inh)
            
            spikes_exc.append(spk_exc)
            spikes_inh.append(spk_inh)
        
        
        spike_trains_exc = torch.stack(spikes_exc, dim=)  
        spike_trains_inh = torch.stack(spikes_inh, dim=)
        
        
        rates = spike_trains_exc.mean(dim=)  
        output = .readout(rates)
        
         return_spikes:
             output, (spike_trains_exc, spike_trains_inh)
         output
    
     ():
        
        batch_size, time_steps, n_neurons = spike_trains.shape
        
        
         torch.no_grad():
             b  (batch_size):
                 i  (n_neurons):
                     j  (n_neurons):
                         i == j:
                            
                        
                        
                        times_i = torch.where(spike_trains[b, :, i] > )[].()
                        times_j = torch.where(spike_trains[b, :, j] > )[].()
                        
                         (times_i) ==   (times_j) == :
                            
                        
                        
                        delta_t = times_i.unsqueeze() - times_j.unsqueeze()
                        
                        
                         delta_t > :
                            dw = .A_plus * torch.exp(-delta_t / .tau_stdp)
                        :
                            dw = -.A_minus * torch.exp(delta_t / .tau_stdp)
                        
                        
                        .w_ee.data[i, j] += reward_signal * dw.mean()
            
            
            .w_ee.data = torch.clamp(.w_ee.data, =)
            .w_ei.data = torch.clamp(.w_ei.data, =)
            .w_ie.data = torch.clamp(.w_ie.data, =)

Step 3: Autonomous Agent

class EMBERAgent:
    """
    EMBER autonomous agent with self-organized behavior
    """
    def __init__(self, state_dim, action_dim, env, 
                 network_size=256, learning_rate=0.001):
        self.state_dim = state_dim
        self.action_dim = action_dim
        self.env = env
        
        # Intrinsic motivation
        self.motivation = IntrinsicMotivationModule(state_dim)
        
        # Spiking network
        self.network = PlasticRecurrentSNN(
            state_dim, network_size, action_dim, network_size
        )
        
        # Optimizer for network weights
        self.optimizer = torch.optim.Adam(
            self.network.parameters(), lr=learning_rate
        )
        
        # Experience buffer
        self.buffer = []
        self.buffer_size = 10000
    
    def select_action(self, state, epsilon=0.1):
        """
        Select action using network output
        
        Args:
            state: Current state
            epsilon: Exploration rate
        
        Returns:
            action: Selected action
        """
        state_t = torch.FloatTensor(state).unsqueeze(0)
        
        if np.random.random() < epsilon:
            # Random exploration
            return self.env.action_space.sample()
        
        with torch.no_grad():
            action_logits, _ = self.network(state_t)
            action_probs = torch.softmax(action_logits, dim=-)
            action = torch.multinomial(action_probs, ).item()
        
         action
    
     ():
        
         (.buffer) < batch_size:
             
        
        
        batch = np.random.choice(.buffer, batch_size, replace=)
        
        states = torch.FloatTensor([t[]  t  batch])
        actions = torch.LongTensor([t[]  t  batch])
        next_states = torch.FloatTensor([t[]  t  batch])
        
        
        intrinsic_rewards = []
         s, a, ns  (states, actions, next_states):
            a_onehot = torch.zeros(.action_dim)
            a_onehot[a] = 
            r, _ = .motivation.compute_intrinsic_reward(
                s.unsqueeze(), a_onehot.unsqueeze(), ns.unsqueeze()
            )
            intrinsic_rewards.append(r)
        
        intrinsic_rewards = torch.stack(intrinsic_rewards)
        
        
        action_logits, (spikes_exc, spikes_inh) = .network(states)
        
        
        action_probs = torch.softmax(action_logits, dim=-)
        selected_probs = action_probs.gather(, actions.unsqueeze())
        
        loss = -(torch.log(selected_probs) * intrinsic_rewards.detach()).mean()
        
        
        .optimizer.zero_grad()
        loss.backward()
        .optimizer.step()
        
        
        .network.apply_stdp(spikes_exc, intrinsic_rewards.mean().item())
        
         loss.item()
    
     ():
        
        state = .env.reset()
        total_intrinsic_reward = 
        
         step  (max_steps):
             render:
                .env.render()
            
            
            action = .select_action(state)
            
            
            next_state, env_reward, done, info = .env.step(action)
            
            
            state_t = torch.FloatTensor(state).unsqueeze()
            next_state_t = torch.FloatTensor(next_state).unsqueeze()
            action_t = torch.zeros(.action_dim)
            action_t[action] = 
            
            intrinsic_r, components = .motivation.compute_intrinsic_reward(
                state_t, action_t.unsqueeze(), next_state_t
            )
            
            total_intrinsic_reward += intrinsic_r.item()
            
            
            .buffer.append((state, action, next_state))
             (.buffer) > .buffer_size:
                .buffer.pop()
            
            
             step %  == :
                .train_step()
            
            state = next_state
            
             done:
                
        
         total_intrinsic_reward, step + 
    
     ():
        
         episode  (n_episodes):
            
            reward, length = .run_episode()
            
             episode %  == :
                ()
            
            
             episode % eval_interval == :
                eval_reward, eval_length = .evaluate()
                ()
    
     ():
        
        rewards = []
        lengths = []
        
         _  (n_episodes):
            r, l = .run_episode(render=)
            rewards.append(r)
            lengths.append(l)
        
         np.mean(rewards), np.mean(lengths)

Applications

1. Autonomous Robotics

• Self-motivated exploration robots
• Adaptive navigation without task specification
• Lifelong learning agents

2. Artificial Life Simulation

• Virtual creatures with emergent behavior
• Ecosystems with self-organized interactions
• Evolutionary robotics

3. Cognitive Neuroscience Models

• Study of intrinsic motivation in biological systems
• Computational models of curiosity
• Goal-directed behavior emergence

4. Educational AI

• Self-motivated learning systems
• Adaptive tutoring agents
• Curiosity-driven knowledge acquisition

Pitfalls

1. Exploration-Exploitation Balance

• Issue: Pure intrinsic motivation can lead to endless exploration
• Mitigation: Gradually shift to external rewards or competence-based goals

2. Emergence Unpredictability

• Issue: Behaviors may not align with designer intentions
• Mitigation: Constrain environment, provide safe exploration spaces

3. Computational Cost

• Issue: Kernel density estimation is expensive
• Mitigation: Use approximate methods, limit memory size

4. Local Minima in Motivation

• Issue: Agent may get stuck in locally interesting patterns
• Mitigation: Multiple motivation sources, meta-learning

Related Skills

• intrinsic-motivation-rl
• self-organizing-transformer
• neuromodulated-synaptic-plasticity
• brain-inspired-snn-pattern-analysis

References

@article{2026ember,
  title={EMBER: Autonomous Cognitive Behaviour from Learned Spiking Neural Network Dynamics},
  journal={arXiv preprint arXiv:2604.12167},
  year={2026}
}