Framework for functional dissociation between cortical and subcortical systems during learning under memory constraints - cortex supports general structure learning while subcortex specializes in reward-based learning
Framework for functional dissociation between cortical and subcortical systems during learning under memory constraints - cortex supports general structure learning while subcortex specializes in reward-based learning
Cortex and Subcortex Memory-Constrained Learning Framework
Overview
This skill provides a theoretical framework for understanding the functional dissociation between cortical and subcortical systems during learning under memory constraints. The framework demonstrates that when cortical memory resources are limited, the cortex supports general structure learning while subcortical circuits specialize in reward-based learning.
The brain integrates flexible, computationally expensive cortical processing with simpler, lower-cost subcortical mechanisms
Memory constraints on cortical resources naturally give rise to different learning strategies
Model-based (cortical) and model-free (subcortical) modules learn in tandem
Memory Constraint Framework:
Memory Budget M = {State Representations × Resolution × Temporal Depth}
When M < M_required:
- Cortical system focuses on general structure
- Subcortical system handles reward exploitation
2. Strategic Memory Allocation
Key Finding: When rewarded states change often, it's advantageous for the model-based module to focus memory resources on capturing general structure of the environment, rather than exploiting current rewards.
Memory Allocation Strategies:
Exploitation-Focused: Allocate memory to current high-reward states
Structure-Focused: Allocate memory to environmental structure regardless of current rewards
Mixed Strategy: Balance between exploitation and structure
Optimal Strategy Selection:
defoptimal_strategy(reward_change_rate, memory_budget):
"""
Determine optimal memory allocation strategy
Parameters:
- reward_change_rate: Frequency of reward state changes
- memory_budget: Available cortical memory resources
Returns:
- strategy: 'structure_focused' | 'exploitation_focused' | 'mixed'
"""if reward_change_rate > threshold:
memory_budget > M_threshold:
:
classMemoryConstrainedMBMF:
def__init__(self, memory_budget, reward_change_rate):
self.memory_budget = memory_budget
self.reward_change_rate = reward_change_rate
defallocate_memory(self, states, rewards):
"""
Allocate limited memory to states
Key insight: Under high reward change rate,
allocate memory to structural states, not high-reward states
"""ifself.reward_change_rate > threshold:
# Structure-focused allocation
memory_states = identify_structural_states(states)
else:
# Exploitation-focused allocation
memory_states = identify_high_reward_states(states, rewards)
return memory_states[:self.memory_budget]
Memory Constraint Parameterization
Memory Budget Measurement:
M_cortex = N_states × Resolution × Temporal_window
Where:
- N_states: Number of representable states
- Resolution: Granularity of state representation
- Temporal_window: Depth of temporal predictions
Constraint Impact:
When $M_{required} > M_{cortex}$: Specialization emerges
When $M_{required} \leq M_{cortex}$: Flexible integration possible
Experimental Validation
Hypotheses Testable in Experimental Data
H1: Cortical Generalization Under Constraint
Cortex represents environmental structure rather than specific reward contingencies
Test: Examine cortical representations across reward schedule changes
H2: Subcortical Reward Specialization
Subcortical activity correlates with current reward values
Test: Compare subcortical responses to reward changes vs. structural changes
H3: Memory Constraint Effects
Memory limitation forces functional dissociation
Test: Manipulate memory demands and observe cortical-subcortical balance
H4: Reward Change Rate Interaction
High reward change rates shift cortical focus to structure
Test: Compare learning across different reward stability conditions
Neural Signature Predictions
Cortical Activity Patterns:
Stable across reward contingencies
Correlated with structural features (transitions, state topology)
Higher in novel/restructured environments
Subcortical Activity Patterns:
Tracks current reward values
Rapid updates with reward changes
Higher in stable reward environments
fMRI/EEG Markers:
Cortical markers:
├── Prefrontal cortex: Environmental model building
├── Hippocampus: Structural memory encoding
└── Posterior parietal: State space representation
Subcortical markers:
├── Striatum: Reward prediction
├── Amygdala: Reward value encoding
└── Dopaminergic system: Reward learning signals
Implementation Guidelines
1. Computational Model Implementation
import numpy as np
classMemoryConstrainedLearningModel:
"""
Implementation of cortex-subcortex learning framework
"""def__init__(self, n_states, n_actions, memory_budget,
reward_change_rate=0.1):
self.n_states = n_states
self.n_actions = n_actions
self.memory_budget = memory_budget
self.reward_change_rate = reward_change_rate
# Model-based (cortical) systemself.mb_transition_model = {}
self.mb_memory_states = set()
# Model-free (subcortical) system self.mf_q_values = np.zeros((n_states, n_actions))
defupdate_mb_memory(self, states_visited):
"""
Update cortical memory allocation
Strategy depends on reward change rate:
- High change: Store structural states
- Low change: Store high-reward states
"""ifself.reward_change_rate > 0.5:
# Structure-focused: states with high transition variability
structural_states = self.identify_structural_states(states_visited)
self.mb_memory_states.update(structural_states)
else:
# Exploitation-focused: states with high average reward
reward_states = self.identify_reward_states(states_visited)
self.mb_memory_states.update(reward_states)
# Enforce memory budgetiflen(self.mb_memory_states) > self.memory_budget:
self.mb_memory_states = set(list(self.mb_memory_states)[:self.memory_budget])
defidentify_structural_states(self, states):
"""
Identify states important for environmental structure
Criteria:
- High branching factor (many possible next states)
- Transition uncertainty
- Central position in state graph
"""
structural_scores = {}
for s in states:
branching = len(self.mb_transition_model.get(s, {}))
uncertainty = self.compute_transition_uncertainty(s)
centrality = self.compute_state_centrality(s, states)
structural_scores[s] = branching + uncertainty + centrality
returnsorted(structural_scores.keys(),
key=lambda x: structural_scores[x], reverse=True)
defcompute_mb_value(self, state):
"""
Compute model-based value with memory constraint
Only uses stored transition model for memory states
"""if state inself.mb_memory_states:
# Full model-based computationreturnself.full_mb_value(state)
else:
# Reduced model-based or rely on model-freereturnself.reduced_mb_value(state)
defupdate_mf_values(self, state, action, reward, next_state):
"""
Update model-free (subcortical) Q-values
Standard RL update without memory constraints
"""
td_error = reward + self.gamma * max(self.mf_q_values[next_state]) - \
self.mf_q_values[state, action]
self.mf_q_values[state, action] += self.alpha * td_error
defdecide_action(self, state):
"""
Combine MB and MF systems weighted by memory availability
"""
mb_value = self.compute_mb_value(state)
mf_value = self.mf_q_values[state]
# Weight depends on memory state inclusion
weight_mb = 1.0if state inself.mb_memory_states else0.3
combined_value = weight_mb * mb_value + (1 - weight_mb) * mf_value
return np.argmax(combined_value)
2. Experimental Analysis Framework
defanalyze_cortical_subcortical_dissociation(neural_data, behavioral_data):
"""
Analyze neural data for cortical-subcortical dissociation
Parameters:
- neural_data: fMRI/EEG recordings with region labels
- behavioral_data: Learning trajectories, reward schedules
Returns:
- dissociation_score: Evidence for functional separation
"""# Extract cortical vs subcortical activity
cortical_activity = neural_data.filter_regions(['PFC', 'HC', 'PPC'])
subcortical_activity = neural_data.filter_regions(['striatum', 'amygdala'])
# Test cortical stability across reward changes
reward_stability_score = test_cortical_stability(
cortical_activity, behavioral_data.reward_changes
)
# Test subcortical reward tracking
reward_tracking_score = test_subcortical_reward_correlation(
subcortical_activity, behavioral_data.rewards
)
# Compute dissociation index
dissociation_score = reward_stability_score * reward_tracking_score
return {
'dissociation_score': dissociation_score,
'cortical_stability': reward_stability_score,
'subcortical_reward_tracking': reward_tracking_score
}
Key Applications
1. Understanding Brain Learning Mechanisms
Explains why cortical and subcortical systems have distinct roles
Provides computational framework for memory-constrained learning
Predicts neural activity patterns under different reward schedules
2. AI/ML Memory-Constrained Systems
Design hybrid systems with limited memory budgets
Optimize memory allocation strategies
Combine model-based and model-free learning efficiently
classMemoryConstrainedNeuroAI:
"""
NeuroAI architecture implementing cortex-subcortex framework
"""def__init__(self, memory_budget):
self.cortical_module = CorticalModule(memory_budget)
self.subcortical_module = SubcorticalModule()
deflearn(self, environment):
"""
Learning with memory-constrained cortical + unlimited subcortical
"""for episode in environment:
# Cortical: Build structure model with limited memoryself.cortical_module.update_structure(episode)
# Subcortical: Learn reward associationsself.subcortical_module.update_rewards(episode)
# Combine for decision
action = self.combine_modules(episode.state)
Theoretical Extensions
1. Multi-Level Memory Constraints
Hierarchical Memory Budgets:
L1 (Primary Cortex): Highest resolution, smallest budget
L2 (Secondary Cortex): Medium resolution, medium budget
L3 (Association Cortex): Low resolution, large budget
Each level specializes in different structural features
2. Dynamic Memory Reallocation
Adaptive Strategy:
Memory allocation can shift based on environmental demands
Rapid reward changes → structure focus
Stable rewards → exploitation focus
3. Developmental Trajectories
Memory Budget Growth:
Early development: Limited cortical memory, heavy subcortical reliance
metrics = {
'cortical_stability_index': 'Correlation of cortical activity across reward changes',
'subcortical_reward_index': 'Correlation of subcortical activity with reward value',
'memory_utilization': 'Number of states in cortical representation',
'learning_efficiency': 'Performance per memory unit'
}
Connections to Other Frameworks
Related Skills
[[agent-memory-framework]]: AI memory architecture
Simplistic cortical-subcortical mapping: Real systems have complex interactions beyond this model
Limitations
Model is theoretical; experimental validation ongoing
Memory budget quantification is simplified
Real cortical-subcortical interactions more complex
Best Practices
Parameter Estimation:
Estimate memory budget empirically from neural capacity
Measure reward change rate from behavioral data
Model Comparison:
Compare with standard MB-MF models
Test predictions in memory-manipulated experiments
Neural Validation:
Use multi-region recordings to test dissociation
Compare cortical stability vs subcortical reward tracking
Future Directions
Open Questions
How does memory budget vary across individuals?
Can memory reallocation be dynamically controlled?
What triggers transition from structure-focused to exploitation-focused?
Research Extensions
Computational: Implement in deep RL architectures
Neural: Test predictions in animal learning experiments
Clinical: Apply to understanding learning disorders
References
Primary Source
arXiv:2606.00667 - Farrell & Toyoizumi (2026)
Related Literature
Daw et al. (2011): Model-based vs model-free arbitration
Keramati et al. (2011): Speed-accuracy tradeoff in MB-MF
Kool et al. (2018): Cognitive resource demands in MB learning
Summary
This framework provides a theoretical foundation for functional dissociation between cortical and subcortical learning systems under memory constraints. Key prediction: cortex specializes in general structure learning while subcortex handles reward-based learning when cortical memory is limited. Testable through neural recordings comparing cortical stability and subcortical reward tracking.
