| name | local-cycles-rnn-computational-ability |
| description | Identifying structural design principles (local cycles) that shape computational abilities of recurrent neural networks. Found that 2- and 3-cycles strongly enhance computational power, and biologically-inspired interneurons dramatically increase capacity. |
| version | 1.0.0 |
| author | Tom Talpir, Elad Schneidman |
| arxiv_id | 2606.23874 |
| submitted_date | 2026-06-22T00:00:00.000Z |
| subjects | q-bio.NC, cs.NE |
| keywords | recurrent neural networks, computational ability, structural design principles, local cycles, interneurons, connectivity, Boolean functions, network architecture |
| activation_words | ["RNN computational ability","neural network structure","local cycles","network connectivity","interneurons","RNN architecture design","computational capacity"] |
Identifying Structural Design Principles Shaping the Computational Abilities of Recurrent Neural Networks
Paper Information
- arXiv ID: 2606.23874
- Authors: Tom Talpir, Elad Schneidman
- Submitted: 22 Jun 2026
- Subjects: Neurons and Cognition (q-bio.NC); Neural and Evolutionary Computing (cs.NE)
- PDF: https://arxiv.org/pdf/2606.23874
Abstract
Understanding how the architecture of neural networks shapes the computations they carry is a central challenge in neuroscience and machine learning. While specific circuit architectures have been linked to particular network computations and theoretical bounds on expressivity of broad classes of networks have been found, we are still missing general principles connecting the structure of finite networks to their computational capabilities.
Core Findings
1. Complete Catalogs of Network-Function Performance
- Trained large collection of different networks to compute large set of Boolean functions
- For small networks, constructed complete "catalogs" revealing computational capacity varies widely
- Most networks show poor performance; most functions are hard to compute
2. Local Cycles as Design Principles
- 2- and 3-cycles strongly enhance computational ability
- Networks with such cycles are often the minimal architectures that can solve particular functions
- Short cycles improved capacity, outperforming acyclic or reachability-matched controls
3. Structural Statistics Predict Performance
- Small set of structural statistics accurately predict networks' performance
- Provides quantitative framework linking connectivity to computation
4. Biological Interneurons Enhance Capacity
- Typical large networks fail to approximate randomly selected functions
- Adding small number of sparsely connected biologically-inspired interneurons dramatically increases computational capacity
- Biologically motivated interneuron design outperforms random connectivity
Methodology
- Training Framework: Large collection of different RNN architectures trained on Boolean functions
- Catalog Construction: Complete mapping of network structure → function performance for small networks
- Statistical Analysis: Structural statistics predicting computational performance
