local-cycles-rnn-computational-ability

name	local-cycles-rnn-computational-ability
description	Identifying structural design principles (local cycles) that shape computational abilities of recurrent neural networks. Found that 2- and 3-cycles strongly enhance computational power, and biologically-inspired interneurons dramatically increase capacity.
version	1.0.0
author	Tom Talpir, Elad Schneidman
arxiv_id	2606.23874
submitted_date	2026-06-22T00:00:00.000Z
subjects	q-bio.NC, cs.NE
keywords	recurrent neural networks, computational ability, structural design principles, local cycles, interneurons, connectivity, Boolean functions, network architecture
activation_words	["RNN computational ability","neural network structure","local cycles","network connectivity","interneurons","RNN architecture design","computational capacity"]

Identifying Structural Design Principles Shaping the Computational Abilities of Recurrent Neural Networks

Paper Information

• arXiv ID: 2606.23874
• Authors: Tom Talpir, Elad Schneidman
• Submitted: 22 Jun 2026
• Subjects: Neurons and Cognition (q-bio.NC); Neural and Evolutionary Computing (cs.NE)
• PDF: https://arxiv.org/pdf/2606.23874

Abstract

Understanding how the architecture of neural networks shapes the computations they carry is a central challenge in neuroscience and machine learning. While specific circuit architectures have been linked to particular network computations and theoretical bounds on expressivity of broad classes of networks have been found, we are still missing general principles connecting the structure of finite networks to their computational capabilities.

Core Findings

1. Complete Catalogs of Network-Function Performance

• Trained large collection of different networks to compute large set of Boolean functions
• For small networks, constructed complete "catalogs" revealing computational capacity varies widely
• Most networks show poor performance; most functions are hard to compute

2. Local Cycles as Design Principles

• 2- and 3-cycles strongly enhance computational ability
• Networks with such cycles are often the minimal architectures that can solve particular functions
• Short cycles improved capacity, outperforming acyclic or reachability-matched controls

3. Structural Statistics Predict Performance

• Small set of structural statistics accurately predict networks' performance
• Provides quantitative framework linking connectivity to computation

4. Biological Interneurons Enhance Capacity

• Typical large networks fail to approximate randomly selected functions
• Adding small number of sparsely connected biologically-inspired interneurons dramatically increases computational capacity
• Biologically motivated interneuron design outperforms random connectivity

Methodology

1. Training Framework: Large collection of different RNN architectures trained on Boolean functions
2. Catalog Construction: Complete mapping of network structure → function performance for small networks
3. Statistical Analysis: Structural statistics predicting computational performance