emergent-generalization-representation-learning

name

emergent-generalization-representation-learning

description

Emergent generalization by representation learning in artificial neural networks. An explicit information bottleneck forcing an RNN to learn a low-dimensional representation is necessary for rotational and out-of-distribution generalization in time-series prediction. Uses information-theoretic causal emergence to characterize the memorization-to-generalization transition (non-monotonic down-min-up trajectory) and finds analogous dynamics in CA1 hippocampal activity of mice learning an alternating maze. Supports a causal role for learned representations in cognition. Activation: neural manifold generalization, information bottleneck RNN, causal emergence representation, out-of-distribution generalization, memorization to generalization transition, CA1 hippocampal dynamics, low-dimensional representation necessary generalization

Emergent Generalization by Representation Learning in Artificial Neural Networks

Overview

Low-dimensional neural manifolds (identified via dimensionality reduction) have improved the interpretability of population-level neural coding. But whether such compact representations are biologically functional or merely descriptive remains contested. This paper shows that an explicit information bottleneck forcing a recurrent neural network (RNN) to learn a low-dimensional representation is necessary for rotational and out-of-distribution (OOD) generalization in a time-series prediction task.

Paper: Emergent Generalization by Representation Learning in Artificial Neural Networks

arXiv: 2607.10430v1 (July 11, 2026) Authors: Hardik Rajpal, Dan Goodman

Core Findings

1. Low-D representation is causal, not incidental: an explicit information bottleneck that compresses the RNN's hidden state to a low-D manifold is required for rotational and OOD generalization (not just correlated with it).