llm-reorganize-representational-geometry-icl

name	llm-reorganize-representational-geometry-icl
description	Large language models reorganize representational geometry during in-context learning, showing that ICL depends on successful online untangling of task-relevant representations with geometric reorganization increasing separability.
created	"2026-05-31T00:00:00.000Z"
source	arXiv:2605.28854
authors	Hua-Dong Xiong, Li Ji-An, Robert C. Wilson, Kwonjoon Lee, Xue-Xin Wei
tags	["neuroscience","LLM","in-context-learning","representational-geometry","neural-representations","classification","untangling"]
activation_keywords	["in-context learning","ICL","representational geometry","neural untangling","LLM classification","prototype algorithm","representational separability"]

Large Language Models Reorganize Representational Geometry During In-Context Learning

Overview

This study provides a geometric account of in-context learning (ICL) in pretrained LLMs, establishing representational geometry as a mechanistic constraint on ICL effectiveness. Key insight: ICL depends on the successful online untangling of task-relevant representations, with geometric reorganization increasing online separability.

Core Methodology

Experimental Design

• Task: In-context classification using model's internal representations with known structure
• Labels: Defined by model's own internal representations
• Goal: Study how representational structure affects ICL performance

Key Hypothesis

Inspired by neuroscience view of classification as "untangling of neural representations":

• ICL depends on successful online untangling of task-relevant representations
• LLMs reorganize representational geometry during ICL
• Increased separability correlates with better ICL performance

Key Findings

1. Representational Structure Correlation

• ICL performance correlates systematically with representational structure
• Tasks with better inherent separation are easier to learn in-context
• Represents a mechanistic constraint on ICL capability

2. Geometric Reorganization

• Successful ICL accompanied by geometric reorganization
• Reorganization increases online separability
• Active reshaping of representations during classification

3. Prototype-like Algorithm Discovery

• LLM behavior well-described by prototype-like algorithm
• Algorithm integrates evidence while reshaping representations
• Supports classification through prototype formation

Technical Insights

Neuroscience Connection

Classification in brain = Untangling neural representations

• Similar principle applies to LLM ICL
• High-dimensional representation space transformations
• Geometric structure determines learning difficulty

Prototype Algorithm Characteristics

# Conceptual prototype-like ICL behavior
class PrototypeICL:
    def classify(self, examples):
        # 1. Integrate evidence from examples
        prototypes = self.compute_prototypes(examples)
        
        # 2. Reshape representations for separability
        transformed_repr = self.untangle_representations(prototypes)
        
        # 3. Increase online separability
        enhanced_repr = self.enhance_geometry(transformed_repr)
        
        # 4. Classify based on prototype proximity
        return self.nearest_prototype(enhanced_repr)

Representational Geometry Metrics

• Separability: Distance between class clusters
• Untangling: Linear separability after transformation
• Online reorganization: Dynamic geometry changes during ICL
• Prototype formation: Class representative formation

Practical Applications

LLM ICL Optimization

1. Design tasks with better representational separation
2. Use prompts that encourage geometric untangling
3. Leverage prototype formation in examples
4. Consider representational structure in prompt engineering

Understanding ICL Limitations

• Quantify gap between pretrained representations and ICL exploitation
• Identify representational bottlenecks
• Predict ICL performance from representational geometry

Research Questions Addressed

1. How does representational geometry shape ICL effectiveness?
2. What geometric reorganization occurs during successful ICL?
3. Can we predict ICL performance from representational structure?
4. What algorithm best describes LLM ICL behavior?

Theoretical Framework

Geometric Account of ICL

• High-dimensional representation space transformations
• Online untangling as key mechanism
• Representational geometry as mechanistic constraint
• Prototype formation as algorithmic implementation

Mechanistic Constraints

1. Pretrained representation structure: Limits what ICL can exploit
2. Online untangling capability: Determines learning speed
3. Geometric reorganization: Enables adaptation
4. Prototype formation: Supports classification

Limitations

• Focus on classification tasks (not generation)
• Internal representation analysis (not all architectures)
• Synthetic tasks with known structure
• May not capture all ICL mechanisms

Future Directions

• Generalization to other task types
• Architecture comparison studies
• Real-world task application
• Training optimization based on geometry
• Transfer learning implications

Related Work

• In-context learning mechanisms
• Neural representation untangling
• Prototype-based classification
• Representational geometry analysis
• Neuroscience classification models

References

• arXiv:2605.28854 - Full paper
• Neuroscience classification literature
• LLM mechanistic interpretability

Activation

Use when:

• Studying in-context learning mechanisms
• Analyzing representational geometry in LLMs
• Designing ICL tasks with optimal structure
• Understanding prototype formation in LLMs
• Predicting ICL performance from representations
• Connecting neuroscience classification to AI