neurocogmap-llm-cognitive-organization

name	neurocogmap-llm-cognitive-organization
description	NeuroCogMap framework for mapping cognitive functions in LLMs using neuroscience-inspired methodology. Analyzes hallucination, bias, refusal, sycophancy, and memory capabilities via parcel-functional annotation and cross-model functional correspondence.
trigger_words	["neurocogmap","cognitive organization","LLM cognitive mapping","parcel annotation","functional correspondence","cognitive audit"]
category	neuroscience

Overview

NeuroCogMap (arXiv:2607.00397) reveals cognitive organization of Large Language Models by mapping LLM internal representations to human brain-like cognitive parcels. Uses LLM-judge alignment, parcel-functional annotation, and cross-model functional correspondence to audit and understand LLM cognitive capabilities.

Core Methodology

1. Parcel-Functional Annotation

• Map LLM hidden states to cognitive "parcels" (functional brain-like regions)
• Each parcel corresponds to a specific cognitive capability
• Audit parcels using functional description prompts and quality scoring

2. Cross-Model Parcel Matching

• Compare parcel activations across different LLM architectures
• Identify universal vs model-specific cognitive organizations
• Use LLM-judge alignment prompts to verify functional correspondence

3. Capability Auditing Pipeline

• Hallucination Detection: Semantic entropy, SelfCheckGPT, hidden probing, logits SVM
• Bias Evaluation: Multi-attribute choice, Shepard categorization
• Refusal/Jailbreak Analysis: User-conditioned attention, attention probing
• Sycophancy Detection: Paired-preference evaluation, belief/control classifiers
• Memory Testing: Episodic long-term memory, multi-attribute choice, intertemporal choice

Implementation Patterns

Parcel Activation Ranking

1. Identify high-activation neurons for a given task
2. Generate functional descriptions via LLM prompts
3. Score description quality using LLM judge
4. Confirm functional redundancy through intervention
5. Map parcel to human cortical function via similarity comparison

Cross-Model Correspondence

1. Extract parcel embeddings from model A
2. Extract parcel embeddings from model B
3. Compute similarity between parcel pairs
4. Use LLM judge to verify functional correspondence
5. Build correspondence matrix across all parcel pairs

Pitfalls

• Scale: 79-page paper with extensive datasets — focus on methodology, not benchmark results
• Evaluation complexity: Multiple evaluation formats (hallucination, bias, jailbreak, sycophancy) — use targeted subset
• Parcel granularity: Too fine-grained parcels lead to noise; aggregate to meaningful cognitive clusters

Verification Steps

1. Run parcel-functional annotation on a specific capability (e.g., reasoning)
2. Verify parcel activation patterns match expected cognitive function
3. Cross-validate with hidden probing and attention probing baselines
4. Compare results across at least 2 different LLM architectures

Activation

neurocogmap, cognitive mapping, LLM cognitive audit, parcel analysis, functional correspondence, hallucination detection, bias evaluation, LLM interpretability, cognitive organization