arag-hierarchical-retrieval

name	arag-hierarchical-retrieval
title	A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.03442
keywords	["Retrieval-Augmented Generation","Agent Autonomy","Hierarchical Interfaces","Information Seeking","Tool Calling"]
description	Enable LLM agents to autonomously retrieve information across multiple granularities using keyword search, semantic search, and chunk read tools. Simple ReAct-based loop with hierarchical interfaces outperforms dense retrieval by allowing adaptive information seeking without complex graph construction.

A-RAG: Agent-Driven Hierarchical Retrieval

Standard RAG systems retrieve all relevant passages upfront, but this assumes the retriever knows exactly what to find. A-RAG gives LLM agents autonomy to iteratively explore documents using three complementary tools: keyword search for entity lookups, semantic search for concept matching, and chunk read for full document access. This hierarchical approach mirrors human information-seeking behavior and enables agents to refine queries based on intermediate findings.

The key insight is that agents learn effective search strategies better than fixed retrievers. By exposing multiple retrieval granularities, the system allows adaptive information gathering without requiring complex graph construction or learning separate retrieval policies.

Core Concept

A-RAG operates on three retrieval tools that agents can call autonomously:

1. Keyword Search: Exact text matching for specific entities and terms
2. Semantic Search: Embedding-based similarity to find conceptually related passages
3. Chunk Read: Access full document content after identifying relevant sections

Rather than ranking documents globally, agents iteratively refine their information gathering strategy, mimicking how humans explore documents interactively.

Architecture Overview

• Document Index: Standard chunking (~1000 tokens) with keyword and embedding indices
• Retrieval Tool Set: Three callable tools with distinct semantics
• Agent Loop: ReAct-style reasoning with tool calls and observations
• Tracking Mechanism: Remembers previously-read chunks to avoid redundant queries
• Integration Layer: Works with existing LLM APIs that support tool calling

Implementation

Step 1: Create Hierarchical Document Index

Build indices supporting all three retrieval modalities.

# Hierarchical document indexing
from sentence_transformers import SentenceTransformer

class HierarchicalDocumentIndex:
    def __init__(self, model_name: str = "sentence-transformers/all-MiniLM-L6-v2"):
        """
        Index documents for hierarchical retrieval.

        Args:
            model_name: Sentence embedding model
        """
        .chunks = {}  
        .chunk_embeddings = {}  
        .keyword_index = {}  
        .embedding_model = SentenceTransformer(model_name)

     ():
        
        
        words = text.split()
        chunks_text = []

         i  (, (words), chunk_size):
            chunk_text = .join(words[i:i+chunk_size])
            chunk_id = 

            .chunks[chunk_id] = chunk_text
            chunks_text.append(chunk_text)

            
             keyword  chunk_text.lower().split():
                 keyword   .keyword_index:
                    .keyword_index[keyword] = []
                .keyword_index[keyword].append(chunk_id)

        
        embeddings = .embedding_model.encode(chunks_text)
         chunk_id, embedding  (
            [  i  (, (words), chunk_size)],
            embeddings
        ):
            .chunk_embeddings[chunk_id] = embedding

     () -> []:
        
        query_terms = query.lower().split()
        chunk_scores = {}

         term  query_terms:
             term  .keyword_index:
                 chunk_id  .keyword_index[term]:
                    chunk_scores[chunk_id] = chunk_scores.get(chunk_id, ) + 

        
        ranked = (chunk_scores.items(), key= x: x[], reverse=)
         [chunk_id  chunk_id, _  ranked[:top_k]]

     () -> []:
        
        query_embedding = .embedding_model.encode(query)

        
        similarities = {}
         chunk_id, embedding  .chunk_embeddings.items():
            sim = np.dot(query_embedding, embedding)
            similarities[chunk_id] = sim

        ranked = (similarities.items(), key= x: x[], reverse=)
         [chunk_id  chunk_id, _  ranked[:top_k]]

     () -> :
        
         .chunks.get(chunk_id, )