knowledge-base-rag

name	knowledge-base-rag
description	Knowledge Base RAG implements the complete Retrieval-Augmented Generation pipeline: document ingestion, intelligent chunking, embedding generation, vector store indexing, semantic retrieval, and grounded response generation.

Knowledge Base RAG

Part of Agent Skills™ by googleadsagent.ai™

Description

Knowledge Base RAG implements the complete Retrieval-Augmented Generation pipeline: document ingestion, intelligent chunking, embedding generation, vector store indexing, semantic retrieval, and grounded response generation. The agent builds RAG systems that answer questions from private knowledge bases with cited sources and reduced hallucination.

RAG solves the fundamental limitation of large language models: they cannot access information created after their training cutoff or proprietary information they were never trained on. By retrieving relevant documents from a vector store and injecting them into the prompt context, RAG grounds the model's responses in factual, up-to-date, organization-specific knowledge.

The quality of a RAG system depends on chunking strategy more than model choice. This skill encodes production-tested chunking approaches: semantic chunking that preserves paragraph coherence, recursive splitting that respects document structure (headings, code blocks, tables), and overlap windows that maintain context across chunk boundaries. Each strategy is matched to the document type for optimal retrieval quality.

Use When

• Building question-answering systems over private documents
• Creating a searchable knowledge base from documentation, wikis, or PDFs
• Reducing hallucination by grounding LLM responses in retrieved facts
• Implementing semantic search across large document collections
• Building customer support bots with product-specific knowledge
• The user asks about RAG, vector search, or document embedding

How It Works

graph TD
    A[Documents: PDF, MD, HTML] --> B[Ingestion Pipeline]
    B --> C[Extract Text + Metadata]
    C --> D[Intelligent Chunking]
    D --> E[Generate Embeddings]
    E --> F[Index in Vector Store]
    G[User Query] --> H[Embed Query]
    H --> I[Semantic Search: Top-K]
    I --> J[Re-rank Results]
    J --> K[Construct Prompt with Context]
    K --> L[LLM Generation]
    L --> M[Response with Citations]

The pipeline has two phases: offline ingestion (documents to vectors) and online retrieval (query to answer). The re-ranking step applies a cross-encoder to refine the initial vector search results, improving precision before the generation step.

Implementation

from dataclasses import dataclass
import hashlib

@dataclass
class Chunk:
    text: str
    metadata: dict
    embedding: list[float] | None = None

    @property
    def id(self) -> str:
        return hashlib.sha256(self.text.encode()).hexdigest()[:16]

class RecursiveChunker:
    def __init__(self, max_tokens: int = 512, overlap: int = 64):
        self.max_tokens = max_tokens
        self.overlap = overlap
        self.separators = ["\n## ", "\n### ", "\n\n", "\n", ". ", " "]

    def chunk(self, text: str, metadata: dict) -> list[Chunk]:
        chunks = self._split(text, self.separators)
        return [
            Chunk(text=c.strip(), metadata={**metadata, "chunk_index": i})
            for i, c in (chunks)  c.strip()
        ]

     () -> []:
          separators  ._token_count(text) <= .max_tokens:
             [text]

        sep = separators[]
        parts = text.split(sep)
        chunks, current = [], 

         part  parts:
            candidate = current + sep + part  current  part
             ._token_count(candidate) > .max_tokens  current:
                chunks.append(current)
                overlap_text = current[-.overlap * :]
                current = overlap_text + sep + part
            :
                current = candidate

         current:
            chunks.append(current)

        result = []
         chunk  chunks:
             ._token_count(chunk) > .max_tokens:
                result.extend(._split(chunk, separators[:]))
            :
                result.append(chunk)
         result

     () -> :
         (text) // 

 :
     ():
        .embedder = embedder
        .store = vector_store
        .llm = llm
        .chunker = RecursiveChunker()

      () -> :
        all_chunks = []
         doc  documents:
            chunks = .chunker.chunk(doc[], doc[])
             chunk  chunks:
                chunk.embedding =  .embedder.embed(chunk.text)
            all_chunks.extend(chunks)

         .store.upsert(all_chunks)
         (all_chunks)

      () -> :
        query_embedding =  .embedder.embed(question)
        results =  .store.search(query_embedding, top_k=top_k)

        context = .join(
              r  results
        )

        prompt = 

        response =  .llm.generate(prompt)
         {: response, : [r.metadata  r  results]}

Best Practices

• Use recursive chunking that respects document structure (headings, paragraphs, code blocks)
• Set chunk size to 256-512 tokens with 10-15% overlap for most use cases
• Re-rank vector search results with a cross-encoder before passing to the LLM
• Include source metadata in every chunk for citation generation
• Deduplicate chunks by content hash before indexing to avoid retrieval noise
• Instruct the LLM to say "I don't know" when the context lacks the answer

Platform Compatibility

Platform	Support	Notes
Cursor	Full	Pipeline code generation
VS Code	Full	Python/TS RAG implementation
Windsurf	Full	RAG workflow support
Claude Code	Full	End-to-end RAG building
Cline	Full	Vector store integration
aider	Partial	Code-level support

Related Skills

• AI Chat Studio
• Workflow Orchestration
• Knowledge Base Injection
• Entity Memory Management

Keywords

rag retrieval-augmented-generation vector-search embeddings chunking knowledge-base semantic-search document-qa

---

© 2026 googleadsagent.ai™ | Agent Skills™ | MIT License