production-rag-architecture

name	production-rag-architecture
description	Use when designing or reviewing production RAG systems, including ingestion, chunking, embeddings, Pinecone namespaces, hybrid retrieval, reranking, context assembly, and routing.
version	1.0.0
author	Hermes Agent
license	MIT
platforms	["linux","macos","windows"]
metadata	{"hermes":{"tags":["rag","pinecone","retrieval","embeddings","reranking","chunking","architecture"],"related_skills":["principal-ai-architect","rag-evaluation-observability","architecture-diagram","huggingface-hub"]}}

Production RAG Architecture

Overview

Use this skill to design, review, or document production Retrieval Augmented Generation systems. Treat RAG as a pipeline: ingestion, chunking, embeddings, indexing, retrieval, fusion, reranking, context assembly, generation, evaluation, and observability.

The core principle: every retrieval strategy must have a reason to exist, a namespace or index boundary, a golden dataset slice, and measurable quality/cost/latency tradeoffs.

When to Use

Use when asked about:

• Pinecone namespace design
• chunking strategy
• embedding model selection
• hybrid dense+sparse retrieval
• reranking policy
• metadata schema
• context assembly and citations
• RAG pipeline review
• retrieval routing or multi-namespace strategy

Production RAG Flow

Source systems
  -> normalize and authorize
  -> chunk by strategy
  -> embed with model tier
  -> upsert into namespace with metadata
  -> classify query intent and risk
  -> select namespace(s) and filters
  -> retrieve dense/sparse candidates
  -> fuse and optionally rerank
  -> assemble context with citations
  -> route model by risk/complexity/confidence
  -> generate grounded answer
  -> trace, score, regress, iterate

Design Standards

Ingestion

• deterministic document IDs from source URI, version, and content hash
• deterministic chunk IDs from document ID, chunk index, strategy, and embedding model
• idempotent upserts
• ingestion manifests with counts, failures, namespace, embedding model, and timestamp
• no destructive namespace rebuilds without explicit migration and rollback plan

Metadata

Minimum fields:

Field	Purpose
doc_id	stable source identity
chunk_id	stable chunk identity
source_uri	audit and citation path
source_type	policy, CRM, ticket, transcript, contract, docs
title	citation label
section	heading or local semantic section
version	source version or content hash
namespace_strategy	retrieval strategy used
chunk_index	neighboring context expansion
access_scope	authorization and tenancy
updated_at	freshness

Namespace Taxonomy

Recommended pattern:

{domain}-{chunk_strategy}-{chunk_size_or_shape}-{embedding_tier}

Examples:

acq-default-512-small
acq-semantic-256-large
acq-hybrid-1024-large
acq-longform-1024-large

Namespaces should represent retrieval strategies and access boundaries, not random document buckets.

Chunk Strategy Matrix

Strategy	Best for	Risk	Eval focus
semantic 150-300 tokens	short factual Q&A	weak synthesis context	MRR, Precision@k
default 350-650 tokens	general Q&A	average edge behavior	Recall@k, relevance
longform 800-1200 tokens	synthesis and policy reasoning	token waste, lost-in-middle	NDCG@k, completeness, latency
parent-child	precision + rich context	more complex implementation	groundedness, citation accuracy
hybrid lexical+dense	exact names/IDs/acronyms	indexing complexity	lexical recall

Embedding Selection

• choose cheap/small embeddings for broad default retrieval when quality stays above threshold
• choose large/high-quality embeddings for high-value semantic nuance
• include embedding model/tier in namespace metadata
• never silently mix incompatible embedding models in a namespace
• evaluate changes against the same golden dataset

Hybrid Retrieval

Use hybrid retrieval for exact names, IDs, acronyms, policy numbers, ticket IDs, error messages, and contractual phrases.

Reranking Policy

Use reranking when candidate quality is mixed and top-3 precision matters. Avoid reranking for low-risk, latency-sensitive, high-confidence exact matches.

Reranking must report:

• NDCG@k and MRR lift
• p95 latency impact
• incremental cost
• skip policy

Common Pitfalls

1. One namespace for everything. This hides strategy tradeoffs and makes evaluation noisy.
2. Chunking without eval slices. Chunking should be justified by query types.
3. Embedding model drift. Mixing vectors from different models corrupts retrieval semantics.
4. Reranking by default. Reranking adds latency/cost and needs measured lift.
5. No authorization metadata. Production RAG must preserve access boundaries.
6. No no-answer behavior. Systems must know when retrieved context is insufficient.

Verification Checklist

• Namespace naming and purpose documented
• Metadata schema supports citation, freshness, and access control
• Chunking strategy has matching eval slice
• Embedding model choice has quality/cost rationale
• Hybrid retrieval is used only where lexical recall matters
• Reranking policy includes measured lift and latency/cost impact
• Context assembly includes citation and token budgeting rules
• Regression thresholds are defined