rag-architect

name	rag-architect
description	Design, evaluate, and evolve production retrieval-augmented generation systems. Use when a task requires evidence-grounded search over private or specialized corpora, retrieval architecture selection, ingestion and permission design, hybrid search, reranking, citation integrity, offline evaluation, online monitoring, or diagnosis of retrieval failures.
zh_description	用于以评测为先设计和诊断生产级 RAG，包括权限、混合检索、重排、引用和监控。
version	2.0.0
author	seaworld008
source	in-house
source_url
tags	["architect", "rag", "retrieval", "evaluation", "search"]
created_at	2026-03-04
updated_at	2026-07-27
quality	5
complexity	advanced

RAG Architect

Design from measured retrieval failures, not from a fashionable vector database or chunk size.

Start With the Product Contract

Define:

• users and authorization boundary
• answerable and unanswerable question classes
• source corpus, languages, formats, and update rate
• freshness and deletion requirements
• latency, cost, availability, and data-residency constraints
• citation and audit requirements
• acceptable abstention and false-answer rates

Do not build RAG when direct context, structured queries, a conventional search index, or a deterministic API is simpler and more reliable.

Build the Evaluation Set First

Create a versioned dataset before tuning:

{
  "id": "policy-017",
  "query": "Can a contractor export customer data?",
  "expected_source_ids": ["policy-access-4.2"],
  "required_facts": ["contractors cannot export", "exception requires DPO approval"],
  "forbidden_claims": ["contractors always have export access"],
  "user_acl": ["policy:employee"],
  "as_of": "2026-07-01",