메뉴
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
Route RAG performance work for latency, caching, indexing, filtering, batching, and query optimization.
| name | semantic-chunking |
| title | Semantic Chunking |
| description | Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | chunking |
| tags | ["semantic","nlp","sentence-boundary","context-preservation"] |
Semantic chunking divides documents into segments based on natural language boundaries and semantic meaning rather than fixed character counts. This approach preserves context integrity and improves retrieval relevance by keeping related ideas together.
Fixed-size chunking (e.g., 1000 tokens) often cuts through mid-sentence, mid-paragraph, or mid-concept, resulting in:
Use sentence boundary detection rather than simple period splitting. This handles abbreviations, decimal numbers, and other edge cases.
Why: Simple period/regex splitting fails on "Dr. Smith", "3.14", "U.S.A.", etc.
Group sentences into chunks based on semantic cohesion rather than fixed counts.
Why: Similarity-based chunking adapts to document density - dense technical sections get smaller chunks, narrative sections get larger ones.
Leverage document structure (headings, markdown, HTML tags) to identify natural sections.
Why: Document structure often encodes semantic boundaries (chapters, sections, subsections).
Adjust chunk sizes based on content characteristics (code blocks, lists, tables).
Why: Different content types need different chunking strategies for optimal retrieval.
For practical implementations, compare Sentence-BERT, LlamaIndex Semantic Chunker, LangChain RecursiveCharacterTextSplitter, and spaCy sentence segmentation.
