菜单
Index and retrieve text, images, tables, and other modalities with shared document context.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
| name | rag-for-multimodal-content |
| title | RAG for Multimodal Content |
| description | Index and retrieve text, images, tables, and other modalities with shared document context. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | data-type-handling |
| tags | ["multimodal","images","text","vision","clip"] |
Multimodal RAG extends retrieval to include images, videos, audio, and mixed media content alongside text. This skill covers handling multiple modalities, using multimodal embedding models (like CLIP), and implementing cross-modal retrieval where text queries can find image content and vice versa.
Standard RAG systems are text-only and miss valuable information in other media:
Select a model that supports your required modalities.
Why: CLIP and similar models create a shared embedding space where text and images can be compared directly.
Parse documents containing mixed media content.
Why: Proper parsing separates different modalities while preserving their relationships within the document.
Store with named vectors for different modalities.
Why: Named vectors allow the same document to have multiple embeddings for different modalities and search strategies.
Search across text and images with mixed queries.
Why: Cross-modal search enables natural queries like "show me the architecture diagram" to retrieve relevant images.
Extend support to video and audio modalities.
Why: Video and audio add complexity but enable rich multimedia RAG applications.
Useful implementations include OpenAI CLIP, MM-Embed paper, LlamaIndex multimodal docs, and Sentence Transformers docs.
