Menu
Index and retrieve text, images, tables, and other modalities with shared document context.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Use this skill when building, debugging, or improving Retrieval-Augmented Generation systems, including chunking, vector database selection, hybrid search, reranking, multimodal RAG, code documentation RAG, retrieval latency, and production RAG architecture.
Chunk nested documents into parent-child levels so retrieval can move from broad sections to fine-grained passages.
Use semantic boundaries and embedding similarity to chunk text for higher-relevance retrieval.
Route RAG chunking decisions across semantic, hierarchical, sliding-window, contextual-header, and framework-selection strategies.
Use overlapping windows to preserve context across chunk boundaries while controlling retrieval size.
Reduce retrieval latency with caching, batching, and index-level optimization.
| name | rag-for-multimodal-content |
| title | RAG for Multimodal Content |
| description | Index and retrieve text, images, tables, and other modalities with shared document context. |
| allowed-tools | ["Read","Grep","Glob","Bash"] |
| category | data-type-handling |
| tags | ["multimodal","images","text","vision","clip"] |
Multimodal RAG extends retrieval to include images, videos, audio, and mixed media content alongside text. This skill covers handling multiple modalities, using multimodal embedding models (like CLIP), and implementing cross-modal retrieval where text queries can find image content and vice versa.
Standard RAG systems are text-only and miss valuable information in other media:
Select a model that supports your required modalities.
Why: CLIP and similar models create a shared embedding space where text and images can be compared directly.
Parse documents containing mixed media content.
Why: Proper parsing separates different modalities while preserving their relationships within the document.
Store with named vectors for different modalities.
Why: Named vectors allow the same document to have multiple embeddings for different modalities and search strategies.
Search across text and images with mixed queries.
Why: Cross-modal search enables natural queries like "show me the architecture diagram" to retrieve relevant images.
Extend support to video and audio modalities.
Why: Video and audio add complexity but enable rich multimedia RAG applications.
Useful implementations include OpenAI CLIP, MM-Embed paper, LlamaIndex multimodal docs, and Sentence Transformers docs.
