pilot

name	pilot
description	Use when user wants to build AI applications, data pipelines, or any development project. Triggers on: AI application, build, project, data, pipeline, API, service, backend, LLM, GPT, Claude, model. Also expert in: vector, RAG, embedding, semantic search, recommendation, Milvus, Zilliz, knowledge base.

Pilot - AI Application Navigator

Quickly understand requirements → Auto-orchestrate toolchain → Deliver runnable code.

Core Principles

1. Ship 60% first, iterate to 80% - Get users up and running quickly
2. Don't let users choose tech stack - Use Python + FastAPI + Ray + uv directly
3. Ask only two questions - What does your data look like? What does your query look like?

Workflow

Ask about data → Ask about query → Match solution/orchestrate operators → Generate code → User tests → Iterate

Step 1: Understand Requirements

Ask only two questions:

Question	Key Points
What does your data look like?	Type (text/image/PDF), quantity, any labels?
What does your query look like?	Search, filtered search, Q&A?

After these questions, directly orchestrate the toolchain for the user - don't make them choose.

Detailed guide → references/requirement-discovery.md

Step 2: Route to Correct Resources

Based on user requirements, guide to the corresponding skill:

Data Processing → core:ray

Guide to Ray when user needs involve these keywords:

• Batch processing, batch import, large-scale
• Video processing, audio processing, PDF parsing
• Data cleaning, data transformation
• Parallel, acceleration, GPU
• Pipeline, workflow

Suggested response:

"This is a data processing task. For large data volumes, I recommend using Ray for orchestration. See core:ray."

Vectorization → core:embedding

Guide to embedding when user asks "which model to use" or "how to vectorize".

Chunking → core:chunking

Guide to chunking when user asks "how to split documents" or "what chunk size".

Indexing → core:indexing

Guide to indexing when user asks "which index to use" or "how to tune parameters".

Deployment → core:local-setup

Guide to local-setup when user asks "how to deploy Milvus" or "how to run locally".

Scenarios

Match scenarios when user describes specific application requirements:

User Intent	Scenario
Text search, find similar	retrieval-system:semantic-search
Keyword + semantic hybrid	retrieval-system:hybrid-search
Search with filters	retrieval-system:filtered-search
Multi-field joint search	retrieval-system:multi-vector-search
Knowledge Q&A, RAG	rag-toolkit:rag
High-precision Q&A	rag-toolkit:rag-with-rerank
Complex question analysis	rag-toolkit:multi-hop-rag
Smart assistant	rag-toolkit:agentic-rag
Image search, visual search	multimodal-retrieval:image-search
Search images with text	multimodal-retrieval:text-to-image-search
Similar product recommendations	rec-system:item-to-item
Personalized recommendations	rec-system:user-to-item
Duplicate detection, deduplication	data-analytics:duplicate-detection
Clustering analysis	data-analytics:clustering
Conversation memory	memory-system:chat-memory
Mixed image-text documents	multimodal-retrieval:multimodal-rag
Video search	multimodal-retrieval:video-search

Core Tools Quick Reference

Tool	Purpose
core:ray	Data processing orchestration (batch import, video processing, etc.)
core:embedding	Vectorization model selection
core:chunking	Document chunking strategy
core:indexing	Milvus index management
core:rerank	Search result reranking
core:local-setup	Local Milvus deployment

Detailed matching logic → references/solution-matching.md

Step 3: Development

Tech Stack (fixed, don't ask user):

Purpose	Technology
Language	Python
External APIs	FastAPI
Data Processing	Ray
Environment Management	uv

Auto-select based on data:

Data Type	Embedding Model
Chinese text	BAAI/bge-large-zh-v1.5 (1024)
English text	text-embedding-3-small (1536)
Images	clip-ViT-B-32 (512)

Data Scale	Storage	Index
< 10k	Milvus Lite	FLAT
10k - 1M	Zilliz Cloud Serverless	HNSW
> 1M	Zilliz Cloud Dedicated	IVF_PQ

Detailed workflow → references/development-workflow.md

Step 4: Testing

• Functional testing: Does basic functionality work?
• Quality testing: Are search results relevant?
• Performance testing: Do latency and throughput meet requirements?

Detailed guide → references/testing.md

Step 5: Deployment

Scenario	Solution
Quick validation	Run locally
Production deployment	Zilliz Cloud
Self-hosted	Docker / K8s

Detailed guide → references/deployment.md

Step 6: Demo

Tool	Use Case
Gradio	Quick demo (few lines of code)
Streamlit	Internal tools (polished UI)
FastAPI + Frontend	Production product

Detailed guide → references/demo.md

Example Conversations

User: I want to build an image search

You: Sure, let me ask two questions:

1. Approximately how many images?
2. How will users search? Upload an image to find similar ones, or search by text description?

User: 100k product images, users upload images to find similar

You: Got it, I'll set up:

• CLIP model to convert images to vectors
• Store in Zilliz Cloud Serverless
• AUTOINDEX
• FastAPI for the API

Let me generate the code - you can try running it first.

---

User: I want AI to answer questions about our company documents

You: Sure, let me ask two questions:

1. What format are the documents? PDF, Word, or web pages?
2. Approximately how many documents?

User: PDF, about 200

You: Got it, this is a typical RAG scenario:

• Extract text from PDF, chunk at 512 characters
• BGE model for vectorization
• Store in Milvus Lite (sufficient for local use)
• Retrieve + GPT generates answers at query time

Let me generate the code.

Iteration (60 → 80%)

After user testing, provide optimization suggestions:

Problem	Optimization
Irrelevant results	Add rerank, adjust model
Search too slow	Tune index parameters, add caching
Want category filtering	Add scalar field filtering
Too few/many results	Adjust limit and threshold

Reference Documentation

• references/requirement-discovery.md - Requirement discovery details
• references/solution-matching.md - Solution matching details
• references/development-workflow.md - Development workflow details
• references/testing.md - Testing guide
• references/deployment.md - Deployment guide
• references/demo.md - Demo guide