| name | ai-engineer |
| description | Guides production AI engineering—LLM apps, RAG, agents, eval harnesses, observability, cost/latency,
and safe deployment. Use when building chatbots, copilots, retrieval, agent workflows, model routing,
or LLM API integration—not research synthesis (ai-researcher), AI policy (ai-risk-governance),
red-team (ai-redteam), prompt-only work (prompt-engineer), non-AI ADRs (senior-system-architecture),
token programs (ai-token-improvement-plan-engineer), solution architecture
(applied-ai-architect-commercial-enterprise), skills portfolio (ai-skill-manager), or vertical AI
engineering management (engineering-manager-vertical-ai-products).
Agent prompt authoring and eval harness design: prompt-engineer-agent-prompts-evals.
Safeguard inference gateways and moderation infra: ml-infrastructure-engineer-safeguards.
Safeguard classifier research: ml-research-engineer-safeguards.
|
AI Engineer
When to Use
- Building chatbots, copilots, or retrieval-augmented generation systems
- Designing multi-step agent workflows with tool use
- Integrating OpenAI, Anthropic, or local models into products
- Setting up RAG pipelines (chunk, embed, index, retrieve, rerank, generate)
- Building evaluation harnesses and regression suites for LLM features
- Optimizing cost/latency through model routing, caching, or context strategy
- Planning safe deployment of generative features (canary, kill switch, monitoring)
When NOT to Use
- Academic literature synthesis or research methodology →
ai-researcher
- Organizational AI policy, regulation, or risk tiering →
ai-risk-governance
- Adversarial safety testing and jailbreak campaigns →
ai-redteam
- Prompt-only tuning without system architecture changes →
prompt-engineer
- Enterprise-wide non-AI system integration ADRs →
senior-system-architecture
- Token/cost improvement program planning and roadmap →
ai-token-improvement-plan-engineer
- Commercial/enterprise AI solution architecture →
applied-ai-architect-commercial-enterprise
- Skills portfolio governance and batch validation →
ai-skill-manager
- Agent prompts, golden evals, judge rubrics →
prompt-engineer-agent-prompts-evals
Related skills
| Need | Skill |
|---|
| Prompt templates and agent message design | prompt-engineer |
| Offline experiments, statistics, classical ML | data-scientist |
| Papers, benchmarks, research methodology | ai-researcher |
| Policies, model cards, governance | ai-risk-governance |
| Red-team and jailbreak campaigns | ai-redteam |
| Persistent memory design | ai-memory-developer |
| Context window and token budgeting | ai-context-engineer |
| Token cost improvement plan and roadmap | ai-token-improvement-plan-engineer |
| AI production ops and release governance | ai-lead-ops |
| Cross-system boundaries and platform ADRs | senior-system-architecture |
| Commercial/enterprise AI architecture | applied-ai-architect-commercial-enterprise |
| Agent skills catalog and validation | ai-skill-manager |
| Safeguard serving stack and policy runtime | ml-infrastructure-engineer-safeguards |
| Safety model R&D and benchmark design | ml-research-engineer-safeguards |
Core Workflows
1. Solution shaping
- Define user job, success metric, and failure modes
- Decide: single LLM call vs RAG vs multi-step agent
- Choose model tier (quality vs cost vs latency)
- Identify data sources, PII boundaries, and retention
- Plan human-in-the-loop for high-risk actions
See references/solution_patterns.md for RAG vs fine-tune vs agent decision tree.
2. RAG pipeline
ingest → chunk → embed → index → retrieve → rerank → generate → cite
Checklist:
See references/rag_pipeline.md for chunking, eval metrics, and freshness.
3. Agents and tools
- Tools: narrow schemas, idempotent where possible, timeouts
- Loop: plan → act → observe → stop condition
- Cap iterations and token budget
- Log tool calls for audit; redact secrets in traces
See references/agents_tools.md for ReAct patterns and failure handling.
4. Evaluation before launch
| Layer | Measure |
|---|
| Retrieval | Recall@k, MRR on golden questions |
| Generation | Faithfulness, answer relevance (LLM-judge + human sample) |
| Safety | Refusal rate on policy violations |
| Ops | p95 latency, cost per session |
Ship only when regression suite passes on CI for golden set.
See references/evaluation_ops.md for datasets, CI eval, and monitoring.
5. Production operations
- Version prompts and models; canary new versions
- Monitor drift, error rate, tool failures, spend
- Kill switch for model or feature flag
- Incident runbook for toxic output or data leak
See references/evaluation_ops.md for production monitoring.
When to load references
- Architecture choices →
references/solution_patterns.md
- RAG implementation →
references/rag_pipeline.md
- Agents and tools →
references/agents_tools.md
- Eval and production →
references/evaluation_ops.md
