ai-token-improvement-plan-engineer

name

ai-token-improvement-plan-engineer

description

Guides creation of AI token and cost improvement plans—baseline audits, spend attribution, optimization initiative backlog (prompt, context, model routing, RAG, agents), impact estimates, quality guardrails, measurement KPIs, and phased rollout with owners. Use when building a token reduction roadmap, cost optimization program, LLM unit-economics improvement plan, or executive brief on cutting inference spend without breaking evals—not for hands-on context layout implementation (ai-context-engineer), single-prompt rewrites (prompt-engineer), RAG pipeline build (ai-engineer), or AI ops cadence and vendor governance (ai-lead-ops). For end-to-end commercial/enterprise AI solution architecture before cost programs, use applied-ai-architect-commercial-enterprise. Empirical token studies and ablations: research-engineer-scientist-tokens.

AI Token Improvement Plan Engineer

When to Use

• Audit where tokens are spent (by feature, model, tenant, turn type)
• Prioritize cost-saving initiatives with estimated savings and risk
• Draft a phased improvement plan with metrics and rollback criteria
• Define eval gates so cost cuts do not regress quality or safety
• Brief leadership on trade-offs (model tier, context size, agent depth)

When NOT to Use

• Implement context packing or compression code → ai-context-engineer
• Rewrite one prompt or agent tool schema → prompt-engineer
• Build RAG ingest, chunking, or agent runtime → ai-engineer
• Weekly cost review ritual and release governance → ai-lead-ops
• Memory store architecture → ai-memory-developer
• Adversarial safety testing → ai-redteam
• AI policy and regulatory mapping → ai-risk-governance

Related skills

Need	Skill
Context budget and truncation	ai-context-engineer
Prompt patterns and eval of wording	prompt-engineer
RAG and agent implementation	ai-engineer
Production ops and cost reviews	ai-lead-ops
Memory write/read policy	ai-memory-developer
Safety regression testing	ai-redteam
AI solution architecture (commercial/enterprise)	applied-ai-architect-commercial-enterprise
Token efficiency experiments and benchmarks	research-engineer-scientist-tokens

Core Workflows

1. Scope and baseline

Define:

• Surface area — chat, copilot, batch, agents, embeddings-only
• Time window — 7/30 days; exclude anomalies (launches, incidents)
• Segmentation — model, feature flag, tenant tier, environment

Collect baseline metrics (see references/token_audit.md).

Deliverable: current state table with top 5 cost drivers (≥70% of spend if possible).

2. Categorize spend

Bucket each driver:

Category	Examples
Input bloat	Long system prompts, duplicated docs, full chat history
Output bloat	Verbose defaults, no max_tokens, unconstrained agents
Model choice	Opus-class for simple classification
Retrieval	Over-fetching chunks, huge tool results in context
Agent loops	Extra tool rounds, retry storms
Infrastructure	Re-embedding unchanged corpora, log payloads in prompts

Tag fixed per request vs scales with users/sessions.

3. Initiative backlog

For each idea record:

• Description and owner team
• Estimated savings (% tokens or $/month) with assumptions
• Effort (S/M/L) and dependencies
• Quality risk (low/med/high) and required evals
• Measurement — metric that proves success

Prioritize with impact × confidence ÷ effort; never ship without eval plan.

See references/improvement_backlog.md and references/optimization_playbook.md.

4. Quality and safety guardrails

Every initiative must list:

• Golden-set evals (task accuracy, format)
• Safety set (if user-facing)
• Latency check (p95)
• Rollback trigger (e.g., CSAT drop, eval regression >X%)

See references/measurement_and_kpis.md.

5. Phased plan

Typical phases:

Phase	Focus	Duration
0 — Measure	Instrumentation, dashboards, attribution	1–2 weeks
1 — Quick wins	max_tokens, model routing, prompt trim, cache	2–4 weeks
2 — Structure	RAG top-k, history compression, tool output limits	4–8 weeks
3 — Architecture	Agent budget caps, routing policies, batch/offline	8+ weeks

Each phase: goals, initiatives, owners, exit criteria.

See references/rollout_plan.md.

6. Executive summary

## Situation — spend and growth rate
## Target — $ or tokens/session goal by date
## Top drivers — ranked list
## Plan — 3–5 initiatives per phase
## Risks — quality, latency, eng capacity
## Asks — budget, headcount, eval time

Hand ongoing ops to ai-lead-ops after plan approval.

When to load references

• Baseline and attribution → references/token_audit.md
• Backlog template and scoring → references/improvement_backlog.md
• Technique catalog → references/optimization_playbook.md
• KPIs and eval gates → references/measurement_and_kpis.md
• Phasing and rollback → references/rollout_plan.md