| name | inference-placement-advisor |
| description | Decide where AI inference should run — edge/on-device vs cloud, batch vs realtime, GPU vs CPU — with the embodied-carbon trade-offs named honestly. Use this skill whenever the user asks whether to run models on-device/locally/at the edge or in the cloud, whether to batch inference jobs, or how to serve a model efficiently. Part of Lean Agentic AI Skills; emits lean-findings.json. |
Inference Placement Advisor
Advisor skill. Input: model size/class, request pattern (volume, latency SLO, burstiness), device fleet characteristics if edge is on the table. Output: lean-findings.json with placement recommendations.
Placement moves all four SCI terms at once, which is why glib answers ("edge is greener!") are usually wrong. This skill's value is naming the tensions.
Subject type: emit subject.type: "ai-inference" in findings.
Decision axes
- Edge/on-device — wins: no network transfer per request (E↓), uses hardware the user already owns (M shared with the device's other purposes), data locality (privacy bonus). Loses: pushes compute to less efficient silicon than datacenter accelerators for large models; forces small/quantized models (fine if quality holds — eval it); shifts battery drain to users (name it — that's someone else's E); update distribution costs.
Right when: model fits small (embeddings, small LLMs, classic ML), requests frequent per user, connectivity unreliable, privacy matters.
- Cloud realtime — wins: efficient accelerators, high utilization if traffic is steady, big models feasible. Loses: network round-trip per request, provisioned-capacity idle if traffic is spiky (E+M waste at low utilization).
Right when: model large, latency SLO moderate, traffic steady enough to keep utilization high.
- Cloud batch — wins: highest utilization, off-peak/carbon-aware scheduling possible (I↓ — pair with carbon-aware-scheduler), spot/preemptible hardware (M shared). Loses: latency by definition.
Right when: results aren't needed interactively (embeddings backfill, nightly scoring, evaluation runs). Any "realtime" pipeline whose consumers actually read results hourly is a batch job in disguise — a classic high-severity finding.
- Hybrid ladder — small on-device model handles the common case, escalates hard cases to cloud (compose with model-right-sizer). Often the best of both; costs complexity.
Honesty rules
Never claim edge is categorically greener — the M and device-efficiency tensions are real; state which effect likely dominates for THIS workload and what measurement would confirm (device power profiling, per-request transfer sizes). Utilization assumptions must come from the user's traffic data, not hope.
Not this skill's job
Which region (region-selector), when to run batches (carbon-aware-scheduler), model selection (model-right-sizer).
