Ejecuta cualquier Skill en Manus
con un clic

Ejecuta cualquier Skill en Manus con un clic

edge-inference

Estrellas25

Forks7

Actualizado1 de junio de 2026 a las 09:15

Edge / on-device AI inference architecture: running models on MCUs, NPUs, mobile, and mini-PCs; quantization for edge, TOPS/memory/energy budgets, TinyML, and the latency case for on-device vs cloud. Architect-level. USE WHEN: designing on-device/edge AI, "edge inference", "on-device", "NPU", "TinyML", "quantization", "Jetson", "Coral", "Hailo", local LLM on small hardware, offline/low-latency inference, energy-constrained ML. DO NOT USE FOR: cloud GPU serving (use `inference-serving-topology`); choosing datacenter accelerators (use `ai-hardware-selection`); RAG app code (use rag skills).

Instalación

Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.

Ejecutar en Manus

Fuente

claude-dev-suite

claude-dev-suite/claude-dev-suite

Abrir repositorio de GitHub Ver repositorios del creador

Descarga

Ejecutar en Manus

Ocupaciones relacionadasSOC

Basado en la clasificación ocupacional SOC

Ingenieros de hardware informáticoArquitectura e ingeniería·SOC 17-2061

SKILL.md

readonly

name	edge-inference
description	Edge / on-device AI inference architecture: running models on MCUs, NPUs, mobile, and mini-PCs; quantization for edge, TOPS/memory/energy budgets, TinyML, and the latency case for on-device vs cloud. Architect-level. USE WHEN: designing on-device/edge AI, "edge inference", "on-device", "NPU", "TinyML", "quantization", "Jetson", "Coral", "Hailo", local LLM on small hardware, offline/low-latency inference, energy-constrained ML. DO NOT USE FOR: cloud GPU serving (use `inference-serving-topology`); choosing datacenter accelerators (use `ai-hardware-selection`); RAG app code (use rag skills).
allowed-tools	Read, Grep, Glob

Edge / On-Device Inference

Why on-device (the decision drivers)

Latency: on-device token/inference in ~tens of ms vs 200–500ms cloud round-trip — decisive for voice, AR, control loops.
Privacy / offline / cost: data never leaves the device; works without connectivity; no per-call cloud cost.
Cost of doing it: tight memory/energy/thermal budgets; smaller models; more engineering.

The hardware tiers (match model to silicon)

Tier	Silicon	Typical model
MCU / TinyML	Cortex-M + tiny NPU (sub-$1 class)	KB-MB models: keyword spotting, anomaly detection (TFLite Micro)
Mobile / AI-PC	Phone NPU, laptop NPU (tens of TOPS)	Quantized 3–8B LLMs, vision
Edge box	Jetson Orin/Thor, Coral, Hailo (40+ TOPS)	7–13B LLMs, multi-camera vision

Architectural levers

Quantization is the key enabler: FP16 → INT8 → INT4 trades accuracy for memory/throughput/energy. Most edge LLMs run INT4/INT8. Validate accuracy loss.
Model choice: small instruct/distilled models (e.g. 3–8B class) and vision models sized to the NPU's memory bandwidth, not just TOPS.
Runtime: TFLite/LiteRT, ONNX Runtime, ExecuTorch, llama.cpp/Ollama, vendor SDKs (TensorRT for Jetson). Pick what targets the accelerator.
Budgets: state the TOPS, RAM, and energy-per-inference budget up front — they bound everything. Batch=1, KV-cache memory dominates LLM RAM.

When to recommend edge vs cloud vs hybrid

Hard latency / offline / privacy / per-unit cost → on-device.
Large model / variable load / centralized updates → cloud serving.
Both needs → hybrid (small local model + cloud escalation): see hybrid-edge-cloud.

Más de este repositorio

mismo repositorio

cyber-physical

claude-dev-suite/claude-dev-suite

Cyber-physical / industrial control system (ICS-SCADA) architecture in general: the Purdue model levels, control loops, IT/OT convergence, OT security (IEC 62443, segmentation, zero-trust for OT), safety, determinism, and redundancy. Architect-level, beyond any specific DCS/PLC product. USE WHEN: designing/evaluating industrial control, SCADA, robotics, energy, automotive, or IoT-at-scale systems, "OT security", "Purdue model", "PLC/RTU/ SCADA architecture", "IT/OT convergence", "IEC 62443", "control loop", safety instrumented systems. DO NOT USE FOR: specific DCS platforms / IEC 61131 / ISA formats (use the `industrial/*` skills); pure RTOS scheduling (use `embedded-rtos`); app security (use `security-architecture`).

2026-06-0125

distributed-ledger

claude-dev-suite/claude-dev-suite

Distributed-ledger / blockchain architecture in general (engine-agnostic): when a ledger beats a database, consensus families (PoW/PoS/BFT), L1 vs L2 (rollups, channels, sidechains), permissioned vs permissionless, UTXO vs account models, and the scalability trilemma. Architect-level, not coin-specific. USE WHEN: designing/evaluating a blockchain or DLT system in general, "L2", "rollup", "zk vs optimistic", "PoS vs PoW", "permissioned ledger", "consortium chain", "UTXO vs account", "do we even need a blockchain", token/state design. DO NOT USE FOR: Bitcoin-specific work (use the `bitcoin/*` skills); raw consensus internals (use `distributed-consensus`); ordinary app data (use database / data-intensive skills).

2026-06-0125

game-engine-architecture

claude-dev-suite/claude-dev-suite

Game-engine architecture, engine-agnostic: the game loop (fixed vs variable timestep), ECS vs scene-graph/OOP, the render pipeline, core subsystems (physics, audio, animation, assets, memory), and netcode models. Architect- level, beyond any specific engine. USE WHEN: designing/evaluating a game engine or game's architecture in general, "game loop", "ECS", "entity component system", "render pipeline", "forward vs deferred", "rollback netcode", "lockstep", custom engine vs off-the-shelf, data-oriented game design. DO NOT USE FOR: Unity-specific work (use the `gamedev/unity-*` skills); low-level cache/SIMD detail (use `hardware-aware-design`); web graphics (use graphics skills).

2026-06-0125

agentic-architecture

claude-dev-suite/claude-dev-suite

Architecture of LLM agent systems: orchestration topologies (single agent, supervisor/sub-agents, pipelines, networks), memory/context strategy, the tool layer, and human-in-the-loop/control. Architect-level system design, not prompt wording. USE WHEN: designing agentic/LLM-agent systems, "agent orchestration", "multi-agent", "supervisor", "sub-agents", "tool use", "agent memory", "human-in-the-loop", workflow vs autonomous agent, agent topology/control. DO NOT USE FOR: single prompt/RAG retrieval design (use rag skills); model serving (use `inference-serving-topology`); provider routing (use `model-gateway-routing`).

2026-06-0125

ai-hardware-selection

claude-dev-suite/claude-dev-suite

Selecting accelerators for AI workloads: GPU vs TPU vs NPU vs FPGA vs CPU, and the metrics that actually decide it — memory capacity & bandwidth, TOPS/ FLOPS, interconnect, and cost/Watt. Architect-level hardware-fit reasoning. USE WHEN: choosing AI hardware/accelerators, "which GPU", "TPU vs GPU", "NPU", "FPGA", "HBM/memory bandwidth", "TOPS", "cost per token", VRAM sizing for a model, training vs inference hardware, accelerator interconnect. DO NOT USE FOR: serving software topology (use `inference-serving-topology`); on-device runtimes (use `edge-inference`); generic CPU perf (use systems/hardware-aware-design).

2026-06-0125

hybrid-edge-cloud

claude-dev-suite/claude-dev-suite

Hybrid edge-cloud AI architecture: local-first inference with cloud escalation, model cascading, and splitting the workload across device and datacenter to balance latency, cost, privacy, and quality. Architect-level topology. USE WHEN: designing systems that combine on-device and cloud AI, "local-first", "cloud fallback", "model cascade", "escalation", "hybrid inference", routing by confidence/complexity, edge+cloud trade-offs. DO NOT USE FOR: pure on-device (use `edge-inference`); pure cloud serving (use `inference-serving-topology`); multi-provider API routing (use `model-gateway-routing`).

2026-06-0125

name	edge-inference
description	Edge / on-device AI inference architecture: running models on MCUs, NPUs, mobile, and mini-PCs; quantization for edge, TOPS/memory/energy budgets, TinyML, and the latency case for on-device vs cloud. Architect-level. USE WHEN: designing on-device/edge AI, "edge inference", "on-device", "NPU", "TinyML", "quantization", "Jetson", "Coral", "Hailo", local LLM on small hardware, offline/low-latency inference, energy-constrained ML. DO NOT USE FOR: cloud GPU serving (use `inference-serving-topology`); choosing datacenter accelerators (use `ai-hardware-selection`); RAG app code (use rag skills).
allowed-tools	Read, Grep, Glob

Edge / On-Device Inference

Why on-device (the decision drivers)

Latency: on-device token/inference in ~tens of ms vs 200–500ms cloud round-trip — decisive for voice, AR, control loops.
Privacy / offline / cost: data never leaves the device; works without connectivity; no per-call cloud cost.
Cost of doing it: tight memory/energy/thermal budgets; smaller models; more engineering.

The hardware tiers (match model to silicon)

Tier	Silicon	Typical model
MCU / TinyML	Cortex-M + tiny NPU (sub-$1 class)	KB-MB models: keyword spotting, anomaly detection (TFLite Micro)
Mobile / AI-PC	Phone NPU, laptop NPU (tens of TOPS)	Quantized 3–8B LLMs, vision
Edge box	Jetson Orin/Thor, Coral, Hailo (40+ TOPS)	7–13B LLMs, multi-camera vision

Architectural levers

Quantization is the key enabler: FP16 → INT8 → INT4 trades accuracy for memory/throughput/energy. Most edge LLMs run INT4/INT8. Validate accuracy loss.
Model choice: small instruct/distilled models (e.g. 3–8B class) and vision models sized to the NPU's memory bandwidth, not just TOPS.
Runtime: TFLite/LiteRT, ONNX Runtime, ExecuTorch, llama.cpp/Ollama, vendor SDKs (TensorRT for Jetson). Pick what targets the accelerator.
Budgets: state the TOPS, RAM, and energy-per-inference budget up front — they bound everything. Batch=1, KV-cache memory dominates LLM RAM.

When to recommend edge vs cloud vs hybrid

Hard latency / offline / privacy / per-unit cost → on-device.
Large model / variable load / centralized updates → cloud serving.
Both needs → hybrid (small local model + cloud escalation): see hybrid-edge-cloud.