atomr-infer

Use when deploying atomr-infer to a cluster — choosing feature flags, the `remote-only` invariant, `atomr-infer serve --config` project files, hot-swap and credential rotation, container image sizes. Triggers on writing a `inference.toml`, a Dockerfile for an inference service, configuring `kubectl apply` / Helm, or asking "how do I deploy atomr-infer to production".

Use when adding a new backend to atomr-infer — implementing `ModelRunner`, plugging into the rollup, slotting a new crate into the publish dep-order. Triggers on writing `impl ModelRunner for ...`, asking "how do I add Bedrock / Cohere / a custom CUDA kernel package", or considering a fork.

Use when composing multi-runtime pipelines in atomr-infer — hybrid local→remote escalation, fallback on `RateLimitExceeded` / `CircuitOpen`, dynamic batching, cascade routing, replica pools, hot-swap, speculative decoding, MoE. Triggers on writing an actor that calls multiple `Deployment`s, using `inference::accel_patterns::*`, or asking "how do I escalate from local to OpenAI when confidence is low".

Use when standing up the first atomr-infer deployment in a consumer project, choosing feature flags for the `inference` rollup, or writing a `Deployment` value object. Triggers on adding `inference = ...` to Cargo.toml, writing a `Deployment {...}` literal, configuring `atomr-infer serve`, or asking "how do I get atomr-infer running".

Use when choosing a backend for a model deployment in atomr-infer, configuring `RuntimeConfig`, deciding between local Rust-native (Candle / cudarc / mistralrs) vs Python (vLLM) vs FFI (TensorRT / ORT) vs remote (OpenAI / Anthropic / Gemini / LiteLLM). Triggers on writing a `Deployment.runtime = ...` field, choosing a feature flag, asking "what's the right backend for X model".

Use when debugging atomr-infer symptoms — `InferenceError::RateLimited` storms, `CircuitOpen`, `ContentFiltered`, `CudaContextPoisoned`, mailbox backpressure, gateway 429s, missing fallbacks, the `remote-only` invariant violation. Triggers on a stack trace mentioning `inference-*`, an unexplained typed error, or asking "why is my deployment returning X".

Use when standing up zero-config local Gemma 4 on a workstation with a working GPU + Python + vLLM, picking a Gemma 4 variant (E2B / E4B, base vs `-it`), or wiring the `gemma-default` feature. Triggers on enabling `--features gemma-default`, setting `ATOMR_INFER_GEMMA_*` env vars, asking "how do I run Gemma locally", or seeing a `gemma-local` deployment auto-appear.

Use when wiring a remote inference provider (OpenAI / Anthropic / Gemini / LiteLLM) in a atomr-infer project — credentials, rate limits, retries, circuit breakers, cost estimation, fallback chains. Triggers on configuring `OpenAiConfig` / `AnthropicConfig` / `GeminiConfig` / `LiteLlmConfig`, handling 429 / `InferenceError::RateLimited` / `CircuitOpen`, or asking "how do I add OpenAI to my deployment".