review-topic-local-vs-remote-execution

# Review topic: Local vs remote / hosted execution compatibility ## When this applies Load when a diff does any of: - Adds/edits a `run()` dispatch on `self._step_execution_mode` (LOCAL vs REMOTE) or a `run_locally` / `run_remotely` pair. - Adds/edits a block's `get_restrictions()` or `get_air_gapped_availability()` where the block reaches the network or needs same-process / LAN access (PTZ/ONVIF camera, local filesystem, GPU-resident handles). - Touches runtime routing env: `USE_INFERENCE_MODELS`, `DISABLED_INFERENCE_MODELS_BACKENDS`, `WORKFLOWS_STEP_EXECUTION_MODE`, `WORKFLOWS_REMOTE_API_TARGET`, `LOCAL_INFERENCE_API_URL`, `_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT`, or backend selection in `inference/core/models/inference_models_adapters.py` / `inference/models/utils.py`. - Gates behavior on hardware/deps: CUDA version, Jetson (`jetson`, `l4t`, JP5/JP6), ONNX execution providers, TRT — where one runtime path is fixed/tested but siblings are not. - Changes hosted/serverless/dedicated cold-start / processing-time headers or WebRTC/modal worker plumbing. This is a behavior/content trigger, not a single directory — the paths above are hints. **Cross-refs (owned elsewhere, not reviewed here):** - Serverless auth/usage fail-open behavior → `review-topic-auth-and-tenant-security`. - Stateful-block restriction contract (in-process trackers/counters/cooldown/append-log needing a SOFT restriction) → `review-topic-workflow-state-management`. ## Review checklist - **BLOCK** — `run()` dispatches on `StepExecutionMode` with an explicit `else: raise ValueError` for unknown modes (missing else swallows new modes silently). - **BLOCK** — a runtime-routing change (`USE_INFERENCE_MODELS`, `DISABLED_INFERENCE_MODELS_BACKENDS`, platform default) leaves BOTH the on and off value working and tested; the legacy ONNX registry entries are still present. Breaking the DEFAULT runtime with no working fallback is a production regression. - **BLOCK** — a block that raises / hard-fails on a runtime the platform will still offer carries a matching `RuntimeRestriction` (right `severity`, `applies_to_step_execution_modes`, `applies_to_runtimes`); metadata that lies makes the UI offer a path that fails at runtime. - **FLAG** — `run_locally` and `run_remotely` have matching signatures and forward every manifest param; a param wired into only one path is a divergence users hit only on the other runtime. - **FLAG** — a new network/LAN/same-process block overrides `get_air_gapped_availability()` if it needs internet, and adds a HARD restriction excluding hosted/dedicated (LAN) or REMOTE step execution (same-process). - **FLAG** — a platform-specific default (`_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT`, Windows/Jetson forks) keeps the non-default OS working; the diff must not assume one platform. - **FLAG** — CUDA/Jetson/ONNX-provider introspection changes are correct on every runtime, not just the author's box (one-runtime introspection bugs mis-route model loading). - **NIT** — restriction present but mis-scoped or wrong severity (HARD where SOFT fits, missing `applies_to_runtimes`). ### Not blocking - Do NOT demand a hosted-platform test for a change with no observable hosted/serverless behavior difference (pure local refactor, docstrings, typing). Hosted coverage under `tests/inference/hosted_platform_tests/` is required only when the diff changes what a request returns on lambda / serverless / dedicated. - Do NOT demand a restriction for a block that is genuinely runtime-agnostic (stateless, no network, no LAN, no GPU-handle assumption). - Do NOT demand a flip-the-flag-both-ways test for env vars that only tune values (URLs, timeouts) without forking code paths. ## Standards 1. **Execution-mode dispatch is exhaustive.** `run()` branches on `self._step_execution_mode` and terminates in `else: raise ValueError(f"Unknown step execution mode: ...")`. Canonical shape: `inference/core/workflows/core_steps/models/roboflow/object_detection/v1.py` — `run()` branches `StepExecutionMode.LOCAL` / `StepExecutionMode.REMOTE`, `run_locally` via `self._model_manager`, `run_remotely` via `InferenceHTTPClient` honoring `WORKFLOWS_REMOTE_API_TARGET` / `LOCAL_INFERENCE_API_URL`. (`StepExecutionMode` is defined in `inference/core/workflows/prototypes/block.py` and re-exported from `core_steps/common/entities.py`.) 2. **Local and remote paths stay semantically equivalent.** `run_locally` and `run_remotely` accept the same params and return the same shape; every manifest field is forwarded on both paths (one-path param wiring is a silent divergence). 3. **Metadata must match behavior.** A block's `get_restrictions()` / `get_air_gapped_availability()` declarations are consumed by the describe endpoint and the platform UI to decide where a block may run. The declaration and the actual `run()` guard must agree — a code path that raises for a runtime needs a `RuntimeRestriction` for it, and vice-versa. LAN shape: `onvif_movement/v1.py::NO_LAN_FROM_HOSTED_RESTRICTION` (HARD + `Runtime.HOSTED_SERVERLESS`/`Runtime.DEDICATED_DEPLOYMENT` because hosted can't reach a customer LAN). Do NOT gate a non-model block on `StepExecutionMode` - that enum only says where model steps run; a block with no remote path always executes in-process, and a REMOTE-scoped restriction is silent under the default LOCAL mode on hosted runtimes. 4. **Runtime routing has no orphaned path.** A new backend, env flag, or hardware gate leaves BOTH the on and off value working and tested; the legacy path (ONNX registry entries) stays intact. Backend routing lives in `inference/core/models/inference_models_adapters.py` + `inference/core/env.py` (`USE_INFERENCE_MODELS`, `_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT`, `DISABLED_INFERENCE_MODELS_BACKENDS`). Rollout history: #2096 added per-deployment `DISABLED_INFERENCE_MODELS_BACKENDS`; #2128 added missing registry entries so RFDETR-NAS works under inference-models; #2136 reverted making inference-models the default backend (the default flip is contentious — review carefully). 5. **Per-platform introspection is correct on every runtime.** CUDA-version / Jetson / ONNX-provider detection that gates model loading must hold across platforms, not just the author's (CUDA acquisition fixed in #2515; Jetson environment detection in runtime introspection + dockerfiles fixed in #1796). 6. **Hosted-only code paths get hosted coverage.** Behavior that differs on lambda / serverless / dedicated is exercised in `tests/inference/hosted_platform_tests/` (parametrized over `PlatformEnvironment` via `conftest.py`; see `test_workflows.py`, `test_roboflow_models.py`). Hosted-only plumbing regresses silently under local-only tests (remote cold-start header aggregation reverted in #2222; `ROBOFLOW_ENVIRONMENT` propagation to the modal webrtc worker fixed in #2179; dedicated `hosted=True`/`DEDICATED_DEPLOYMENT_ID` usage attribution fixed in #817). ## Key files & Reference PRs Files: - `inference/core/workflows/prototypes/block.py` — the contract: `StepExecutionMode`, `Runtime`, `Severity`, `RuntimeRestriction`, presets (`STATEFUL_VIDEO_HTTP_SOFT_RESTRICTION`, `COOLDOWN_HTTP_SOFT_RESTRICTION`), and `get_restrictions` / `get_air_gapped_availability` / `get_execution_engine_compatibility` hooks. Read first. - `inference/core/workflows/core_steps/models/roboflow/object_detection/v1.py` — canonical LOCAL/REMOTE dispatch. - `inference/core/workflows/core_steps/sinks/onvif_movement/v1.py` — LAN block done right (one HARD runtime-keyed restriction, shared with `v1_tensor.py`; no `step_execution_mode` gate). - `inference/core/workflows/core_steps/cache/common.py` — in-process cache caveat done right (SOFT, runtime-keyed, not narrowed by step mode or input mode). - `inference/core/models/inference_models_adapters.py` + `inference/core/env.py` — backend routing with per-platform default and per-deployment disable. - `tests/inference/hosted_platform_tests/conftest.py` (+ `test_workflows.py`, `test_roboflow_models.py`) — hosted/serverless/localhost parametrization. Reference PRs: #2096, #2128, #2136 (backend rollout / default flip), #2515, #1796 (per-platform introspection), #2222, #2179, #817 (hosted-only plumbing).