create-workflow-block

name

create-workflow-block

description

Author a new Roboflow Workflows block under `inference/core/workflows/core_steps/**`. Trigger on a new/changed `vN.py` or `vN_tensor.py` defining a `WorkflowBlockManifest` + `WorkflowBlock` pair; new `load_blocks()`/`KINDS_SERIALIZERS` entries in `core_steps/loader.py` (either flag branch); new `Kind(...)` in `execution_engine/entities/types.py`; or a request to "create/add a workflow block" or "expose X as a workflow step" (model / transform / sink / visualization / flow control).

Creating a Workflow block

The playbook for authoring a new Roboflow Workflows block — a roboflow_core/{family}@v{N} step under inference/core/workflows/core_steps/.

How to use this skill. Read What a Workflow is and Anatomy of a block, run the New-block checklist, then open the one Block category reference under references/ that matches what you're building for its mental model and canonical example blocks. Always match an existing sibling block before writing a line. Authoritative depth lives in docs/workflows/**; this skill is a map into it. Reviewer-side counterpart: review-workflows-blocks.

What a Workflow is (mental model)

A Workflow is a JSON definition (inputs + steps + outputs) written in the Workflows language. It is not run directly — a Compiler parses it against the pool of installed blocks, and an Execution Engine (EE) runs the resulting DAG. As a block author you never touch the EE internals; you write a Python class the EE instantiates and calls.

• Definition → Compiler → Execution Engine. Each definition declares an EE version (version: 1.x means >=1.x,<2.0.0). A step is one instance of a block ({"type": "...@v1", "name": "my_step", ...}); step inputs are either literals or selectors ($inputs.<name>, $steps.<step>.<output>) resolved at runtime. Concepts: docs/workflows/understanding.md, docs/workflows/workflow_execution.md, docs/workflows/definitions.md.
• Data flows as typed kinds. A step output has a kind (e.g. image, object_detection_prediction); a downstream input declaring the same kind is assumed compatible — compile-time verification without runtime checks. Kinds are defined as Kind(...) constants in inference/core/workflows/execution_engine/entities/types.py (the per-kind docs under docs/workflows/kinds/ are build-time generated).
• Batch orientation & dimensionality. The EE is batch-oriented: it fans data through steps as batches, and each datapoint sits at a dimensionality level. A block can keep, increase (crop-per-detection), or decrease that level — the single most important concept for non-trivial blocks. See docs/workflows/workflow_execution.md and docs/workflows/create_workflow_block.md.
• Compiler/EE roles: docs/workflows/workflows_compiler.md, docs/workflows/workflows_execution_engine.md.

Anatomy of a block

A block is two classes in a vN.py module: a WorkflowBlockManifest (the schema/prototype for a step declaration) and a WorkflowBlock (the logic). Primary guide: docs/workflows/create_workflow_block.md. Confirmed real shape: inference/core/workflows/core_steps/transformations/detection_offset/v1.py.

Manifest — a pydantic model subclassing WorkflowBlockManifest (from inference.core.workflows.prototypes.block):

• type: Literal["roboflow_core/<family>@v1", "<Alias>"] — the discriminator the Compiler uses to pick this manifest when parsing a step. The Literal may carry a legacy alias as a second value (see the type field in detection_offset/v1.py, ["roboflow_core/detection_offset@v1", "DetectionOffset"]).
• Inputs are ordinary fields. Use Selector(kind=[...]) for data references and Union[<literal type>, Selector(kind=[...])] when a value may be hardcoded or selected (e.g. Union[PositiveInt, Selector(kind=[INTEGER_KIND])]). Wrap each with pydantic Field(description=..., examples=...) — the user-facing docs are generated from these descriptions/examples (see Docs are autogenerated below). model_config = ConfigDict(json_schema_extra={...}) carries UI metadata (name, block_type, icon).
• @classmethod describe_outputs() -> List[OutputDefinition] — one OutputDefinition(name=..., kind=[...]) per output the run() dict must supply. For manifest-dependent outputs, return a single name="*", kind=[WILDCARD_KIND] and add instance method get_actual_outputs().
• @classmethod get_execution_engine_compatibility() -> Optional[str] — semver range, e.g. ">=1.3.0,<2.0.0"; gates loading.
• Batch/dimensionality hooks (all classmethods, opt-in): get_parameters_accepting_batches() → names of params delivered as Batch[...]; get_parameters_accepting_batches_and_scalars() → mixed params; get_output_dimensionality_offset() → +1/-1 when the block changes nesting level.

Block — subclass WorkflowBlock:

• @classmethod get_manifest() -> Type[WorkflowBlockManifest] returns the manifest class.
• def run(self, ...) -> BlockResult — the EE calls this with kwargs matching manifest input names. Image inputs arrive as WorkflowImageData (use .numpy_image); scalar params arrive as plain values; batch params (if declared) arrive as Batch[...]. Return a dict {output_name: value} (or, for batch/dimensionality-increasing blocks, a list of such dicts — a None entry drops that datapoint downstream). __init__ may hold reusable state (persists across run() calls, e.g. per-frame video).
• Imports: WorkflowBlock, WorkflowBlockManifest, BlockResult from inference.core.workflows.prototypes.block; Batch, OutputDefinition, WorkflowImageData from inference.core.workflows.execution_engine.entities.base.

Universal invariants (every category)

These hold regardless of category — the per-category references only add nuance on top.

• Output keys == describe_outputs(). Every key your run() returns must be named in describe_outputs(), on every return path (empty / error / early-exit branches included), and the names must stay stable across versions — downstream steps bind to them.
• A block is invisible until registered. For roboflow_core, add the import + load_blocks() list entry in inference/core/workflows/core_steps/loader.py (see DetectionOffsetBlockV1). External plugins expose it via load_blocks() in the plugin __init__.py (WORKFLOWS_PLUGINS="plugin_a,plugin_b"). Forgetting this means the block does not exist to the EE.
• New kind ⇒ serializer. A kind not already round-trippable needs a serializer/deserializer pair registered in KINDS_SERIALIZERS/KINDS_DESERIALIZERS in loader.py (e.g. deserialize_rgb_color_kind for RGB_COLOR_KIND), or a new key written into sv.Detections.data handled in core_steps/common/serializers.py. Unregistered kinds fall through to serialize_wildcard_kind and may be dropped on the REMOTE / API boundary. Reuse existing detection/image kinds unless you truly need a new one.
• Model/resource dependencies are declared. Any block that loads a model or external resource overrides discover_dependent_resources() on its manifest (helpers roboflow_platform_model() / third_party_model(); see google_gemini/v3.py) so workflow resource discovery and model pre-loading see the dependency before execution starts. This is a general block contract — not a tensor-native feature — and video pipelines rely on it to pre-load models ahead of the first frame.
• Parent-coordinate attach is load-bearing. Any block that produces or moves sv.Detections must attach parent-coordinate metadata (attach_parents_coordinates_to_batch_of_sv_detections / attach_parents_coordinates_to_sv_detections in core_steps/common/utils.py, or WorkflowImageData.create_crop(...)). Omitting it silently breaks re-projecting boxes onto the original frame for cropped/tiled inputs.
• Stateful ⇒ restrictions + LOCAL-only where required. A block holding cross-frame state must key it by image.video_metadata.video_identifier, evict it (bounded cache), and declare get_restrictions() returning the relevant RuntimeRestriction constants from inference/core/workflows/prototypes/block.py — STATEFUL_VIDEO_HTTP_SOFT_RESTRICTION, STILL_IMAGE_INPUT_SOFT_RESTRICTION, COOLDOWN_HTTP_SOFT_RESTRICTION (SOFT), or a Severity.HARD restriction on StepExecutionMode.REMOTE when remote execution is incoherent (then also raise NotImplementedError in run()). State silently no-ops behind stateless/multi-replica HTTP.
• LOCAL vs REMOTE parity. Model blocks with a run_locally/run_remotely split must mirror every knob into InferenceConfiguration on the REMOTE path (note renames like confidence → confidence_threshold). A knob wired to only one path is a correctness bug, not a style nit.
• Docs are autogenerated — do NOT hand-write them. docs/workflows/blocks/<block>.md (and docs/workflows/kinds/index.md) are generated by development/docs/build_block_docs.py (write_individual_block_pages / write_kinds_docs). A hand-written page gets overwritten. Instead, put rich content into Field(description=..., examples=...) and the manifest long-description — the generator renders the page from those.

Tensor-native siblings (vN_tensor.py)

Since the tensor-data-representation merge (#2357), blocks that consume or produce images or predictions ship in pairs: the numpy implementation (vN.py, runs when ENABLE_TENSOR_DATA_REPRESENTATION is off — the default) and a tensor-native sibling (vN_tensor.py, registered under the same block name in loader.py's flag-on branch, operating on inference_models native objects — Detections/InstanceDetections, the (KeyPoints, Detections) tuple, torch tensors). Scalar/text/flow-control blocks that never touch images or predictions need only the numpy implementation.

• Siblings are standalone re-implementations. Re-implement the numpy behavior verbatim against native objects; never import the numpy manifest to bind or delegate methods (wholesale subclassing of a full class is acceptable; surgical method-binding is banned). Stability over DRY, same as versioning.
• Manifest surface must be IDENTICAL across siblings. Field set, supported model variants, and batch-parameter declarations diverging per flag make workflow definitions non-portable between deployments. Output-kind lists must match deliberately, including order — the output serializer picks the first kind whose serializer accepts the value, so kind order changes the wire payload (instance-segmentation v1–v3 siblings declaring the RLE kind first flipped serialized masks from polygon points to rle_mask under the flag).
• General contracts hold on BOTH siblings. discover_dependent_resources() is a general block contract (see Universal invariants) — the tensor-specific trap is parity: a declaration only on the numpy side silently disappears flag-on and model pre-loading skips the block's model (full cold load on the stream's first frame). Sink blocks must honor disable_sinks in both siblings too.
• Host-mirror hygiene. Native detections may carry a per-box CPU mirror of xyxy/class_id/confidence in bboxes_metadata that downstream visualizations prefer over tensor reads. Any tensor sibling that rewrites geometry must strip it (strip_host_mirror_metadata / HOST_MIRROR_KEYS in common/tensor_native.py) — a carried mirror makes downstream blocks draw the OLD geometry with no error.
• Mutation declarations. WorkflowImageData caches sibling representations lazily; a block mutating the image in place (copy_image=False paths) must call declare_numpy_image_mutated() / declare_tensor_image_mutated() on the object it mutated, or cached representations desync silently.
• Empty detections & mask carriers. Handle the zero-detections case without materializing device→host; accept BOTH mask carriers (dense torch stack and InstancesRLEMasks) wherever masks are consumed — which carrier arrives depends on producer settings and the dense-mask env override.
• Custom Python (dynamic) blocks declare tensor_compatibility in the manifest: legacy_compatibility (default) gets the engine's conversion boundary — user code still receives/returns documented sv.Detections/numpy shapes; tensor_native receives native objects directly, fails at compile time when the flag is off, and is local-execution-only.

New-block checklist

• New package dir with __init__.py + vN.py; two classes (Manifest + Block).
• type Literal roboflow_core/{family}@v{N} (+ legacy alias if replacing one).
• Inputs are Selector(kind=[...]) / Union[literal, Selector(...)] with rich Field(description=, examples=) (docs are generated from these).
• describe_outputs() matches every key run() emits — on every path (empty / error branches included).
• get_execution_engine_compatibility() set to the TRUE minimum EE version you rely on — look up which EE version introduced each capability you use in the EE changelog in the roboflow/docs repo (workflows/developer-guide/execution-engine-changelog.md); do not copy-paste the default range. If a capability you need is still under ## Unreleased, the EE version must be placed and bumped first (maintainer-coordinated) so your range can reference it.
• Batch / dimensionality hooks declared if the block batches inputs or changes nesting level.
• Registered in core_steps/loader.py (load_blocks()); any new kind in types.py + load_kinds() + a serializer/deserializer pair.
• Block loads a model/external resource: discover_dependent_resources() overridden on the manifest.
• Producing/moving detections: parent-coordinate metadata attached.
• Stateful block: state keyed by video_identifier + evicted; get_restrictions() declared; NotImplementedError on StepExecutionMode.REMOTE if remote is incoherent.
• Unit test under tests/workflows/unit_tests/core_steps/... + integration test under tests/workflows/integration_tests/execution/....
• Image/prediction block: tensor sibling vN_tensor.py authored per Tensor-native siblings (manifest surface + kind order identical; discover_dependent_resources() and, for sinks, disable_sinks on both) and registered in loader.py's flag-on branch.
• Tests pass in BOTH flag directions (CI runs both): per-file _TENSOR_ONLY/_NUMPY_ONLY skipif markers for one-mode tests; tensor-native integration tests take the shared image_as_workflow_input conftest fixture so every image input is exercised as numpy AND torch.Tensor submission; assertions mirrored across siblings.
• If the EE itself changed, add its user-facing entry under ## Unreleased in the EE changelog in the roboflow/docs repo (workflows/developer-guide/execution-engine-changelog.md); maintainers bump the EE version at release time.
• Do NOT hand-write docs/workflows/blocks/<block>.md — it is generated from manifest field descriptions.

Reviewer's checklist for block PRs: review-workflows-blocks.

Kinds & selectors (the type system)

Kinds are the Workflows type system. Each kind pairs a semantic name (image, point), a Python representation blocks receive (e.g. object_detection_prediction → sv.Detections), and an optional serialized representation for the wire. No polymorphism — express alternatives as a union, i.e. Selector(kind=[A_KIND, B_KIND]) (see the multi-kind predictions input in detection_offset/v1.py).

• Where kinds live: Kind(...) constants in inference/core/workflows/execution_engine/entities/types.py (e.g. IMAGE_KIND, OBJECT_DETECTION_PREDICTION_KIND, FLOAT_ZERO_TO_ONE_KIND, WILDCARD_KIND). Per-kind docs pages under docs/workflows/kinds/ are build-time generated from these.