comfyui-node-datatypes

# ComfyUI Data Types ComfyUI uses specific data types for node inputs and outputs. Understanding tensor shapes and data formats is essential. ## Complete Type Reference ### Tensor/Data Types | Type | V3 Class | Format | Description | |---|---|---|---| | IMAGE | `io.Image` | `torch.Tensor [B,H,W,C]` float32 0-1 | Batch of RGB images | | MASK | `io.Mask` | `torch.Tensor [H,W]` or `[B,H,W]` float32 0-1 | Grayscale masks | | LATENT | `io.Latent` | `{"samples": Tensor[B,C,H,W] or [B,C,T,H,W], "noise_mask"?: Tensor, "batch_index"?: list[int], "type"?: str}` | Latent space (4D image / 5D video) | | CONDITIONING | `io.Conditioning` | `list[tuple[Tensor, PooledDict]]` | Text conditioning with pooled outputs | | AUDIO | `io.Audio` | `{"waveform": Tensor[B,C,T], "sample_rate": int}` | Audio data | | VIDEO | `io.Video` | `VideoInput` ABC | Video data (abstract base class) | | SIGMAS | `io.Sigmas` | `torch.Tensor` 1D, length steps+1 | Noise schedule | | NOISE | `io.Noise` | Object with `generate_noise()` | Noise generator | | LORA_MODEL | `io.LoraModel` | `dict[str, torch.Tensor]` | LoRA weight deltas | | LOSS_MAP | `io.LossMap` | `{"loss": list[torch.Tensor]}` | Loss map | | TRACKS | `io.Tracks` | `{"track_path": Tensor, "track_visibility": Tensor}` | Motion tracking data | | WAN_CAMERA_EMBEDDING | `io.WanCameraEmbedding` | `torch.Tensor` | WAN camera embeddings | | LATENT_OPERATION | `io.LatentOperation` | `Callable[[Tensor], Tensor]` | Latent transform function | | TIMESTEPS_RANGE | `io.TimestepsRange` | `tuple[int, int]` | Range 0.0-1.0 | | DICT | `io.Dict` | `dict` | Generic dictionary | | ARRAY | `io.Array` | `list` | Generic list/array | ### Model Types (opaque, typically pass-through) | Type | V3 Class | Python Type | |---|---|---| | MODEL | `io.Model` | `ModelPatcher` | | CLIP | `io.Clip` | `CLIP` | | VAE | `io.Vae` | `VAE` | | CONTROL_NET | `io.ControlNet` | `ControlNet` | | CLIP_VISION | `io.ClipVision` | `ClipVisionModel` | | CLIP_VISION_OUTPUT | `io.ClipVisionOutput` | `ClipVisionOutput` | | STYLE_MODEL | `io.StyleModel` | `StyleModel` | | GLIGEN | `io.Gligen` | `ModelPatcher` (wrapping Gligen) | | UPSCALE_MODEL | `io.UpscaleModel` | `ImageModelDescriptor` | | BACKGROUND_REMOVAL | `io.BackgroundRemoval` | `BackgroundRemovalModel` (e.g. BiRefNet) | | LATENT_UPSCALE_MODEL | `io.LatentUpscaleModel` | Any | | SAMPLER | `io.Sampler` | `Sampler` | | GUIDER | `io.Guider` | `CFGGuider` | | HOOKS | `io.Hooks` | `HookGroup` | | HOOK_KEYFRAMES | `io.HookKeyframes` | `HookKeyframeGroup` | | MODEL_PATCH | `io.ModelPatch` | Any | | AUDIO_ENCODER | `io.AudioEncoder` | Any | | AUDIO_ENCODER_OUTPUT | `io.AudioEncoderOutput` | Any | | PHOTOMAKER | `io.Photomaker` | Any | | POINT | `io.Point` | Any | | FACE_ANALYSIS | `io.FaceAnalysis` | Any | | BBOX | `io.BBOX` | Any | | SEGS | `io.SEGS` | Any | ### 3D Types | Type | V3 Class | Python Type | Description | |---|---|---|---| | MESH | `io.Mesh` | `MESH(vertices, faces)` | 3D mesh with vertices + faces tensors | | VOXEL | `io.Voxel` | `VOXEL(data)` | Voxel data tensor | | SPLAT | `io.Splat` | `SPLAT` | Gaussian splat data | | FILE_3D | `io.File3DAny` | `File3D` | Any supported 3D format | | FILE_3D_GLB | `io.File3DGLB` | `File3D` | Binary glTF | | FILE_3D_GLTF | `io.File3DGLTF` | `File3D` | JSON-based glTF | | FILE_3D_FBX | `io.File3DFBX` | `File3D` | FBX format | | FILE_3D_OBJ | `io.File3DOBJ` | `File3D` | OBJ format | | FILE_3D_STL | `io.File3DSTL` | `File3D` | STL format (3D printing) | | FILE_3D_USDZ | `io.File3DUSDZ` | `File3D` | Apple AR format | | FILE_3D_PLY | `io.File3DPLY` | `File3D` | PLY (point cloud / splat) | | FILE_3D_SPLAT | `io.File3DSPLAT` | `File3D` | .splat gaussian splat file | | FILE_3D_SPZ | `io.File3DSPZ` | `File3D` | Compressed splat (.spz) | | FILE_3D_KSPLAT | `io.File3DKSPLAT` | `File3D` | .ksplat format | | FILE_3D_SPLAT_ANY | `io.File3DSplatAny` | `File3D` | Any splat format | | FILE_3D_POINT_CLOUD_ANY | `io.File3DPointCloudAny` | `File3D` | Any point cloud format | | SVG | `io.SVG` | `SVG` | Scalable vector graphics | | LOAD_3D | `io.Load3D` | `Model3DDict` (see below) | 3D model with renders | | LOAD_3D_ANIMATION | `io.Load3DAnimation` | Same as Load3D | Animated 3D model | | LOAD3D_CAMERA | `io.Load3DCamera` | `CameraInfo` (see below) | 3D camera info | | LOAD3D_MODEL_INFO | `io.Load3DModelInfo` | `list[Model3DTransform]` | Per-model transforms (position/quaternion/scale) | **Load3D.Model3DDict**: `{"image": str, "mask": str, "normal": str, "camera_info": CameraInfo, "recording"?: str, "model_3d_info"?: list[Model3DTransform]}` **Load3DCamera.CameraInfo** (right-handed, Y-up, camera looks down -Z): required keys `position`, `target`, `zoom`, `cameraType` (`'perspective' | 'orthographic'`); optional keys `quaternion` (camera world rotation), `fov` (vertical, degrees, perspective only), `aspect`, `near`, `far`, `frustum` (orthographic only: `{left, right, top, bottom}`). **Load3DModelInfo.Model3DTransform**: `{"position": dict, "quaternion": dict, "scale": dict}` in world space. ### Widget Types (create UI controls) | Type | V3 Class | Python Type | Description | |---|---|---|---| | INT | `io.Int` | `int` | Integer with min/max/step | | FLOAT | `io.Float` | `float` | Float with min/max/step/round | | STRING | `io.String` | `str` | Text (single/multi-line) | | BOOLEAN | `io.Boolean` | `bool` | Toggle with labels | | COMBO | `io.Combo` | `str` | Dropdown selection | | COMBO (multi) | `io.MultiCombo` | `list[str]` | Multi-select dropdown | | COLOR | `io.Color` | `str` (hex) | Color picker, default `#ffffff` | | COLORS | `io.Colors` | `list[str]` (hex) | Color palette (list of colors) | | BOUNDING_BOX | `io.BoundingBox` | `{"x": int, "y": int, "width": int, "height": int}` | Rectangle region | | BOUNDING_BOXES | `io.BoundingBoxes` | `list[{"x", "y", "width", "height", "metadata": dict}]` | Multiple labeled regions | | CURVE | `io.Curve` | `list[tuple[float, float]]` | Spline curve points | | RANGE | `io.Range` | `RangeInput` (min/max + optional midpoint) | Levels/range editor with gradient display | | IMAGECOMPARE | `io.ImageCompare` | `dict` | Image comparison widget | | WEBCAM | `io.Webcam` | `str` | Webcam capture widget | | HISTOGRAM | `io.Histogram` | `list[int]` | Histogram bin counts | ### Special Types | Type | V3 Class | Description | |---|---|---| | `*` (ANY) | `io.AnyType` | Matches any type | | COMFY_MULTITYPED_V3 | `io.MultiType` | Accept multiple specific types on one input | | COMFY_MATCHTYPE_V3 | `io.MatchType` | Generic type matching across inputs/outputs | | COMFY_AUTOGROW_V3 | `io.Autogrow` | Dynamic growing inputs | | COMFY_DYNAMICCOMBO_V3 | `io.DynamicCombo` | Combo that reveals sub-inputs per option | | COMFY_DYNAMICSLOT_V3 | `io.DynamicSlot` | Connection slot that reveals sub-inputs when connected (not yet used by core nodes) | | FLOW_CONTROL | `io.FlowControl` | Internal testing only | | ACCUMULATION | `io.Accumulation` | Internal testing only | ## IMAGE Type Images are `torch.Tensor` with shape `[B, H, W, C]`: - **B** = batch size (1 for single image) - **H** = height in pixels - **W** = width in pixels - **C** = channels (3 for RGB, values 0.0-1.0) ```python import torch import numpy as np from PIL import Image as PILImage class ImageProcessor(io.ComfyNode): @classmethod def define_schema(cls): return io.Schema( node_id="ImageProcessor", display_name="Image Processor", category="image", inputs=[io.Image.Input("image")], outputs=[io.Image.Output("IMAGE")], ) @classmethod def execute(cls, image): b, h, w, c = image.shape result = torch.clamp(image * 1.5, 0.0, 1.0) return io.NodeOutput(result) ``` ### Loading / Saving Images ```python from PIL import ImageOps # Load from file → tensor def load_image(path): img = PILImage.open(path) img = ImageOps.exif_transpose(img) # fix rotation from camera EXIF if img.mode == "I": # handle 16-bit images img = img.point(lambda i: i * (1 / 255)) img = img.convert("RGB") return torch.from_numpy(np.array(img).astype(np.float32) / 255.0).unsqueeze(0) # Tensor → save to file def save_image(tensor, path): if tensor.dim() == 4: tensor = tensor[0] PILImage.fromarray(np.clip(255.0 * tensor.cpu().numpy(), 0, 255).astype(np.uint8)).save(path) # Batch operations batch = torch.cat([img1, img2], dim=0) # stack into batch single = image[i] # extract from batch [H,W,C] single_batch = image.unsqueeze(0) # add batch dim [1,H,W,C] ``` ## MASK Type `torch.Tensor` with shape `[H, W]` or `[B, H, W]`, values 0.0-1.0. ```python # Invert mask inverted = 1.0 - mask # Mask ↔ Image conversion alpha = mask.unsqueeze(0).unsqueeze(-1) # [1,H,W,1] gray_mask = 0.299*img[:,:,:,0] + 0.587*img[:,:,:,1] + 0.114*img[:,:,:,2] image_from_mask = mask.unsqueeze(-1).repeat(1, 1, 1, 3) # [B,H,W,3] # Ensure batch dim if mask.dim() == 2: mask = mask.unsqueeze(0) # [1, H, W] ``` ## LATENT Type Dict with typed keys: ```python class LatentDict(TypedDict): samples: torch.Tensor # [B, C, H, W] (image) or [B, C, T, H, W] (video) - required noise_mask: NotRequired[torch.Tensor] batch_index: NotRequired[list[int]] type: NotRequired[str] # only for "audio", "hunyuan3dv2" ``` **Image models** (SD1.5, SDXL, SD3, Flux): 4D `[B, C, H, W]` — SD1.5/SDXL = 4 channels, SD3/Flux = 16 channels. Latent dimensions are 1/8 of pixel dims.