Local ComfyUI image generation end-to-end: from a blank machine (download, install, start, fetch models, workflows) through queue/poll/download and chat embeds. Defaults to http://127.0.0.1:8188 (Flux.2 Klein + Qwen3-ready). Use when the user wants images and has Remedy + a chat provider — even if ComfyUI is not installed yet.
Local ComfyUI image generation end-to-end: from a blank machine (download, install, start, fetch models, workflows) through queue/poll/download and chat embeds. Defaults to http://127.0.0.1:8188 (Flux.2 Klein + Qwen3-ready). Use when the user wants images and has Remedy + a chat provider — even if ComfyUI is not installed yet.
Local image generation via ComfyUI
REST API, driven by Remedy’s built-in comfyui tool.
You can start with nothing installed. If the user asks for images and
ComfyUI is missing, follow From scratch bootstrap below (with their approval
for downloads / shell). Then generate with the native tool.
This skill pairs with Remedy Desktop (or CLI) connected to any LLM provider
(xAI, OpenAI, Ollama, …). The provider runs the agent; ComfyUI runs on the
user’s GPU/CPU for pixels.
Do notlist_dir the whole disk hunting for ComfyUI. Use action=locate /
status. Discovery is built in for every OS/user.
Agent decision tree (always start here)
Call comfyui / action=status (or locate).
API up → jump to Generate images.
Install found, API down → Start ComfyUI (use start_hint from the tool).
Nothing found → From scratch bootstrap (this is expected on a fresh PC).
After models are in place and the server is up → Generate images.
Never invent success. Never paste DSML/tool XML into chat. Prefer the comfyui
tool over raw curl (especially on Windows).
From scratch bootstrap (fresh install)
Goal: machine has no ComfyUI → user can generate a PNG from Remedy chat.
Ask once before large downloads (disk: tens of GB for full Flux.2 Klein;
smaller if using 4B). Confirm GPU type: NVIDIA (default), AMD, Intel, or CPU-only.
flux-2-klein-base-9b-fp8.safetensors (or a 4B Klein fp8 if VRAM-limited)
models/diffusion_models/
Text encoder
qwen_3_8b_fp8mixed.safetensors
models/text_encoders/
VAE
flux2-vae.safetensors
models/vae/
How to obtain them (agent guidance):
Prefer Hugging Face / official Comfy model docs for current Flux.2 Klein
and Qwen3 encoder packages. Filenames must match the table (or edit the
workflow / tool defaults if the user downloads alternate names).
ComfyUI Manager (if installed) or browser download → save into the folders above.
huggingface-cli download … is fine when the user has HF access and disk.
After downloads, restart ComfyUI so it rescans models.
VRAM guidance (agent should ask / recommend):
Hardware
Suggestion
≤8 GB VRAM
4B Klein (if available), 512×512, close other GPU apps
12–16 GB
9B fp8 often OK at 512–768
24 GB+
9B fp8 comfortable; higher res later
Rough disk: several GB to 20+ GB depending on UNET size — state this before download.
If the user only wants a smoke test and already has SD1.5/SDXL checkpoints, you
may queue a different API-format workflow they provide — but Remedy’s built-in
generate path expects the Klein filenames above unless you customize workflow JSON.
Phase D — Workflows (write / adapt)
Use the bundled starter (default)
Path (inside this skill):
scripts/workflows/txt2img_flux2_klein.json
Or let the tool build it: comfyui / action=generate with a prompt
(internally uses the same graph + injects the text prompt).
Write / edit a workflow
Preferred for custom graphs: open http://127.0.0.1:8188 → build in UI →
Save (API Format) → save .json under the project (e.g. assets/workflows/).
API format is a dict keyed by string node ids, each with class_type + inputs.
Links are [node_id, output_index] pairs — not the pretty UI format.
Edit with file_write / editor; keep filenames in loaders aligned with disk.
Queue via CLI helper or POST /prompt (see below). For chat UX, stick to
action=generate when the Klein graph is enough.
