| name | style-transfer-training |
| description | Route an image style-transfer task to the correct training architecture before any money is spent. Use when a user has a corpus of images in a target style (art style, historical photos, product shots, illustration) and wants to apply that look to new user-supplied photos. Covers the paired-vs-unpaired decision gate, style LoRA vs edit-model LoRA, dataset curation, identity preservation, and mandatory cheap-baseline evaluation. Trigger on "train a style LoRA", "apply my corpus style to a photo", "image to image style transfer", "reimagine this portrait as X", "fine-tune on my image dataset", or any request to train a diffusion model on a private image collection. |
| version | 1.0.0 |
| author | delorenj |
| license | MIT |
| metadata | {"hermes":{"tags":["image-generation","lora","style-transfer","diffusion","fal.ai","flux","qwen","evaluation","fine-tuning"],"category":"mlops"}} |
Style Transfer Training
Getting a private image corpus to restyle user photos is a solved, commodity
problem. It goes wrong almost exclusively at the architecture-selection step,
in the first thirty seconds, and everything downstream becomes heroic effort to
rescue a choice that was never questioned.
Your job is the decision gate, not the pipeline. Work the gate before you
write a line of training code.
The Decision Gate
Answer these three in order. Do not skip ahead.
1. Do you have PAIRED data?
A pair is the same subject, twice, differing only in style — a "before" and
an "after" of the identical scene.
| You have | Train | Do NOT train |
|---|
| Unpaired style exemplars (a folder of images that all look like the target) | Style LoRA on a text-to-image base (Flux, Qwen-Image, SDXL) | An edit-model LoRA |
| Genuine before/after pairs of the same subject | Edit-model LoRA (Qwen-Image-Edit, Flux Kontext) | — |
Almost everyone has the first row and thinks they need the second.
The single most common failure: reaching for an edit model
(Qwen-Image-Edit, Flux Kontext) because it markets itself as
"content-preserving style transfer," then discovering it requires paired
supervision you do not have, then manufacturing fake pairs to feed it.
If you find yourself fabricating the missing half of a dataset, stop —
you picked the wrong trainer. Go back to row one.
2. Have you tested zero-shot?
Modern base edit models are strong. Before training anything, run the plain base
model on a representative input with your target prompt. One inference,
~$0.02–0.05. You may need no training at all, and either way you now have the
floor your LoRA must beat.
Never skip this. It is the cheapest decisive experiment available and it
costs less than 1% of a training run.
3. Where does identity preservation come from?
Inference time, not training time. This trips people up constantly.
- Edit models condition on the input image in latent space — subject
preservation is inherent to the architecture. You do not teach it.
- For a t2i style LoRA, identity comes from the inference mode: img2img at
moderate denoise (~0.5–0.7), or a face-identity adapter (IP-Adapter, InstantID,
PuLID) layered on top.
If someone proposes a training-data trick to "teach identity preservation," they
have misdiagnosed the problem. See references/anti-patterns.md.
Dataset Curation
Volume is not the asset you think it is. Style LoRAs converge on 20–150
well-curated images. A corpus of thousands of mixed-quality images will train a
worse style LoRA than 100 hand-picked ones, because the model averages toward
whatever is most common — usually background, margin, and scan artifact.
Curation checklist:
- Consistent framing. Pick one crop convention and hold it.
- Subject dominates the frame. If the subject occupies 25% of the image and
the rest is border, mount, or margin, the LoRA learns the border.
- Decide about frames, mounts, and borders explicitly. If the physical
artifact (card mount, deckled edge, polaroid frame) is part of the style
goal, keep it consistently. If not, crop it out consistently. Never mix
policies within one dataset.
- Cull aggressively. Damaged, off-style, badly exposed, or atypical images
hurt more than they help.
- Captioning. For style-only training, a single trigger token often beats
detailed captions. Many trainers expose an
is_style flag that disables
subject segmentation and auto-captioning — use it.
Cost Ladder
Always run the cheapest decisive experiment first.
| Step | Cost | Purpose |
|---|
| 1. Zero-shot base, held-out input | ~$0.02 | Establishes the floor. May end the project. |
| 2. Existing LoRA (if any), same seed/settings | ~$0.02 | Proves whether the LoRA does anything at all. |
| 3. Curate 100–150 images | free | Highest-leverage step in the whole process. |
| 4. Style LoRA training run | ~$2–4 | Only after 1–3. |
| 5. Identity adapter, if drift observed | ~$0.03/img | Add only if needed. |
Do not proceed to a step until the previous one has actually run and been
looked at by a human.
Evaluation
Evaluation discipline is where agent-built pipelines fail silently. Read
references/evaluation-protocol.md before writing any eval code. The
non-negotiables:
- Eval input must come from the deployment domain (what real users will
upload), never from the training corpus.
- Always compare base vs. LoRA on identical input, seed, and settings.
- An API returning an image is not success. A human must look at it.
Provider Notes
Verify current endpoints and pricing before quoting — this space moves fast.
As of early-to-mid 2026 on fal.ai:
fal-ai/flux-lora-fast-training — unpaired style LoRA, has an explicit
is_style flag that disables segmentation/auto-captioning. ~$2/run.fal-ai/flux-lora-portrait-trainer — portrait-tuned variant.fal-ai/qwen-image-trainer — Qwen-Image (t2i) LoRA.fal-ai/qwen-image-edit-trainer / -2509-trainer — edit model, requires
real _start/_end pairs. Only reach for this if the gate says row two.
Managed API (fal, Replicate) for prototyping; serverless GPU (Modal) once
traffic justifies keeping something warm.
Hard Guardrails
These are prohibitions, not preferences. Full rationale in
references/anti-patterns.md.
- Never construct identity pairs (
_start byte-identical to _end). The
global optimum of that objective is "copy the input" — you are paying to
train a no-op.
- Never evaluate on training-corpus images. A test that cannot fail has
told you nothing.
- Never report API success as model success.
- Never fabricate missing training data with non-equivalent operations.
Pillow cannot colorize a grayscale image — it can gradient-map a tint, which
is a different thing. Synthetic data that does not land in the real
deployment domain creates a domain gap that surfaces at inference.
- Never skip the zero-shot baseline before a paid training run.
Reference Files
references/anti-patterns.md — the failure modes above, with the mechanism
behind each and how to detect them in an existing pipeline.references/evaluation-protocol.md — the base-vs-LoRA comparison procedure
and the regression checks that keep an eval honest.
