| name | qwen-image-edit-aipc-finetune-08-validation |
| description | Step 8 of 8 (final) of the Qwen-Image-Edit AI PC fine-tuning walkthrough. Use after qwen-image-edit-aipc-finetune-07-training. Validate that the trained LoRA learned the intended edit: optional in-training TensorBoard sampling, post-training base-vs-LoRA visual comparison with inference_compare.py, quality/precision configs, and a three-dimension visual evaluation checklist. Only use once training has produced a checkpoint.
|
Step 8 — Validation
Series position: Step 8 of 8 (final step).
Validation answers: did the LoRA learn the intended edit? The approach has three layers of
increasing depth — use as many as needed.
Prerequisites:
- Training complete; at least one checkpoint written (Step 7)
- Test samples set aside (Step 2)
- Conda env active; model paths known
8.1 In-training visual monitoring (TensorBoard sampling)
The framework can generate sample outputs during training at regular intervals and log them to
TensorBoard, giving a visual sense of progress without extra code.
(Source: docs/guide/validation_sampling.md, qwen-image-finetune v3.1.0)
Configure in the training YAML:
validation:
enabled: true
steps: 100
max_samples: 2
seed: 42
dataset:
class_path: "qflux.data.dataset.ImageDataset"
init_args:
dataset_path:
- split: test
repo_id: <your-dataset>
selected_control_indexes: [1]
processor:
class_path: "qflux.data.preprocess.ImageProcessor"
init_args:
process_type: center_crop
target_size: [384, 672]
controls_size: [[384, 672], [512, 512]]
Start TensorBoard (in a separate terminal):
tensorboard --logdir=outputs/<run_name>/logs
Then open the Images tab to compare outputs across checkpoints.
⚠️ 32 GB AI PC constraint: validation.enabled: true is the explicit exception to the normal
Step 5 fit-phase skip_text_encoder requirement, forcing text_encoder to stay in memory during
fit. On 32 GB machines this causes OOM. Options:
- Recommended: keep
validation.enabled: false during training; run §8.2 post-training instead.
- Train on a 64 GB+ machine where the extra ~15 GB fits.
- Step 6 §6.2 NF4 text_encoder reduces text_encoder from ~15 GB to ~5.5 GB, which may make
in-training validation feasible on 32 GB — not tested with
validation.enabled: true.
8.2 Post-training visual comparison (inference_compare.py)
After training completes, run inference twice on a held-out test set — once with the base model and
once with the trained LoRA — then compare the outputs.
Getting a test split
Use whichever source matches how you prepared your test set in Step 2 §2.8:
- Image directory (held-out folder with
training_images/ + control_images/): use --test-dir
- Parquet (directory containing
data/*.parquet, or a direct .parquet file path): use --test-parquet
- CSV (same column format as the training CSV): use
--test-csv
All three sources produce the same output; choose the one that matches your data.
Save the script as inference_compare.py in the project root:
import argparse, csv, io, json, os, re, sys
from pathlib import Path
import PIL.Image
import torch
from qflux.data.config import Config, TrMode, load_config_from_yaml
try:
from qflux.utils.memory_probe import MemoryProbe
except ImportError:
MemoryProbe = None
_IMG_EXTS = (".png", ".jpg", ".jpeg", ".webp", ".bmp")
_IMG_DIRS = ["training_images", "images", "target_images", "target", "targets"]
_CTRL_DIRS = ["control_images", , , ]
_CTRL_EXTRA = re.()
_MASK_PAT = re.()
():
PIL.Image.(path).convert()
():
:
pyarrow.parquet pq
ImportError:
SystemExit()
parquet_path.is_dir():
candidates = (parquet_path.glob())
candidates:
candidates = (parquet_path.glob())
candidates:
SystemExit()
table = pq.read_table(candidates[].as_posix())
:
table = pq.read_table(parquet_path.as_posix())
rows = table.(, n).to_pylist()
samples = []
i, row (rows):
ctrl_imgs = []
entry row.get() []:
(entry, ) entry entry[]:
ctrl_imgs.append(PIL.Image.(io.BytesIO(entry[])).convert())
(entry, (, )):
ctrl_imgs.append(PIL.Image.(io.BytesIO(entry)).convert())
target_entry = row.get()
tgt =
(target_entry, ) target_entry.get():
tgt = PIL.Image.(io.BytesIO(target_entry[])).convert()
samples.append({: row.get(, (i)), : ctrl_imgs,
: tgt, : row.get(, )})
samples
():
images_dir = ((dataset_dir / d d _IMG_DIRS (dataset_dir / d).exists()), )
control_dir = ((dataset_dir / d d _CTRL_DIRS (dataset_dir / d).exists()), )
images_dir control_dir :
SystemExit()
stems = [f.stem f (images_dir.iterdir())
f.suffix.lower() _IMG_EXTS
_MASK_PAT.search(f.name) _CTRL_EXTRA.search(f.name)]
samples = []
stem stems:
(samples) >= n:
target_path = ((images_dir / e _IMG_EXTS
(images_dir / ).exists()), )
main_ctrl = ((control_dir / e _IMG_EXTS
(control_dir / ).exists()), )
target_path main_ctrl :
extra_ctrls, idx = [],
:
ec = ((control_dir / e _IMG_EXTS
(control_dir / ).exists()), )
ec :
extra_ctrls.append(ec); idx +=
prompt_path = (images_dir / ) (images_dir / ).exists() \
(control_dir / ) (control_dir / ).exists()
prompt = prompt_path.read_text(encoding=).strip() prompt_path
ctrl_imgs = [_open_image((main_ctrl))] + [_open_image((p)) p extra_ctrls]
samples.append({: stem, : ctrl_imgs,
: _open_image((target_path)), : prompt})
samples:
SystemExit()
samples
():
(csv_path, encoding=, newline=) f:
rows = (csv.DictReader(f))
rows:
SystemExit()
control_cols = (c c rows[] c.startswith())
control_cols:
SystemExit()
samples = []
i, row (rows[:n]):
ctrl_imgs = [_open_image(row[c]) c control_cols
row.get(c) os.path.exists(row[c])]
ctrl_imgs:
tgt_path = row.get(, )
tgt = _open_image(tgt_path) tgt_path os.path.exists(tgt_path)
samples.append({: row.get(, (i)), : ctrl_imgs,
: tgt, : row.get(, )})
samples:
SystemExit()
samples
():
args.test_parquet:
_read_parquet_samples(Path(args.test_parquet), n)
args.test_dir:
_read_local_dir_samples(Path(args.test_dir), n)
_read_csv_samples(Path(args.test_csv), n)
():
config.trainer_type == :
qflux.trainer.qwen_image_edit_trainer QwenImageEditTrainer
QwenImageEditTrainer(config)
config.trainer_type == :
qflux.trainer.qwen_image_edit_plus_trainer QwenImageEditPlusTrainer
QwenImageEditPlusTrainer(config)
NotImplementedError()
():
p = argparse.ArgumentParser(description=)
p.add_argument(, required=)
src = p.add_mutually_exclusive_group(required=)
src.add_argument(,
=)
src.add_argument(,
=)
src.add_argument(,
=)
p.add_argument(, required=)
p.add_argument(, =, default=)
p.add_argument(, =, default=)
p.add_argument(, default=)
p.add_argument(, =, default=)
p.add_argument(, choices=[, ], default=,
=
)
p.add_argument(, default=,
=
)
args = p.parse_args()
out_dir = Path(args.output_dir)
out_dir.mkdir(parents=, exist_ok=)
config = load_config_from_yaml(args.config)
config.mode = TrMode.fit
config.cache.use_cache =
config.data.init_args.use_cache =
config.validation.enabled =
config.predict.devices.text_encoder = args.text_encoder_device
args.text_encoder_path :
config.model.text_encoder_path = args.text_encoder_path
args.text_encoder_device == :
config.model.text_encoder_path =
(config.model, , ):
SystemExit(
)
args.lora_weight:
config.model.lora.pretrained_weight = args.lora_weight
()
:
config.model.lora.pretrained_weight =
()
samples = _load_test_samples(args, args.num_samples)
()
torch.manual_seed(args.seed)
label = args.lora_weight
():
trainer = _instantiate_trainer(config)
trainer.setup_predict()
manifest = {: label, : args.lora_weight, : args.config,
: args.num_inference_steps, : args.seed, : []}
i, s (samples):
sid = s[]
()
:
ctrl_w, ctrl_h = s[][].size
out_imgs = trainer.predict(image=s[], prompt=s[],
height=ctrl_h, width=ctrl_w,
num_inference_steps=args.num_inference_steps)
out_img = out_imgs[] (out_imgs, ) out_imgs
fname =
out_img.save(out_dir / fname)
s[] :
s[].save(out_dir / )
manifest[].append({: sid, : s[], : fname,
: s[] })
Exception e:
traceback
()
manifest[].append({: sid, : ,
: traceback.format_exc()})
(out_dir / , ) f:
json.dump(manifest, f, indent=)
()
MemoryProbe :
MemoryProbe(label=):
_run()
:
_run()
__name__ == :
main()
Running (from the qwen-image-finetune project root, with conda env active):
REM Activate environment first (same as training)
call "C:\Program Files (x86)\Intel\oneAPI\setvars.bat" --force
call conda activate qwen-image-edit-xpu
set QFLUX_DOTENV_LOADED=1
set PYTHONUTF8=1
REM --- Using an image directory test set (--test-dir) ---
REM Step 1 — base model (no LoRA)
python inference_compare.py ^
--config config.yaml ^
--test-dir <path\to\test-folder> ^
--output-dir outputs\compare\<run>\base ^
--num-samples 5
REM Step 2 — LoRA-applied
python inference_compare.py ^
--config config.yaml ^
--test-dir <path\to\test-folder> ^
--output-dir outputs\compare\<run>\lora ^
--num-samples 5 ^
--lora-weight outputs\<run>\checkpoint-<step>\pytorch_lora_weights.safetensors
REM --- Using a parquet file or directory (--test-parquet) ---
REM (replace --test-dir with --test-parquet <path\to\test.parquet>)
REM --- Using a CSV file (--test-csv) ---
REM (replace --test-dir with --test-csv <path\to\test.csv>)
Each output directory contains sample_NN_<id>.png (model output),
sample_NN_<id>__target.png (ground-truth if available), and manifest.json with prompts and metadata.
Quality and speed trade-offs
Two parameters worth tuning to improve output quality, independently of the LoRA weights themselves:
-
Model precision: higher-precision DiT and text encoder consistently produce better results.
Three configurations, ordered by quality:
| Config | DiT | Text encoder | Required system RAM |
|---|
| A | NF4 on XPU | NF4 on XPU | 32 GB + XPU cap (Step 1 §1.3) |
| B (default) | NF4 on XPU | bf16 on CPU | 32 GB |
| C | bf16 on XPU | bf16 on CPU | ≥64 GB |
inference_compare.py defaults to Config B (text_encoder on CPU) regardless of what the
training YAML specifies. Select a different config:
- B (default): nothing to do — the script forces the text_encoder to CPU.
- A: pass
--text-encoder-device xpu:0 together with --text-encoder-path <dir>, the
pre-quantized NF4 text_encoder from Step 6 §6.2 (or set model.text_encoder_path in the YAML).
The script requires it for Config A and errors out otherwise.
On a 32 GB machine, first increase the iGPU shared-memory percentage (Step 1 §1.3) —
the default is prone to OOM that can crash the whole machine under Config A.
- C (≥64 GB): set
model.quantize_type to a non-NF4 value (or remove it) so the DiT loads in
bf16; keep the text_encoder on CPU (default). The NF4-trained LoRA applies to the bf16 base
unchanged. Note the bf16 DiT is ~38 GB, so it needs the Step 1 §1.3 shared-memory ceiling raised
to fit on the XPU even on a 64 GB machine.
-
Inference steps (--num-inference-steps, default 20): more steps improve quality at the cost
of time. Start with the default and increase only if the output quality is insufficient.
32 GB AI PC — constraints by config:
- Config B (default): system RAM approaches 31–32 GB. On some machines, bf16 text_encoder
(~15 GB) may cause OOM alongside NF4 DiT. If this happens, fall back to Config A by setting
model.text_encoder_path (Step 6 §6.2) — trades some output quality for reliability.
- Config A (both models on XPU): highest XPU demand of the three, rising with resolution. If
you see
UR_RESULT_ERROR_DEVICE_LOST or a silent crash, raise the GPU shared memory ceiling
(Step 1 §1.3) — the default registry value is often insufficient for this configuration.
- Higher inference steps (50+): XPU memory accumulates across the denoising loop; confirm
stability at the default 20 steps before increasing.
Other options:
--num-samples 3 for a quicker comparison with fewer samples.- Monitor with
python monitor_xpu_memory.py (Step 7) from a second terminal.
8.3 Visual evaluation checklist
When inspecting the base vs. LoRA outputs, apply these three questions for each sample. They align
with established image-editing evaluation dimensions: ⁴
| Dimension | Question to ask |
|---|
| Instruction adherence | Does the LoRA output follow the edit prompt? (e.g., for "Add the character to the image" — is the character present in the right place?) |
| Edit quality | Does the edit look natural and seamless? (no obvious artefacts, blurring, or inconsistency at edit boundaries) |
| Detail preservation | Are unedited regions unchanged between base and LoRA outputs? (background, non-target areas should look identical) |
A successful LoRA should pass all three on most samples. Failure patterns:
- No change: base and LoRA outputs are identical → training had no effect; check loss trajectory and training steps.
- Instruction ignored: character not added, wrong edit → LoRA learned something unrelated; check prompt consistency in dataset.
- Degraded quality: artefacts, color shifts, garbled regions → possible overfitting or learning rate too high.
For a small dataset (~30 samples), use smooth loss rather than raw per-step loss to judge
convergence — raw loss is noisy on small datasets due to repeated batches.
⁴ ImgEdit (Ye et al., arXiv:2505.20275, May 2025), https://github.com/PKU-YuanGroup/ImgEdit. Introduces instruction adherence, editing quality, and detail preservation as the three evaluation dimensions for image editing; adopted here as a manual inspection framework. Evaluates image editing models generally, not specifically qwen-image-finetune.
PASS signal
- Both
inference_compare.py runs exit 0
- Output directories contain PNG files and
manifest.json with no errors
- Visual inspection: LoRA outputs visibly differ from base outputs; §8.3 checklist applied to judge
the direction and quality of the change
This is the final step — the fine-tuning walkthrough is complete.
Troubleshooting
TypeError: unsupported operand type for //: 'NoneType' and 'int': height/width not passed
to trainer.predict(), so prepare_embeddings receives batch["height"] = None. The bundled
inference_compare.py already applies the fix (derive from the first control image's PIL size). If
you see this in a custom script, apply the same pattern:
ctrl_w, ctrl_h = s["control_images"][0].size
out_imgs = trainer.predict(..., height=ctrl_h, width=ctrl_w, ...)
ValueError: offline mode — must specify weight_name: loading a LoRA checkpoint when
HF_HUB_OFFLINE=1 is set, because diffusers.load_lora_adapter() calls _best_guess_weight_name()
which raises in offline mode even for local paths. Fix (already applied in
qflux/trainer/base_trainer.py): if pretrained_weight ends in .safetensors, extract the parent
directory and filename and pass weight_name explicitly to load_lora_adapter(). If your qflux
lacks this fix, either upgrade, or pass the checkpoint directory (not the full file path) with
the filename pytorch_lora_weights.safetensors (PEFT default).
UR_RESULT_ERROR_DEVICE_LOST during inference: XPU out-of-memory. Use Config B (text_encoder on
CPU), reduce --num-samples, or raise the registry GPU memory ceiling (Step 1 §1.3) and reboot.
Inference hangs in the VAE (no error, no progress): do not debug by adding
torch.xpu.synchronize(), self.vae.to("cpu"), or extra .to(...) moves inside decode_vae_latent
— that may introduce new hangs and does not fix the cause. Instead, overwrite decode_vae_latent with
the original upstream implementation from qflux/trainer/qwen_image_edit_trainer.py (inherited
unchanged by QwenImageEditPlusTrainer) and re-run.
