check-models

# Check Model Integrations End-To-End Run a consolidated model-integration audit that combines loader checks, detect/routing checks, reference-catalog checks, and pipeline contract checks. ## When To Use - A new model family was added and needs a completeness audit - Existing model support appears inconsistent across detection, loading, and UI references - A custom pipeline was ported and needs contract validation - You want a pre-PR integration quality gate for model-related changes ## Guidance - Consult `.github/instructions/core.instructions.md` for relevant core runtime and model integration guidance before proceeding. ## Combined Scope This skill combines four audit surfaces. Run them in this order unless user scope limits categories: 1. Loader consistency (`check-loaders` equivalent) 2. Detect/routing parity (`check-detect-routing` equivalent) 3. Reference-catalog integrity (`check-reference-catalog` equivalent) 4. Pipeline API contract conformance (`check-pipeline-contracts` equivalent) ## Primary Files - `pipelines/model_*.py` - `modules/sd_detect.py` - `modules/sd_models.py` - `modules/modeldata.py` - `data/reference-base.json` - `data/reference-cloud.json` - `data/reference-quantized.json` - `data/reference-distilled.json` - `data/reference-nunchaku.json` - `data/reference-community.json` - `models/Reference/` Pipeline files as needed: - `pipelines/<model>/pipeline.py` - `pipelines/<model>/model.py` ## Audit A: Loader Consistency For each target model loader in `pipelines/model_*.py`, verify: - Correct `sd_models.path_to_repo(checkpoint_info)` and `sd_models.hf_auth_check(...)` usage - Load args built with `model_quant.get_dit_args(...)` where applicable - No duplicated kwargs (for example duplicate `torch_dtype`) - Correct component loading path (`generic.load_transformer`, `generic.load_text_encoder`, tokenizer/processor) - Proper post-load hooks (`sd_hijack_te`, `sd_hijack_vae`) where required - Correct `pipe.task_args` defaults where needed - Cleanup and `devices.torch_gc(...)` present Flag stale patterns, missing hooks, or conflicting load behavior. ## Audit B: Detect/Routing Parity Verify model family alignment across: - `modules/sd_detect.py` detection heuristics - `modules/sd_models.py` load dispatch branch - `modules/modeldata.py` reverse classification from loaded pipeline class Checks: - Family is detectable by name/repo conventions - Dispatch routes to the intended loader - Loaded pipeline class is classified back to the same model family - Branch ordering does not cause broad matches to shadow specific families ## Audit C: Reference Catalog Integrity Verify references for model families intended to appear in model references. Checks: - Correct category file placement by type: - base -> `data/reference-base.json` - cloud -> `data/reference-cloud.json` - quant -> `data/reference-quantized.json` - distilled -> `data/reference-distilled.json` - nunchaku -> `data/reference-nunchaku.json` - community -> `data/reference-community.json` - Required fields present per entry (`path`, `preview`, `desc` when expected) - Duplicate repo/path collisions across reference files are intentional or flagged - Preview filename convention is consistent - Referenced preview file exists in `models/Reference/` (or explicitly placeholder if intentional) - JSON validity for touched reference files ## Audit D: Pipeline API Contracts For custom pipelines (`pipelines/<model>/pipeline.py`), verify: - Inherits from `diffusers.DiffusionPipeline` - Registers modules correctly - `from_pretrained` wiring is coherent with actual artifact layout - `encode_prompt` semantics are consistent with tokenizer/text encoder setup - `__call__` supports expected public args for its task and does not expose unsupported generic args - Batch and negative prompt behavior are coherent - Output conversion aligns with model output domain (latent vs pixel space) - `output_type` and `return_dict` behavior are consistent ## Runtime Validation (Preferred) When feasible: - Import-level smoke tests for loaders and pipeline modules - Lightweight loader construction checks without full heavy generation where possible - One minimal generation/sampling pass for changed model families If runtime checks are not feasible, report limitations clearly. ## Reporting Format Return findings by severity: 1. Blocking integration failures 2. Contract mismatches (load/detect/reference/pipeline) 3. Consistency and quality issues 4. Optional improvements For each finding include: - model family - layer (`loader`, `detect-routing`, `reference`, `pipeline-contract`) - file location - mismatch summary - minimal fix Also include summary counts: - loaders checked - model families checked for detect/routing parity - reference files checked - pipeline contracts checked - runtime checks executed vs skipped ## Pass Criteria A full pass requires all of the following in audited scope: - loader path is coherent and non-conflicting - detect/routing/modeldata parity holds - reference entries are valid, categorized correctly, and have preview files - custom pipeline contracts are consistent with actual model behavior If any area is intentionally out of scope, mark as partial pass with explicit exclusions.