scoring-pipeline

Estrelas0

Forks0

Atualizado1 de abril de 2026 às 03:21

Python scoring pipeline architecture — models, workers, engine, and how to extend the system.

Instalação

Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.

Fonte

synthet

synthet/image-scoring-gallery

Abrir repositório GitHub Ver repositórios do creator

Download

Ocupações relacionadasSOC

Baseado na classificação ocupacional SOC

Desenvolvedores de softwareInformática e Matemática·SOC 15-1252

SKILL.md

readonly

name	scoring-pipeline
description	Python scoring pipeline architecture — models, workers, engine, and how to extend the system.

Scoring Pipeline

The image-scoring project uses a multi-stage pipeline to assess image quality using neural network models. This skill covers the architecture, key modules, and how to safely make changes.

Architecture Overview

Input (folder/file)
  → PrepWorker (RAW conversion, thumbnails, skip checks)
    → ScoringWorker (GPU inference: MUSIQ, LIQE, TOPIQ)
      → ResultWorker (DB upsert, XMP metadata write, cleanup)

The pipeline uses producer-consumer queues with threading.Thread workers.

Key Modules

Module	Purpose
`modules/pipeline.py`	Worker classes: `PrepWorker`, `ScoringWorker`, `ResultWorker`, `ImageJob` dataclass
`modules/scoring.py`	`ScoringRunner` — orchestrates batch/single scoring, Fix DB, and metadata repair
`modules/engine.py`	`BatchImageProcessor` — low-level batch processing engine
`modules/config.py`	`load_config()`, `get_config_value()` — reads `config.json`
`modules/utils.py`	Path conversion (WSL ↔ Windows), hashing, thumbnail utilities

Model Wrappers

File	Model	Framework	Score Range
`musiq/run_all_musiq_models.py`	MUSIQ (SPAQ, AVA, KonIQ, PaQ2PiQ)	TensorFlow	0–100 (normalized to 0–1)
`modules/liqe.py`	LIQE (language-image quality evaluator)	PyTorch (pyiqa)	1–5 (normalized to 0–1)
`modules/topiq.py`	TOPIQ-IAA	PyTorch (pyiqa)	0–1
`modules/qalign.py`	Q-Align	PyTorch	1–5

Scoring Formulas (Current)

General  = 0.50 × LIQE + 0.30 × AVA + 0.20 × SPAQ
Technical = LIQE (primary)
Aesthetic = 0.60 × AVA + 0.40 × SPAQ

All composite scores are stored as floats 0.0–1.0 in the database.

Hybrid Environment

Windows: Runs the WebUI, PostgreSQL Docker, and file management.
WSL 2 (Linux): Runs GPU inference (TensorFlow + PyTorch with CUDA).
CRITICAL: Database access from WSL must use TCP (port 5432). PostgreSQL runs in Docker on the Windows host.

How to Add a New Model

Create a wrapper in modules/ (e.g., modules/newmodel.py) that exposes a score(image_path) → float method.
Import and call it in ScoringWorker.process() inside modules/pipeline.py.
Add the raw score column to the DB schema in modules/db.py → _init_db_impl().
Update the composite formulas in modules/scoring.py → ScoringRunner.fix_image_metadata().
Add normalization logic if the model's native range isn't 0–1.

Configuration

All config lives in config.json at the project root. Key sections:

database — filename, user, password
scoring — force_rescore_default, model weights

Mais deste repositório

mesmo repositório

changelog-commit-push

synthet/image-scoring-gallery

Update CHANGELOG.md and package.json via compiled release_bump harness, then commit/push only when the user asks. Use for /release, changelog updates, or shipping gallery versions.

2026-07-280

gallery-electron-ts

synthet/image-scoring-gallery

Works in image-scoring-gallery on electron/db.ts contract, IPC, apiService, React/Vite, and alignment with sibling image-scoring-backend schema. Use for desktop app bugs, database.engine modes, API URL/port from webui.lock, or TypeScript changes under electron/ or src/. Triggers include Electron gallery, db.ts, IPC, apiService, config.api, PostgreSQL vs api engine, and Vite renderer.

2026-07-280

gallery-ui

synthet/image-scoring-gallery

React component patterns, CSS Modules styling, design tokens, virtualization, and UX constitution for the Driftara Gallery renderer. Use for src/ UI work, styling, FilterPanel/GalleryGrid/ImageViewer, or design:check—not electron/db unless coordinated with gallery-electron-ts.

2026-07-280

validate-implementation

synthet/image-scoring-gallery

Verify implementation against spec AC-n criteria via compiled harness. Use after /implement or /test-and-fix, before /pr-ready. Parses ACs and emits the report; LLM assigns Verified/Failed/Unknown when evidence is not a clean command exit.

2026-07-280

verification-before-completion

synthet/image-scoring-gallery

Use before claiming work is complete, fixed, passing, ready to commit, or ready for PR. Runs scripts/agent_skills/verification_before_completion.py for the claim→proof catalog; LLM interprets output. Never upgrade incomplete verification.

2026-07-280

image-scoring-mcp

synthet/image-scoring-gallery

Driftara Gallery MCP — is-ui-* router-first; backend triage via sibling is-be-mcp search+dispatch.

2026-07-250

Scoring Pipeline

The image-scoring project uses a multi-stage pipeline to assess image quality using neural network models. This skill covers the architecture, key modules, and how to safely make changes.

Architecture Overview

Input (folder/file) → PrepWorker (RAW conversion, thumbnails, skip checks) → ScoringWorker (GPU inference: MUSIQ, LIQE, TOPIQ) → ResultWorker (DB upsert, XMP metadata write, cleanup)

The pipeline uses producer-consumer queues with threading.Thread workers.

Key Modules

Module

Purpose

modules/pipeline.py

Worker classes: PrepWorker, ScoringWorker, ResultWorker, ImageJob dataclass

modules/scoring.py

ScoringRunner — orchestrates batch/single scoring, Fix DB, and metadata repair

modules/engine.py

BatchImageProcessor — low-level batch processing engine

modules/config.py

load_config(), get_config_value() — reads config.json

modules/utils.py

Path conversion (WSL ↔ Windows), hashing, thumbnail utilities

Model Wrappers

File

Model

Framework

Score Range

musiq/run_all_musiq_models.py

MUSIQ (SPAQ, AVA, KonIQ, PaQ2PiQ)

TensorFlow

0–100 (normalized to 0–1)

modules/liqe.py

LIQE (language-image quality evaluator)

PyTorch (pyiqa)

1–5 (normalized to 0–1)

modules/topiq.py

TOPIQ-IAA

PyTorch (pyiqa)

0–1

modules/qalign.py

Q-Align

PyTorch

1–5

Scoring Formulas (Current)

General = 0.50 × LIQE + 0.30 × AVA + 0.20 × SPAQ Technical = LIQE (primary) Aesthetic = 0.60 × AVA + 0.40 × SPAQ

All composite scores are stored as floats 0.0–1.0 in the database.

Hybrid Environment

Windows: Runs the WebUI, PostgreSQL Docker, and file management.

WSL 2 (Linux): Runs GPU inference (TensorFlow + PyTorch with CUDA).

CRITICAL: Database access from WSL must use TCP (port 5432). PostgreSQL runs in Docker on the Windows host.

How to Add a New Model

Create a wrapper in modules/ (e.g., modules/newmodel.py) that exposes a score(image_path) → float method.

Import and call it in ScoringWorker.process() inside modules/pipeline.py.

Add the raw score column to the DB schema in modules/db.py → _init_db_impl().

Update the composite formulas in modules/scoring.py → ScoringRunner.fix_image_metadata().

Add normalization logic if the model's native range isn't 0–1.

Configuration

All config lives in config.json at the project root. Key sections:

database — filename, user, password

scoring — force_rescore_default, model weights