| name | output-validation |
| version | 1.1.1 |
| description | Validates interval-instruction JSON (`{start}->{end}` keys to label lists) and per-frame CSR mask NPZ against video height, width, and frame count, without ground-truth labels. Use for a structural gate after mask generation and before any evaluator. Not for Lighthouse metrics, LLM answer grading, or generic JSON Schema of unrelated APIs. |
| risk | safe |
| source | openrouter-deepsearch |
| date_added | 2026-06-16T00:00:00.000Z |
When to Use
Run this skill after you have generated your outputs (interval instructions JSON, per-frame CSR mask NPZ, etc.) and before you submit or hand them off to any downstream consumer — training pipelines, evaluators, visualizers, or scoring steps.
Trigger keywords: validate outputs, check masks, verify instructions, CSR validation, NPZ check, output self-check, ground-truth-free validation, interval instructions, mask format check.
The artifacts produced by this task follow a strict contract:
- A JSON file of frame-interval keys (
"{start}->{end}") mapped to label lists.
- An NPZ file of per-frame sparse (CSR) masks that must line up with the source video dimensions and frame count.
Downstream consumers assume that contract holds and will fail in confusing, hard-to-trace ways if it does not — an IndexError deep inside a data loader, a silently mis-aligned mask, an evaluator that cannot map an unknown label. A structural self-check catches those problems at the cheapest possible moment.
Every check here is ground-truth-free: it verifies format, range, and internal consistency, never correctness against labels. You can run it long before any scoring step exists, and on data you are not permitted to compare against held-out ground truth.
Prerequisites
- Python 3.10+ with
opencv-python and numpy installed.
- Pin known-good versions of
opencv-python and numpy so CSR and metadata semantics stay stable.
- Import OpenCV as
import cv2. Never use import cv2.cv2 — that is an internal submodule, not a supported public entry point, and relying on it breaks under several wheels.
- Windows host is primary (PowerShell). All commands below work in PowerShell; use forward slashes in
Path() objects inside Python for cross-platform safety.
Procedure
Step 1 — Configure the validator script
Create a file named validate_outputs.py with the configuration block below. Edit the three paths and the ALLOWED_LABELS set to match your project.
"""Ground-truth-free structural validation of interval instructions and CSR masks."""
from __future__ import annotations
import json
import re
from pathlib import Path
from typing import Final
import cv2
import numpy as np
import numpy.typing as npt
VIDEO_PATH: Final[Path] = Path("video.mp4")
INSTRUCTIONS_PATH: Final[Path] = Path("interval_instructions.json")
MASKS_PATH: Final[Path] = Path("masks.npz")
ALLOWED_LABELS: Final[frozenset[str]] = frozenset(
{"person", "vehicle", "animal", "background"}
)
KEY_PATTERN: Final[re.Pattern[str]] = re.compile(r"^(\d+)->(\d+)$")
class ValidationError(Exception):
"""Raised when an output artifact violates the expected contract."""
() -> [, , ]:
path.is_file():
ValidationError()
capture: cv2.VideoCapture = cv2.VideoCapture((path))
:
capture.isOpened():
ValidationError()
frame_count: = (capture.get(cv2.CAP_PROP_FRAME_COUNT))
height: = (capture.get(cv2.CAP_PROP_FRAME_HEIGHT))
width: = (capture.get(cv2.CAP_PROP_FRAME_WIDTH))
:
capture.release()
frame_count <= :
ValidationError(
)
height <= width <= :
ValidationError(
)
frame_count, height, width
() -> [, ]:
path.is_file():
ValidationError()
:
path.(, encoding=) handle:
parsed: = json.load(handle)
json.JSONDecodeError exc:
ValidationError() exc
(parsed, ):
ValidationError(
)
{(key): value key, value parsed.items()}
() -> :
instructions:
ValidationError()
max_index: = -
key, value instructions.items():
: re.Match[] | = KEY_PATTERN.(key)
:
ValidationError(
)
start: = (.group())
end: = (.group())
start > end:
ValidationError(
)
end >= frame_count:
ValidationError(
)
(value, ) (value) == :
ValidationError(
)
label value:
(label, ) label.strip():
ValidationError(
)
label ALLOWED_LABELS:
ValidationError(
)
max_index = (max_index, end)
max_index
() -> :
consecutive: =
masks.files:
consecutive +=
total_data_arrays: = (
name masks.files
name.startswith() name.endswith()
)
total_data_arrays != consecutive:
ValidationError(
)
consecutive
() -> :
prefix: =
component (, , ):
name: =
name masks.files:
ValidationError()
data: npt.NDArray[np.generic] = masks[]
indices: npt.NDArray[np.integer] = masks[]
indptr: npt.NDArray[np.integer] = masks[]
data.ndim != indices.ndim != indptr.ndim != :
ValidationError()
indptr.shape[] != height + :
ValidationError(
)
(indptr[]) != :
ValidationError(
)
(indptr[-]) != indices.size:
ValidationError(
)
data.size != indices.size:
ValidationError(
)
(np.(np.diff(indptr) < )):
ValidationError(
)
indices.size ((indices.()) < (indices.()) >= width):
ValidationError(
)
() -> :
masks.files:
ValidationError()
shape: npt.NDArray[np.integer] = masks[]
shape.shape != (,):
ValidationError(
)
stored_h: = (shape[])
stored_w: = (shape[])
stored_h != height stored_w != width:
ValidationError(
)
frame_count: = _count_frames(masks)
frame_count == :
ValidationError()
index (frame_count):
_validate_csr_frame(masks, index, height, width)
frame_count
() -> :
:
frame_count, height, width = load_video_dims(VIDEO_PATH)
instructions = load_instructions(INSTRUCTIONS_PATH)
max_index = validate_instructions(instructions, frame_count)
MASKS_PATH.is_file():
ValidationError()
np.load(MASKS_PATH) masks:
mask_frames = validate_masks(masks, height, width)
ValidationError exc:
()
(
)
__name__ == :
SystemExit(main())
Step 2 — Run the validator
python validate_outputs.py
On success it exits 0 and prints an [OK] summary. On the first contract violation it exits 1 with a single [FAIL] line naming the specific problem.
Step 3 — Spot-check individual artifacts (optional)
Validate JSON syntax (pretty-prints, exits non-zero on a parse error):
python -m json.tool interval_instructions.json
List the arrays stored in the NPZ (note the explicit close() to release the file handle):
python -c "import numpy as np; f = np.load('masks.npz'); print(f.files); f.close()"
Read the total frame count straight from the video container (release the capture afterward):
python -c "import cv2; cap = cv2.VideoCapture('video.mp4'); print(int(cap.get(cv2.CAP_PROP_FRAME_COUNT))); cap.release()"
What each rule checks and why
| Rule | What it validates | Downstream failure it prevents |
|---|
| Key format | Every key is "{start}->{end}", integers only, start <= end | Consumers split on -> and parse each side as int; a non-integer raises during parsing, and a reversed range silently selects an empty or backwards interval. |
| Coverage | Maximum referenced frame index <= total - 1, consistent with sampling policy | An index at or past the frame count makes the consumer read past the end of the video (IndexError, or worse, a wrong frame from a wrapped/clamped read). |
| Frame count | Number of f_{i}_* groups equals the number of sampled frames, no gaps, no missing CSR components | A gap (e.g. f_0, f_1, then f_3) means a mask is missing and every later frame is mis-indexed relative to the instructions. |
| CSR integrity | Each frame has data, indices, indptr with len(indptr) == H + 1, indptr[0] == 0, indptr[-1] == indices.size, data.size == indices.size, non-decreasing indptr, column indices in [0, W) | A violation either crashes scipy.sparse.csr_matrix reconstruction or, more dangerously, reconstructs a corrupt mask without error. |
| Value validity | JSON values are non-empty lists of label strings, every label in the allowed set | An empty list carries no supervision signal; an out-of-vocabulary label is almost always a typo or generation bug that an evaluator cannot map back to a class. |
Cross-consistency note: Interval keys index the original video frames, while NPZ frames are indexed in sampled order. These live in two different index spaces. Do not assert that "max interval index" equals "mask frame count" — relate them only through your explicit sampling map. The validator keeps the two checks separate for exactly this reason.
Pitfalls
-
Very long or high-resolution videos. Reading the video frame count is cheap (container metadata), but materializing every frame's CSR arrays — or worse, expanding them to dense H×W masks — scales with total stored pixels and can exhaust memory. The validator accesses NPZ members lazily and validates frame-by-frame so peak memory stays bounded. Do not change it to load everything up front.
-
Unreliable container metadata. CAP_PROP_FRAME_COUNT is sometimes an estimate (or 0) for variable-frame-rate, streaming, or partially-written containers. Every range check depends on an accurate count, so the validator treats a non-positive count as a fatal error rather than silently trusting it. If you hit this, re-encode to a constant-frame-rate container before validating.
-
Sensitive footage. Validation reads only frame metadata (count, height, width), never pixel content — but the artifacts you are validating (masks and labels) can still fall under data-protection or privacy rules. Keep them in approved storage and do not copy them to ad-hoc locations to run this check.
-
Wrong OpenCV import. Import as import cv2. The cv2.cv2 form that earlier versions of this skill recommended is an internal submodule, not a supported public entry point, and relying on it breaks under several wheels. Pin known-good versions of opencv-python and numpy.
-
Index space confusion. Interval keys reference original video frames; NPZ f_{i} keys reference sampled frames. Never cross-check them directly without your sampling map.
Verification
Confirm each item before hand-off:
Fastest path — run the full validator:
python validate_outputs.py
Expected success output:
[OK] 12 intervals (max referenced frame 240), 120 mask frames, video 1920x1080 @ 300 frames.
Expected failure output (exits 1):
[FAIL] Key '250->300' references frame 300, but the video only has 300 frames (valid indices 0..299).
Related skills
This validator is the final structural gate in the artifact pipeline. Run it immediately after the upstream steps that produce these files — the step that emits interval_instructions.json and the step that extracts the per-frame CSR masks into masks.npz — and before any ground-truth scoring or evaluation step. Because every check here is ground-truth-free, it can (and should) run earlier than evaluation, so a malformed artifact is rejected before it consumes expensive evaluation compute. If you maintain an explicit frame-sampling map, validate that map alongside this step, since it is the only correct bridge between the instruction (original-frame) and mask (sampled-frame) index spaces.
