| name | paper-coverage-validator |
| description | Validates whether benchmark artifacts cover the paper's required RGB micro and RGB DataLoader sections. Use when checking missing RGB runs, deciding what to run next, validating gcp_runs/output folders, or preparing paper tables. |
Paper Coverage Validator
Use tools.check_paper_coverage to avoid manual folder inspection.
Command
python -m tools.check_paper_coverage gcp_runs output
Validate the RAM-only reduced production-path pass:
python -m tools.check_paper_coverage --profile ram-reduced gcp_runs output
Require optional libraries such as DALI:
python -m tools.check_paper_coverage gcp_runs output --require-optional-libraries
Required Paper Core Sections
image-rgb/micro: AlbumentationsX, torchvision, Kornia, Pillow; summary JSON.image-rgb/pipeline: AlbumentationsX, torchvision, Kornia, Pillow.
9-channel and video benchmarks are still supported by the codebase, docs, and transform sets, but they are not required
for the current RGB paper artifact.
RAM-Reduced Profile
--profile ram-reduced intentionally checks only:
- RGB micro.
- RGB
memory_dataloader_augment.
It does not require decode DataLoader, 9-channel, or video runs.
What To Do With Failures
- Missing directory: run the scenario/mode.
- Missing summary JSON: rerun or fetch
results/ from the GCS run prefix.
- Missing raw pyperf JSON for a micro run: acceptable for sanitized published snapshots, but recover it from run archives
before publishing profiler-level raw timing data.
