test-fixture-sync

name	test-fixture-sync
description	Sync test fixtures in test/data/* by comparing the input and output files present with test fixtures in __init__.py and making appropriate updates, followed by updating the relevant tests to use the current fixtures/files. Use when test data file organization changes, output files are added/removed, or tests fail due to fixture mismatches.

Sync Test Fixtures

Follow this workflow to keep test fixtures aligned with the files on disk.

Scan Files

• List each test dataset under test/data (e.g., kob, mlamd, mnt, t).
• For each dataset, list test/data/<dataset>/input contents.
  • .srt files are fixture candidates.
  • .sup files should be noticed during scanning, but only MLAMD exposes SUP path fixtures. Do not add SUP fixtures for other datasets.
• For each dataset, list the contents of other directories under test/data/<dataset> recursively.
  • These contain data related to "actions" taken on inputs, such as reviewing.
  • Example directories of interest include lang/.
  • .json files are of interest.
• For each dataset, list test/data/<dataset>/output contents.
  • .srt and _image directories are of interest.
• Compare against fixtures and __all__ in test/data/<dataset>/__init__.py.

Suggested commands (PowerShell):

Get-ChildItem -Path test/data/<dataset>/input | Sort-Object Name | Select-Object Name
Get-ChildItem -Path test/data/<dataset>/output | Sort-Object Name | Select-Object Name
Get-ChildItem -Path test/data/<dataset> -Recurse -Filter *.json | Sort-Object FullName | Select-Object FullName
rg -n "output_dir /" test/data/<dataset>/__init__.py

Suggested commands (bash):

ls -1 test/data/<dataset>/input | sort
ls -1 test/data/<dataset>/output | sort
find test/data/<dataset> -name "*.json" | sort
rg -n "output_dir /" test/data/<dataset>/__init__.py

Update Fixtures

• Add/remove fixtures in test/data/<dataset>/__init__.py to match output files/dirs.
• Add input fixtures for .srt files. Do not add input fixtures for .sup files, except for the existing MLAMD *_sup_path fixtures.
• Keep fixture names consistent with filenames (match the naming convention used in the repo).
• Within test/data/<dataset>/__init__.py, group fixtures into:
  • Input fixtures (pointing to files in input/)
  • Action fixtures (pointing to files in other directories such as lang/)
  • Output fixtures (pointing to files in output/)
• Within each group, sort fixtures alphabetically.
• Update __all__ to match fixture names, maintaining the same grouping and order as above.

Update Tests

• Use rg to find fixture usages in test/ and update to the new fixture names.
• Prefer replacing old fixture names with the most semantically equivalent new ones.

Suggested commands:

rg -n "<old_fixture_name>" test
rg -n "fuse_review|clean_validate_review|review_flatten" test

Validate

• Format/check only the Python files you touched:

uv run ruff format <files>
uv run ruff check --fix <files>

• Run pytest:

cd test && uv run pytest

Notes

• Keep changes minimal and consistent with existing naming patterns.
• Do not add fixtures for files that are not tracked by git.
• Ensure files/directories of interest (all extensions and formats, including _image) are actually under version control before adding fixtures for them.
• Keep the grouping/order in __all__ aligned with fixture grouping (input/action/output).