validate-and-fix

name	validate-and-fix
description	Run the full validation pipeline (make test-unit, make verify, make test-integration) and auto-fix trivial failures like black formatting, unused imports, and ruff-fixable lint errors. Use when the user asks to validate, run tests, check the pipeline, or verify changes are clean.
disable-model-invocation	true

Validate & Auto-Fix

Run the project validation pipeline, auto-fix trivial issues, and re-run until green or a real failure is found.

Rules

• Never modify production logic to fix a test. Only fix test expectations, imports, formatting, and lint.
• Never skip or delete a failing test.
• Stop after 3 auto-fix cycles to avoid loops.
• Report real failures clearly; do not attempt speculative fixes.

Step 0: Ensure Dependencies Are Installed

make install-deps 2>&1 | tail -10

This ensures the virtualenv matches the lockfile. Skipping this step is the most common cause of spurious import errors (e.g. llama-index-embeddings-huggingface not found despite being in requirements.txt).

Step 1: Run Unit Tests

make test-unit 2>&1 | tail -40

If all pass, proceed to Step 3 (integration tests). If failures occur, classify each failure (see Step 2).

Step 2: Classify and Fix Failures

For each failure, determine its type:

Auto-fixable (fix immediately, then re-run Step 1):

Type	Fix
Missing await on async call	Add await, mark test async def, add @pytest.mark.asyncio
Test asserts old default value	Update assertion to match new default

Real failures (do not auto-fix):

• Logic errors in production code
• Assertion failures reflecting actual behavior changes
• Import errors from missing dependencies
• Failures in code you did not modify

For real failures: report the test name, file, and error summary, then stop.

Step 3: Run Integration Tests

make test-integration 2>&1 | tail -20

Integration test failures are almost always real failures. Report and stop.

Step 4: Run Verify

make verify 2>&1 | tail -30

This runs: black --check, ruff, woke, pylint, mypy.

If failures occur, apply the same classify-and-fix logic from Step 2. Common fixes at this stage:

Type	Fix
Black formatting	uv run black .
Unused import (ruff F401)	Remove the import
Unused variable (ruff F841)	Remove the assignment
Unused noqa directive (RUF100)	Remove the noqa comment
Other ruff-fixable	uv run ruff check . --fix
Pylint issues from changed code	Fix inline

Re-run make verify after each fix. Proceed to Step 5 when green.

Step 5: Report

Report exactly:

• make test-unit: X passed / Y failed
• make verify: pass/fail (list any remaining issues)
• make test-integration: X passed / Y failed
• Auto-fixes applied (list each: file, what was fixed)
• Cycles used: N/3

Do not include unrelated diagnostics or suggestions.