odh-test-gen

Analyze test cases and recommend placement (component repo vs downstream E2E repo). Use for determining where each test should be implemented based on test level, dependencies, and repository capabilities.

Generate executable test automation code from test case specifications with intelligent placement in component or downstream repos. Use after test cases are reviewed to create production-ready pytest code that follows repository conventions.

Generate individual test case files from an existing test plan. Use after test plan approval to generate individual TC specifications with preconditions, steps, and expected results organized by category and priority.

Generate one complete test file with all functions for assigned test cases, including quality scoring and auto-revision

Publish test plan artifacts to GitHub — creates a branch, commits all artifacts, and opens a PR with optional reviewer assignment. Use after test plan review to make artifacts available for team collaboration and formal review feedback.

Cross-reference existing test plan gaps with new analyzer findings and documentation to determine which gaps are resolved and which remain open. Use after adding new documentation to identify what questions have been answered and what's still missing.

Browser-based UI test execution against live ODH/RHOAI clusters. Loads TCs from a GitHub PR or repo folder via ui_prepare.py, executes each via a persistent Playwright browser, and produces a visual HTML report with PASS/FAIL/BLOCKED/INCOMPLETE verdicts and screenshots. Use for verifying UI test cases against live clusters with visual evidence and screenshot capture.

Update an existing test plan with new documentation (ADR, API specs, design docs). Re-analyzes, updates artifacts, bumps version, and optionally regenerates test cases. Use when requirements evolve or new technical documentation becomes available after initial test plan creation.

Assess PR review comments on a published test plan, let the user decide what to apply, make changes, and push updates to the same branch. Use when receiving PR review feedback to efficiently apply approved changes with human control over what gets updated.

Generate a test plan from a strategy (RHAISTRAT or RHOAIENG issue), with optional ADR for extra technical depth. Use when starting test planning for a new RHOAI feature with a defined Jira strategy.

Analyzes strategy and ADR to extract feature scope, test objectives, and API endpoints/methods under test. Use for extracting technical scope and API surface area from requirements documents.

Reviews a generated test plan for completeness, consistency, and quality using a 5-criteria rubric. Scores, auto-revises, and re-scores (max 2 cycles). Use for automated quality assessment and iterative improvement of generated test plans.

Score an existing test plan using the quality rubric without triggering auto-revision. Use for standalone quality assessment of test plans or evaluating test plans created outside the automated generation pipeline.