std-reviewer

name	std-reviewer
description	Semantic QE review of STD YAML and test stubs against STP traceability, pattern correctness, and PSE quality
model	claude-opus-4-6
version	1.1.0

STD Reviewer Skill

Phase: Post-Generation Review User-Invocable: false

Purpose

Perform a comprehensive semantic QE review of a generated STD (YAML + test stubs). This skill evaluates traceability to the source STP, pattern matching correctness, test step quality, PSE docstring quality, and code generation readiness.

When to Use

Invoked by the review-std command after an STD has been generated by /std-builder.

Two-Layer Review Architecture

This skill uses a two-layer review architecture:

• Layer 1 (General): Shared pipeline rules that apply to any project. These are the same quality rules the generator follows, embedded in this skill file and always active.
• Layer 2 (Project-specific): Domain-specific pattern mappings, helper libraries, and code conventions loaded from {project_context.config_dir}/review_rules.yaml at runtime.

Config Loading

If the review-std command has loaded a review_rules.yaml file and passed it as context, use its contents to enhance checks with project-specific knowledge:

• std_rules.patterns.keyword_to_pattern → keyword-to-pattern mapping table for Dimension 3a
• std_rules.patterns.pattern_to_helpers → pattern-to-helper-library mapping for Dimension 3b
• std_rules.patterns.sig_to_decorator → SIG-to-decorator mapping for Dimension 3c
• std_rules.patterns.closure_scope_required → required closure scope variables for Dimension 2c
• std_rules.patterns.test_id_format → expected test ID format for Dimension 2b
• std_rules.patterns.ginkgo_structure → expected Ginkgo structure for Dimension 6c
• std_rules.timeouts → expected timeout ranges by operation type for Dimension 6d
• std_rules.stub_conventions → stub file conventions (pending markers, package rules) for Dimension 5

Graceful Degradation

If no review_rules.yaml exists for the project, all dimensions still apply using built-in general rules. Project-specific config adds precision (exact pattern mappings, helper library tables) but is not required. The general rules check structural correctness, traceability, and quality regardless of project-specific details.

Input

std_yaml_path: "outputs/std/{JIRA_ID}/{JIRA_ID}_test_description.yaml"
stp_file_path: "outputs/stp/{JIRA_ID}/{JIRA_ID}_test_plan.md"
go_stubs_dir: "outputs/std/{JIRA_ID}/go-tests/"       # may not exist
python_stubs_dir: "outputs/std/{JIRA_ID}/python-tests/" # may not exist
project_context: <from project-resolver, includes repo_rules>
review_rules: <from review_rules.yaml, if available>

repo_rules Integration

When project_context.repo_rules is available, the reviewer validates stubs against the target repository's standards:

From repo_rules.agents_rules (AGENTS.md):

• Verify tier2 marker is NOT explicitly added (it is implicit)
• Verify team markers (network, storage, etc.) are NOT explicitly added
• Verify pytest.skip/skipif is NOT used anywhere
• Verify fixture names are nouns, not verbs
• Verify __test__ = False placement follows the rules (class-level for grouped, after function for standalone)
• Verify conftest.py contains only fixtures (no helpers)
• Verify module docstring contains STP link
• Verify resources are named by function ("client pod"), not generic labels ("pod-A")
• Verify @pytest.mark.incremental is used for dependent tests, not pytest-dependency

From repo_rules.std_format (SOFTWARE_TEST_DESCRIPTION.md):

• Verify PSE docstrings use exact section names: Preconditions:, Steps:, Expected:
• Verify [NEGATIVE] indicator is used for failure scenarios
• Verify assertion wording follows the patterns in the document
• Verify shared preconditions are in class/module docstring, test-specific in test docstring
• Verify each test verifies ONE thing with ONE Expected: assertion
• Verify no fixture names appear in Preconditions (describe in natural language)

Severity: MAJOR for violations of repo_rules. These represent the team's own standards and failing to follow them will cause PR review friction.

Output

A structured review report written to:

outputs/reviews/{JIRA_ID}/{JIRA_ID}_std_review.md

---

Review Dimensions

The review evaluates 7 dimensions. Each dimension produces findings classified as:

Severity	Meaning	Impact on Verdict
CRITICAL	Missing traceability, invalid YAML, or error that would produce broken tests	Blocks approval
MAJOR	Quality issue, wrong pattern, or gap that should be addressed	Does not block but flags for attention
MINOR	Improvement suggestion or stylistic issue	Informational only

---

Dimension 1: STP-STD Traceability

Purpose: Verify complete bidirectional traceability between the STP and STD.

What to check:

1a. Forward Traceability (STP → STD)

Parse Section III of the STP (Requirements-to-Tests Mapping table). For each row:

• Extract: Requirement ID, Requirement Summary, Test Scenario(s), Tier, Priority
• Search the STD YAML scenarios array for a matching scenario

Matching algorithm (deterministic, applied in order):

1. Requirement ID exact match (required): requirement_id in STD must exactly match a Requirement ID in the STP Section III. If no exact match, the scenario is an orphan.

2. Scenario text similarity (required for multi-scenario requirements): When one requirement maps to multiple STP scenarios, match each STD scenario to a specific STP scenario using keyword overlap:

   • Extract keywords from both texts (nouns, verbs, excluding stop words)
   • Calculate overlap: shared_keywords / min(keywords_A, keywords_B)
   • Threshold: overlap >= 0.50 (at least 50% keyword overlap) counts as a match
   • If multiple STP scenarios match above threshold, pick the highest overlap

3. Tier match: tier must match (Tier 1 ↔ "Tier 1", Tier 2 ↔ "Tier 2"). Mismatch is a separate MAJOR finding, not a traceability failure.

4. Priority match: priority must match (P0/P1/P2). Mismatch is a separate MAJOR finding, not a traceability failure.

Match result classification:

• Full match: Requirement ID matches AND keyword overlap >= 0.50 → traced
• Weak match: Requirement ID matches but keyword overlap < 0.50 → MAJOR finding ("STD scenario {id} maps to requirement {req_id} but scenario text diverges significantly from STP — verify correct mapping")
• No match: Requirement ID not found in STP → CRITICAL finding (orphan)

Red flags:

• CRITICAL: STP scenario has no corresponding STD scenario (coverage gap)
• MAJOR: Tier mismatch between STP and STD for same scenario
• MAJOR: Priority mismatch between STP and STD for same scenario

1b. Reverse Traceability (STD → STP)

For each scenario in the STD YAML:

• Verify that requirement_id exists in the STP Section III table
• Verify the scenario description relates to the STP row

Red flags:

• CRITICAL: STD scenario has no corresponding STP row (orphan scenario)
• MAJOR: STD scenario's requirement_id not found in STP

1c. Count Consistency

• document_metadata.total_scenarios matches actual count of scenarios in array
• document_metadata.tier_1_count matches count of scenarios with tier: "Tier 1"
• document_metadata.tier_2_count matches count of scenarios with tier: "Tier 2"
• document_metadata.p0_count matches count of scenarios with priority: "P0"

Red flags:

• CRITICAL: Count mismatch between metadata and actual scenarios

1d. STP Reference

• document_metadata.stp_reference.file points to the actual STP file
• File path is valid and matches the expected pattern

Red flags:

• MAJOR: STP reference file path is wrong or file does not exist

1e. Priority-Testability Consistency

For each scenario, verify that the priority assignment is consistent with testability:

• A P0 (highest priority) scenario must be fully testable. If the scenario's test_objective or test_steps indicate it "cannot be verified at this stage" or "requires infrastructure not yet available", this is a logical contradiction.

Red flags:

• CRITICAL: P0 scenario that is explicitly documented as untestable or deferred — a P0 item cannot be simultaneously highest-priority and untestable. Either downgrade the priority, resolve the testability blocker, or defer to a follow-up STD.

---

Dimension 2: STD YAML Structure (v2.1-enhanced)

Purpose: Verify the STD YAML conforms to v2.1-enhanced specification.

What to check:

2a. Document-Level Structure

• document_metadata section exists with all required fields
• document_metadata.std_version is "2.1-enhanced"
• code_generation_config section exists (v2.1 requirement)
• code_generation_config.std_version is "2.1-enhanced"
• code_generation_config.package_name is inferred from owning_sig
• common_preconditions section exists
• scenarios array exists and is non-empty

2b. Per-Scenario Required Fields

For each scenario in the scenarios array, verify presence of:

Field	Required	Notes
scenario_id	YES	Sequential number
test_id	YES	Format per project config or default TS-{JIRA_ID}-{NUM:03d}
tier	YES	"Tier 1" or "Tier 2"
priority	YES	"P0", "P1", or "P2"
requirement_id	YES	Jira issue key
patterns	YES	Primary pattern + helpers
variables	YES	v2.1: closure_scope array
test_structure	YES	v2.1: describe/context/it
code_structure	YES	Ginkgo structure hint
test_objective	YES	title, what, why, acceptance_criteria
test_data	YES	resource_definitions and/or api_endpoints
test_steps	YES	setup, test_execution, cleanup arrays
assertions	YES	At least 1 assertion per scenario

If project config is loaded, use std_rules.patterns.test_id_format for the expected test ID format. Otherwise, use the default TS-{JIRA_ID}-{NUM:03d}.

Red flags:

• CRITICAL: Missing required field in any scenario
• CRITICAL: test_id does not follow the expected format
• MAJOR: Duplicate scenario_id or test_id values
• MAJOR: tier value not "Tier 1" or "Tier 2"

2c. v2.1-Specific Checks

Universal checks (all tiers):

• All variables.closure_scope includes required variables appropriate for the tier
• All scenarios with setup steps have corresponding cleanup steps

If project config is loaded, use std_rules.patterns.closure_scope_required for the required closure scope variables.

Tier 1 (Go/Ginkgo) checks — apply ONLY to scenarios with tier: "Tier 1":

• test_structure.context.decorators includes: Ordered
• Code templates use = (not :=) for closure variables
• Expect(err) calls use ExpectWithOffset(1, err)
• variables.closure_scope includes ctx and namespace (or per project config)

Tier 2 (Python/pytest) checks — apply ONLY to scenarios with tier: "Tier 2":

• No Ginkgo-specific constructs (Ordered, BeforeAll, ExpectWithOffset) in Tier 2 scenarios
• @pytest.mark.incremental used for dependent tests (not pytest-dependency)
• Fixture names are nouns, not verbs (per repo_rules.agents_rules if available)
• No pytest.skip or pytest.skipif used (per repo_rules.agents_rules if available)

Red flags:

• MAJOR: Missing required variables in closure_scope
• MAJOR: Missing Ordered decorator on a Tier 1 scenario
• MAJOR: Ginkgo-specific constructs found in a Tier 2 scenario (framework mismatch)
• MAJOR: pytest-dependency used instead of @pytest.mark.incremental in Tier 2
• MINOR: := used for closure variable in Tier 1 (should be =)

---

Dimension 3: Pattern Matching Correctness

Purpose: Verify that the auto-generated pattern metadata is correct for each scenario.

What to check:

3a. Primary Pattern Matching

For each scenario, read the test_objective.title and test_steps, then verify the primary pattern assignment.

If project config is loaded, use std_rules.patterns.keyword_to_pattern for the keyword-to-pattern mapping table. Otherwise, apply general heuristics: the primary pattern should match the dominant action/domain of the test scenario based on keywords in the title and steps.

Red flags:

• MAJOR: Primary pattern does not match scenario keywords
• MAJOR: No primary pattern assigned to a scenario
• MINOR: Primary pattern is too generic when a more specific one exists

3b. Helper Library Mapping

Verify that patterns.helpers_required includes the correct libraries for the matched patterns.

If project config is loaded, use std_rules.patterns.pattern_to_helpers for the pattern-to-helper mapping table. Otherwise, verify that helper libraries referenced in the scenario are consistent with the primary pattern and the test steps described.

Red flags:

• MAJOR: Missing required helper library for matched pattern
• MINOR: Extra helper library that is not needed

3c. Decorator Assignment

Verify decorators are correct.

If project config is loaded, use std_rules.patterns.sig_to_decorator for the SIG-to-decorator mapping. Otherwise, verify that:

• Tier decorators match the scenario's tier assignment
• SIG/domain decorators are consistent with the scenario's subject area
• Ordering decorators are present where tests have dependencies

General decorator checks (always apply):

Condition	Expected
Tier 1 scenario	Tier 1 decorator
Tier 2 scenario	Tier 2 decorator
All ordered tests	Ordered decorator

Red flags:

• MAJOR: Wrong tier decorator (Tier1 on a Tier 2 scenario)
• MAJOR: Missing SIG/domain decorator
• MINOR: Missing Ordered decorator (it should always be present)

3d. Pattern Library Validation

If the pattern library exists at {project_context.config_dir}/patterns/tier1_patterns.yaml, verify that all pattern_id values in test_steps reference actual patterns in the library.

Red flags:

• MAJOR: pattern_id references a pattern not in the library
• MINOR: Code template in test step differs significantly from pattern library template

---

Dimension 4: Test Step Quality

Purpose: Evaluate the quality of setup → execution → cleanup steps.

What to check:

4a. Step Completeness

For each scenario:

• test_steps.setup has at least 1 step
• test_steps.test_execution has at least 1 step
• test_steps.cleanup has at least 1 step (resource cleanup)

Red flags:

• CRITICAL: No test_execution steps (nothing to test)
• MAJOR: No cleanup steps (resource leak)
• MINOR: No setup steps (may be intentional if using common_preconditions)

4b. Step Quality

For each step, evaluate:

• action is specific and actionable (not "Do the test")
• command or code reference is provided where applicable
• validation describes the expected outcome
• Step IDs are sequential (SETUP-01, SETUP-02, TEST-01, etc.)

Red flags:

• MAJOR: Vague action ("Verify it works", "Check the result", "Observe behavior")
• MAJOR: Missing validation for a test_execution step
• MAJOR: Uncertain verification language ("may be set", "might appear", "should probably") — verification must be definitive
• MINOR: Non-sequential step IDs

4b.2. Abstraction Level in Test Steps

Test steps and assertions should use user-observable language, not internal component references.

Red flags (MAJOR):

• Internal component names in test steps or assertions (e.g., "controller removes", "handler unmounts", "reconciler updates")
• Internal API object references where user-facing language would suffice
• Implementation-level verbs (reconcile, sync, propagate) in step descriptions

Acceptable:

• User-observable descriptions: "volume is automatically removed from the running instance"
• API-level descriptions when the API is the user interface: "resource status shows Ready"

4c. Logical Flow

Verify the step sequence makes sense:

1. Setup creates resources before they are used in execution
2. Test execution uses resources created in setup
3. Cleanup removes resources created in setup
4. No circular dependencies

Red flags:

• MAJOR: Test step references a resource not created in setup
• MINOR: Cleanup does not reference all resources from setup

4c.2. STP Customer Use Case Alignment

When the STP is available, cross-reference each scenario's test setup against the STP's customer use case descriptions. Test setup should reflect realistic user workflows, not arbitrary technical sequences.

Red flags (MAJOR):

• Test creates resources in a multi-step sequence when the STP describes a single-shot user action.
• Test setup implies a workflow that no real user would follow. If the STP defines the user story as a single operation, the test should reflect that single operation.
• Unnecessary test dependencies between independent scenarios — each scenario should be independently verifiable unless the scenarios explicitly model a sequential workflow (e.g., upgrade before/after). If test B depends on test A but tests A and B verify independent features, flag the dependency.

Red flags (MINOR):

• Test setup uses intermediate verification steps that belong in preconditions (e.g., "Verify no connectivity" as a setup step when it should be a stated precondition)

4d. Upgrade Test Structure

If the STD contains upgrade-related scenarios (Tier 2 scenarios with "upgrade" in title or test_objective), verify they follow the before/after pattern:

Required structure for upgrade scenarios:

1. Before upgrade: Capture state / verify feature works pre-upgrade
2. Perform upgrade: Trigger the upgrade operation
3. After upgrade: Verify same state/feature works post-upgrade

Red flags:

• MAJOR: Upgrade scenario missing "before" verification — if you only check after, you don't know if a regression occurred
• MAJOR: Upgrade scenario missing "after" verification — upgrade test without post-upgrade check is incomplete
• MINOR: Upgrade scenario not placed in the correct test module/directory structure (upgrade tests typically belong in a dedicated upgrade module)

4e. Test Dependency Structure

If multiple scenarios within the STD have explicit or implicit ordering dependencies (e.g., scenario B uses resources created by scenario A), verify the dependency is justified.

Acceptable dependencies:

• Upgrade test sequences (before-upgrade → upgrade → after-upgrade)
• Ordered test classes with early-failure markers (test B depends on test A's resource)
• Sequential lifecycle scenarios (create → modify → verify → delete)

Red flags (MAJOR):

• Scenario B depends on scenario A, but they test independent features — each scenario should be independently verifiable. Unnecessary dependencies make test suites fragile.
• Dependency chain exists but no early-failure marker or ordered decorator is specified — if tests are dependent, the framework must cascade failures correctly.
• Test class creates unnecessary resource sharing between unrelated scenarios just to avoid setup cost — prefer test independence over execution speed.

Red flags (MINOR):

• Dependency exists and is justified, but the dependency direction is not documented in the STD YAML (e.g., no depends_on or ordering note)

4f. Assertion Quality

For each assertion:

• description is specific (not "Verify it works")
• condition is measurable (not "Should be correct")
• priority is assigned (P0 or P1)

Red flags:

• MAJOR: Generic assertion description
• MAJOR: No assertions in a scenario
• MINOR: All assertions are P0 (unrealistic — some should be P1)

---

Dimension 4.5: STD Content Policy

Purpose: Verify that STD artifacts (YAML and stubs) contain only design-level content and do not include implementation details, invalid references, or content that belongs in other pipeline phases.

What to check:

4.5a. Banned Content in STD YAML and Stub Files

Check both the STD YAML document_metadata and stub file docstrings for content that does not belong in the STD phase:

STD YAML red flags (MAJOR):

• related_prs or any PR URL list in document_metadata — PR URLs are implementation artifacts that belong in the STP (which references them in Section I), not in the STD. The STD describes what to test, not what code changed.
• Branch names, commit SHAs, or code review links in metadata

Stub file red flags (MAJOR):

• PR URLs or references (e.g., PR #16412, github.com/.../pull/...) in docstrings — STD is a design document, not tied to specific PRs
• Branch names or commit references
• Developer names or assignees (except QE Owner which is acceptable as "TBD")

Red flags (MINOR):

• References to documents that do not exist or are not part of the project structure
• Hardcoded file paths that assume a specific local directory structure

4.5b. No Implementation Details in Stubs

Stub files should describe what to test, not how to implement it. Stubs are design artifacts for review, not implementation code.

Red flags (MAJOR):

• Fixture implementations (actual fixture code with yield/return) in stub files — fixtures belong in the implementation phase
• Helper function implementations in stub files
• Import of project-internal modules that only make sense at implementation time
• Concrete API calls or SDK usage in stub file bodies (stubs should have pending marker bodies only)

Acceptable in stubs:

• PSE docstrings describing preconditions, steps, and expected results
• Pending marker bodies per project conventions. If project config is loaded, use std_rules.stub_conventions for the expected pending markers (e.g., PendingIt() for Go, pass for Python).
• Standard library imports for type annotations
• Class/function declarations with descriptive names

4.5c. Test Environment Separation

STD stubs should NOT include test environment setup details. Tests assume all infrastructure is already in place.

Red flags (MAJOR):

• Infrastructure device creation or configuration in stubs
• Cluster node setup or labeling logic
• Feature gate enablement code (belongs in test environment, not test design)
• Network/storage infrastructure provisioning

Red flags (MINOR):

• Comments describing environment requirements that belong in the STP's Test Environment section (II.3), not in test code

---

Dimension 5: PSE Docstring Quality (Stub Files)

Purpose: Evaluate the Preconditions/Steps/Expected docstrings in generated test stubs.

What to check:

5a. Go Stubs (if present)

Read each *_stubs_test.go file in the go-stubs directory.

For each pending test block:

• PSE comment block is present
• Preconditions: Specific, not generic. References concrete resources.
  • GOOD: "Running instance with network interface on test network"
  • BAD: "Resource exists"
• Steps: Numbered, actionable, unambiguous.
  • GOOD: "1. Patch resource spec to change network reference"
  • BAD: "1. Change the network"
• Expected: Measurable outcomes.
  • GOOD: "Instance connects to new network; connectivity check succeeds"
  • BAD: "It works"

Additional checks:

• Each pending test block contains a test_id in the expected format
• Module-level comment references STP file (not PR URLs)
• File compiles conceptually (proper test framework structure)

5b. Python Stubs (if present)

Read each test_*_stubs.py file in the python-stubs directory.

For each test function:

• PSE docstring is present in the function body
• Same quality criteria as Go stubs (specific preconditions, numbered steps, measurable expected)
• Test collection is disabled at module level (per project stub conventions)
• Function body contains only the pending marker

5c. PSE Section Classification Strictness

Apply strict PSE section classification rules. Misclassified items cause confusion during implementation and human review.

Preconditions must describe what must be true BEFORE the test begins:

• Resource existence ("Running instance with secondary interface on network1")
• State conditions ("No connectivity to peer instance on network2")
• Configuration ("Feature gate enabled")
• Data recorded at class/setup level ("MAC address recorded", "baseline IP captured")

Steps must describe ACTIONS the test performs:

• API calls ("Patch resource spec to change network reference")
• User operations ("Attach a new interface")
• State changes ("Wait for update to complete")
• Steps should NOT include "Verify..." — verification belongs in Expected

Expected must describe OBSERVABLE OUTCOMES to verify:

• Must include HOW to verify, not just WHAT to verify
  • GOOD: "Connectivity check from instance to peer on network2 succeeds with 0% packet loss"
  • BAD: "Instance is connected to network2" (missing verification method)
• Must be measurable, not qualitative
  • GOOD: "Resource status shows Ready within expected timeout"
  • BAD: "Resource works correctly"

Classification red flags:

• MAJOR: "Verify..." step in the Steps section (should be in Expected)
• MAJOR: Baseline verification in Steps ("Verify no connectivity before change") — this is a precondition that confirms initial state
• MAJOR: Expected result without verification method ("Instance is connected" without specifying connectivity check/API check/status inspection)
• MINOR: Precondition listed as a Step ("Ensure instance is running" — this is a precondition, not a test action)

Red flags:

• CRITICAL: Missing PSE docstring in a test stub
• MAJOR: Generic/vague preconditions, steps, or expected results
• MAJOR: Missing test_id in test name or docstring
• MAJOR: Steps/Expected/Preconditions misclassification (see 5c rules above)
• MAJOR: PSE docstring not standalone-readable — a reader unfamiliar with the STP should be able to understand what the test does from the PSE alone. If the docstring uses abbreviations, domain terms, or references that only make sense with STP context, add a brief inline explanation. Example:
  • BAD: "Steps: 1. Perform measurement X" (what is X? not self-explanatory)
  • GOOD: "Steps: 1. Measure network connectivity downtime during rolling update"
• MINOR: PSE docstring references internal mechanisms instead of user actions

5d. Stub Completeness for Integration Areas

Verify that stubs cover all integration areas defined in the STD scenarios. Common integration areas that require dedicated stubs:

Check for missing stubs:

• If STD has upgrade scenarios, verify upgrade-related stub(s) exist
• If STD has migration scenarios, verify migration-related stub(s) exist
• If STD has snapshot/restore scenarios, verify snapshot-related stub(s) exist

Red flags:

• MAJOR: STD has upgrade scenarios but no upgrade stub file exists
• MAJOR: STD has migration scenarios but no migration stub file exists
• MINOR: Integration area stubs exist but are missing stubs for some scenarios in that area

---

Dimension 6: Code Generation Readiness

Purpose: Evaluate whether the STD YAML will produce valid, compilable test code when passed to the test generation commands.

What to check:

6a. Variable Declarations

For each scenario's variables.closure_scope:

• Variable names are valid identifiers for the target language
• Types are valid for the target language
• initialized_in references a valid lifecycle hook
• used_in references valid lifecycle hooks

Red flags:

• MAJOR: Invalid type in variable declaration
• MAJOR: Variable initialized in a hook that runs after its usage hook (wrong order)
• MINOR: Variable declared but never referenced in code_structure

6b. Import Completeness

Check code_generation_config.imports against the helpers used across all scenarios:

• If any scenario uses a helper library, the import must be present
• Cross-reference patterns.helpers_required against available imports

Red flags:

• MAJOR: Helper library used in scenarios but not in imports
• MINOR: Import listed but no scenario uses it

6c. Code Structure Validity

For each scenario's code_structure:

• Valid test framework structure. If project config is loaded, use std_rules.patterns.ginkgo_structure for the expected structure pattern.
• Proper bracket matching
• test_id placeholder uses correct format
• No syntax errors in the template

Red flags:

• MAJOR: Malformed test framework structure
• MINOR: Missing test_id in test block description

6d. Timeout Appropriateness

Check timeout references in test steps:

• Operations that take time use appropriate timeout constants
• Quick operations don't use oversized timeouts

If project config is loaded, use std_rules.timeouts for expected timeout ranges by operation type. Otherwise, apply general heuristics: long-running operations (resource creation, migration) should have larger timeouts than quick operations (API calls, status checks).

Red flags:

• MINOR: Oversized timeout for a simple operation
• MINOR: No timeout specified for a long-running operation

---

Review Report Format

Generate the review report as markdown:

# STD Review Report: {JIRA_ID}

**Reviewed:**
- STD YAML: {STD_YAML_PATH}
- STP Source: {STP_FILE_PATH}
- Go Stubs: {GO_STUBS_DIR or "N/A"}
- Python Stubs: {PYTHON_STUBS_DIR or "N/A"}

**Date:** {YYYY-MM-DD}
**Reviewer:** QualityFlow Automated Review (v1.1.0)
**Review Rules Schema:** {review_rules._extraction_metadata.schema_version or "N/A"}

---

## Verdict: {APPROVED | APPROVED_WITH_FINDINGS | NEEDS_REVISION}

## Summary

| Metric | Value |
|:-------|:------|
| Dimensions reviewed | {X}/7 |
| Critical findings | {count} |
| Major findings | {count} |
| Minor findings | {count} |
| Confidence | {HIGH / MEDIUM / LOW} |

## Traceability Summary

| Metric | Value |
|:-------|:------|
| STP scenarios | {count} |
| STD scenarios | {count} |
| Forward coverage (STP→STD) | {X}/{Y} ({percent}%) |
| Reverse coverage (STD→STP) | {X}/{Y} ({percent}%) |
| Orphan STD scenarios | {count} |
| Missing STD scenarios | {count} |

---

## Findings by Dimension

### Dimension 1: STP-STD Traceability

{traceability matrix and findings}

### Dimension 2: STD YAML Structure

{structural findings}

### Dimension 3: Pattern Matching Correctness

| Scenario | Primary Pattern | Helpers | Decorators | Status |
|:---------|:----------------|:--------|:-----------|:-------|
| {scenario_id} | {pattern} | {count} | {count} | {PASS/WARN/FAIL} |

{per-scenario findings}

### Dimension 4: Test Step Quality

| Scenario | Setup | Execution | Cleanup | Assertions | Status |
|:---------|:------|:----------|:--------|:-----------|:-------|
| {scenario_id} | {count} | {count} | {count} | {count} | {PASS/WARN/FAIL} |

{per-scenario findings}

### Dimension 4.5: STD Content Policy

{content policy findings: banned references, implementation details, environment separation}

### Dimension 5: PSE Docstring Quality

**Go Stubs:**
{findings per stub file}

**Python Stubs:**
{findings per stub file}

### Dimension 6: Code Generation Readiness

{readiness assessment and findings}

---

## Recommendations

{numbered list of actionable recommendations, ordered by severity}

1. **[CRITICAL]** {recommendation}
2. **[MAJOR]** {recommendation}
3. **[MINOR]** {recommendation}

---

## Confidence Notes

| Factor | Status |
|:-------|:-------|
| STD YAML parseable | {YES/NO} |
| STP file available | {YES/NO} |
| Go stubs present | {YES/NO} |
| Python stubs present | {YES/NO} |
| Pattern library available | {YES/NO} |
| All scenarios reviewed | {YES/NO} |
| Project review rules loaded | {YES/NO} |

**Confidence rationale:** {explanation of why confidence is HIGH/MEDIUM/LOW}

---

Verdict Criteria

Verdict	Criteria
APPROVED	0 critical findings, 0 major findings
APPROVED_WITH_FINDINGS	0 critical findings, 1+ major or minor findings
NEEDS_REVISION	1+ critical findings

---

Confidence Scoring

Level	Criteria
HIGH	STD YAML valid, STP available, stub files present, pattern library available, all 7 dimensions reviewed, review rules default_ratio <= 0.30
MEDIUM	STD YAML valid and STP available, but stubs or pattern library missing, OR review rules default_ratio <= 0.60
LOW	STD YAML valid but STP unavailable (review based on STD alone), OR review rules default_ratio > 0.60

When review_rules._extraction_metadata.default_ratio > 0.50, include a note in Confidence Notes: "Review precision reduced: {X}% of rules using generic defaults. Consider adding project-specific review_rules.yaml or enabling repo_files_fetch."

---

Error Handling

• If STD YAML not found: Return error, do not produce review report
• If STD YAML is invalid YAML: Report CRITICAL finding, attempt partial review of what can be parsed
• If STP file not found: Skip Dimension 1 (traceability). Set confidence to LOW. Note in report.
• If stub files not found: Skip Dimension 5 (PSE quality). Note in report.
• If pattern library not found: Skip Dimension 3d (pattern library validation). Note in report.
• If project_context unavailable: Skip pattern library and import checks. Note in report.
• If review_rules.yaml not found: Continue with general rules only. All dimensions still apply; project-specific precision is reduced. Note in Confidence Notes.

---

End of STD Reviewer Skill