spec-traceability

name	spec-traceability
description	Parses specification documents (EARS, Given-When-Then, SHALL clauses, user stories, tasks) from multiple layouts into a structured intermediate representation, then maps requirements to tests bidirectionally. Use when verifying implementation against a spec, auditing requirement coverage, or computing a traceability matrix.

Spec Traceability

This skill teaches an agent how to:

1. Parse specification markdown (in any of the three supported layouts) into a structured intermediate representation (IR)
2. Map requirements ↔ tests bidirectionally using three escalating strategies
3. Produce a traceability matrix the user can read at a glance

The skill is layout-agnostic — it works with spec-gen, Kiro, plain markdown, and one-requirement-per-file layouts. It does not depend on any other plugin.

When to use

• The user runs /qa:verify-spec
• The test-strategist or test-generator agent needs structured requirement records (not raw prose)
• The qa-auditor agent needs to validate "every requirement in the spec is exercised by at least one test"
• Any time the orchestrating command has resolved a spec and needs to know "what's in it?"

Inputs

The skill operates on whatever discovery output is available. The orchestrator typically passes:

• spec entry from scripts/discover-specs.sh — has format, files, path
• raw markdown content of requirements.md, design.md, tasks.md (or their flat-file equivalents)
• test inventory — list of test file paths, optionally with parsed describe/it names

Step 1: Pick the right parser by spec format

Use the format field from the spec entry:

format	Strategy
spec-gen	Parse requirements.md, design.md, tasks.md as separate files. Use H2/H3 headings for sections.
standard	Same as spec-gen (same file layout, no spec.config.json)
kiro	Same as spec-gen (Kiro uses identical filenames)
flat	Single .md file. Parse H1 sections (# Requirements, # Design, # Tasks); fall back to H2 if H1 is the title.
per-file	One requirement per file under requirements/REQ-NNN.md. Each file is one requirement record. Tasks similarly.
minimal	Best-effort: treat every H2/H3 in any present markdown as a candidate requirement.

If the format is unrecognized, treat it as minimal.

Step 2: Extract requirements into the IR

For each requirements source, extract one record per requirement. The IR shape:

{
  "id": "FR-1.1",
  "type": "functional|nfr|user_story|acceptance",
  "title": "GET /api/file responds with content",
  "text": "When client requests GET /api/file?path=<path>, the server SHALL respond with ...",
  "acceptance": [
    "Given a 100KB .md file in allowedRoots.current, GET returns 200 with content + valid mtime + etag.",
    "Given a path outside roots → 403 outside-allowlist."
  ],
  "source": "specs/markdown-editor/requirements.md#fr-1-1",
  "tags": ["security", "api"],
  "must_have": true
}

2.1 Functional Requirements (FR-N, FR-N.M)

Pattern: bold heading or H3 heading starting with FR-N or **FR-N.M**:.

### FR-1: GET /api/file (read)

**FR-1.1**: When client requests GET /api/file?path=<path>, the server SHALL respond with ...

**FR-1.2**: Response SHALL include header Cache-Control: no-store.

Extract:

• id = exact identifier (FR-1, FR-1.1, FR-1.2)
• title = text after the colon (for headings only)
• text = full requirement body (the SHALL clause)
• acceptance = bullets under the nearest #### Acceptance — FR-N sub-section (see 2.4)

2.2 Non-Functional Requirements (NFR-X-Y)

Same shape, prefixed NFR-. Common subscripts: NFR-S- (security), NFR-P- (performance), NFR-A- (accessibility).

2.3 User Stories (US-N)

Pattern: **US-N**: As a X, I want Y, so that Z.

Extract id, then split text into role / want / so-that for downstream display.

2.4 Acceptance Criteria

Two common shapes:

1. Per-FR sub-section: #### Acceptance — FR-N followed by bullets
2. Inline Given/When/Then bullets directly under the FR

Pattern for the bullets:

- Given a 100KB .md file ..., GET returns 200 with content + valid mtime + etag.
- Given a path outside roots → 403 outside-allowlist.

Each bullet becomes one element in the acceptance array of the corresponding requirement record.

2.5 EARS clauses (when present)

EARS-style sentences observed in real specs:

EARS pattern	Example
WHEN <trigger>, THE SYSTEM SHALL <response>	WHEN file size > 1MB, THE SYSTEM SHALL return 413
IF <condition> THEN THE SYSTEM SHALL <response>	IF path is symlink THEN THE SYSTEM SHALL reject
the <component> SHALL <response>	the server SHALL respond with FileReadResponse
WHILE <state>, THE SYSTEM SHALL <response>	WHILE editor is dirty, THE SYSTEM SHALL warn on close

If the requirement text matches any of these patterns, set type: "functional". Otherwise default to type: "functional" if it starts with FR-, user_story if it starts with US-, nfr if it starts with NFR-.

2.6 Permissive degradation

If none of the patterns above match, degrade gracefully: treat every H2/H3 heading under a section called "Requirements", "Functional Requirements", "User Stories", or similar as one requirement, using the heading text as both id (slugified) and title. Each bullet under the heading becomes an acceptance criterion.

Never refuse to parse. Empty IR is acceptable; the verify command will report "0 requirements parsed" and the user can fix the spec.

Step 3: Extract tasks

For tasks.md (or the # Tasks section of a flat spec):

Pattern: ### Task N.M: Title or - [ ] Task N.M: Title.

{
  "id": "T1.1",
  "title": "Add FileErrorCode + read/write schemas to @harness-visualizer/shared",
  "type": "Shared",
  "priority": "High",
  "effort": "30 min",
  "dependencies": [],
  "status": "completed|in-progress|pending",
  "files_touched": []
}

Status sources (in priority order):

1. Checkbox state — - [x] = completed, - [ ] = pending
2. Heading prefix — ### ✅ Task N.M, ### 🔄 Task N.M
3. progress.md if present — parse a status table
4. Default: unknown

Step 4: Map requirements → tests

Three strategies, applied in order. Use the first that yields a non-empty result for each requirement.

Strategy A: Explicit reference in test name (strongest)

A test description that mentions FR-N.M, NFR-X-Y, US-N, or T1.1 is an explicit binding.

it('FR-1.5: rejects disallowed extensions', () => { ... });
describe('FR-2.5 atomic write', () => { ... });

Mapping confidence: explicit.

Strategy B: Generated mapping (when QA-generated tests)

When the test-generator agent created the tests, it must emit a risks_covered array per test file:

{
  "file": "backend/src/api/file.test.ts",
  "test_count": 8,
  "risks_covered": ["FR-1.1", "FR-1.5", "FR-1.6"]
}

Use this mapping verbatim. Mapping confidence: generated.

Strategy C: LLM post-hoc mapping

For human-written tests with no explicit refs, the agent reads:

• The full requirements IR
• The test source code (or at minimum: file path + all describe/it titles)

…and produces a mapping {requirement_id: [test_file:test_name, ...]}. Mapping confidence: inferred.

When reasoning, prefer specificity:

• A test named it('returns 413 for oversize writes') clearly maps to FR-1.6 (or the 1MB cap requirement) even without the explicit ref.
• A test in the file path backend/src/api/file.test.ts is more likely to map to FR-1 (GET/POST /api/file) than to UI requirements.

Mark every C-mapping as inferred so the user can audit.

Step 5: Build the traceability matrix

For each requirement record, attach:

• covered_by — array of {file, test_name, confidence} objects
• status — covered_passing | covered_failing | covered_unknown | uncovered
• notes — optional caveats (e.g., "only inferred mapping; please verify")

Aggregate:

{
  "spec": "markdown-editor",
  "coverage": {
    "total_requirements": 52,
    "covered": 47,
    "uncovered": 5,
    "passing": 45,
    "failing": 2,
    "coverage_pct": 90.4
  },
  "by_status": {
    "covered_passing": ["FR-1.1", "FR-1.2", "..."],
    "covered_failing": ["FR-2.4"],
    "uncovered": ["FR-1.6", "NFR-S-X3", "..."]
  },
  "requirements": [
    {
      "id": "FR-1.1",
      "title": "GET /api/file responds with content",
      "covered_by": [
        {"file": "backend/src/api/file.test.ts", "test_name": "returns 200 with content", "confidence": "inferred"}
      ],
      "status": "covered_passing"
    }
  ]
}

Step 6: Render the human-readable report

The verify command renders this Markdown summary using the IR + matrix:

# Spec Verification: markdown-editor

**Coverage**: 47/52 requirements (90.4%) — 2 failing, 3 uncovered
**Confidence**: 38 explicit, 9 generated, 5 inferred

## Uncovered Requirements
| ID | Title | Acceptance Criteria | Suggested Test |
| -- | ----- | ------------------- | -------------- |
| FR-1.6 | 413 when > 1MB | Given a 2MB .md → 413 too-large | `backend/src/api/file.test.ts` — failing-on-purpose test |
| NFR-S-X3 | Parent-dir-swap defense | (deferred per spec) | Acknowledged residual risk |

## Failing Tests Mapped to Requirements
| ID | Test | Why it's failing |
| -- | ---- | ---------------- |
| FR-2.4 | `backend/src/api/file.test.ts > conflict detection` | `expected 409 'conflict', got 200 'ok'` |

## Full Matrix
| Requirement | Tests | Status |
| ----------- | ----- | ------ |
| FR-1.1 GET /api/file responds with content | 3 tests | ✓ pass |
| ... | ... | ... |

Output contract

When invoked by the orchestrator, return TWO artifacts:

1. The structured matrix JSON — saved to .state/verification/<date>-<spec>.json
2. The Markdown report — saved to .state/verification/<date>-<spec>.md AND printed to the user

If the user passed --json, emit only (1) to stdout.

Caveats and known limits

• EARS variation in the wild is high. The parser is forgiving — if a SHALL clause is split across multiple lines, joining them by removing intra-paragraph newlines is acceptable.
• Tests in non-Vitest frameworks require a different output parser. The skill assumes Vitest by default; the test-strategist agent can override if Jest/Mocha is detected.
• Tasks → tests mapping is not v1. Tasks are extracted into the IR but not mapped to tests in v1; that's a v2 feature once requirement-mapping is dogfooded.
• Deferred requirements (marked "deferred" or "out of scope") should be parsed but flagged must_have: false and excluded from the "uncovered" count unless --include-deferred.