spec-to-tests

name	spec-to-tests
description	Authors failing tests and reproducers from Build Brief acceptance criteria plus verification_spec before coding starts.
contract_version	1.0.0
side_effect_profile	mutating

Why This Exists

Spec -> tests is the locking stage. The Build Brief names the contract; failing tests and reproducers turn that contract into a harness that blocks guesswork and anchors code to correctness. Coding should begin only after the verifier fails for the documented reason.

Trigger

phase: 3
event: context_assembly_pending
requires:
  - applicability_manifest
  - verification_spec
  - structured acceptance_criteria

This skill runs when gen_tests receives assembled Build Brief context for a task and must author the native failing verifier artifacts that the coder will inherit.

Inputs

Use exactly these inputs:

• task_classification
• verification_spec (primary_verifier, secondary_verifiers, optional target_files, optional expected_failure_mode)
• per-task acceptance_criteria
• files_to_create/modify
• reference_impl path
• optional module_plan; required when the task creates or reshapes modules unless module-plan-check generated one from a registered paved-road pattern
• optional loop_contract_path when the task changes an LLM-driven loop, action admission, test selection, control events, no-progress detection, escalation, or loop maturity
• repo test conventions discovered at runtime

If any acceptance criterion arrives as an unstructured string with no normalized id, given, when, and then, do not invent structure. Emit stuck.

Outputs

• Test files written under the repo's native test root and naming conventions
• When module_plan.applicability=required, the architecture test named in module_plan.architecture_test.test_path is authored before production code and before behavior tests that depend on the new layout.
• When module_plan_source=pattern, the architecture test must also preserve the matched pattern's pure/impure split and exemplar structure. If an explicit plan departs from the pattern, test the stated pattern_deviation_reason.
• .adlc/test_plan.json mapping ac_id -> test_path -> expected_pre_change_failure_reason
• When a Loop Contract is active, each generated test also records coverage_tags and covers_required_tests so bin/adlc loop-test-selection can mechanically prove the mandatory floor and task-signal tests are covered.
• When required Loop Contract tests have actually run, record the independent result refs in execution_evidence_refs and top-level test_result_refs, then use bin/adlc loop-test-selection --require-test-results [path] so coverage cannot pass on tags alone.
• .adlc/pre_change_run.txt capturing stdout from the pre-change run that proves the generated verifier fails for the documented reason

Authoring Workflow

1. Read the task's verification_spec, acceptance_criteria, files_to_create/modify, and reference_impl.
2. If module_plan.applicability=required, author module_plan.architecture_test first. The test must enforce the listed files, one-line responsibilities, pure/impure markings, registered pattern constraints when module_plan_source=pattern, and forbidden catch-all names before any production code is written.
3. Discover the repo's native test root, framework, helper layout, fixture style, and test naming conventions at runtime.
4. Normalize verifier scope against verification_spec.target_files when present.
5. If loop_contract_path is active, read the Loop Contract and list its test_selection.mandatory_floor and test_selection.required_from_task_signals test IDs before authoring tests.
6. Author the smallest failing tests or reproducers that make each acceptance criterion falsifiable before code changes.
7. Tag generated tests with coverage_tags and covers_required_tests when they satisfy a Loop Contract test ID.
8. Run bin/adlc loop-test-selection --loop-contract [loop_contract_path] --test-plan .adlc/test_plan.json --json before returning done when a Loop Contract is active.
9. If a loop-test-result artifact exists for required tests, run bin/adlc loop-test-selection --loop-contract [loop_contract_path] --test-plan .adlc/test_plan.json --require-test-results [loop-test-result path] --json before returning done.
10. Run the generated verifier set before editing production code.
11. Record the failing stdout in .adlc/pre_change_run.txt.
12. Emit .adlc/test_plan.json only after the self-check passes.

Behavior by Task Class

feature

• Translate every acceptance criterion into at least one named test that exercises the public behavior promised by the brief.
• Prefer tests at the narrowest stable seam already used by the repo: public API, service entrypoint, command handler, or component contract.
• Reuse existing fixtures, factories, and helper setup from the discovered test conventions before creating new data scaffolding.
• Make the pre-change failure expose the missing behavior directly in the assertion message, not via a broad "command failed" wrapper.

bugfix

• Start from the observed failure path and encode the regression as the first failing test or reproducer.
• Keep the fixture or input payload as small as possible while still reproducing the bug.
• Assert both the corrected behavior and the absence of the prior broken result when that can be checked deterministically.
• Name the test after the defect condition so the future failure explains what regressed.

build_validation

• Treat the failing build or test command as the primary verifier and add authored tests only when they sharpen the failure around the changed files.
• If a test is needed, bind it to the compile path, generated artifact, or contract that the command is failing on.
• Capture the exact command output that proves the repo still fails before the code change.
• Do not expand maintenance work into unrelated behavioral suites just to satisfy ceremony.

lint_cleanup

• Use the exact lint, fmt, or static-analysis failure as the primary verifier and only author tests when the rule enforces a real code contract.
• Assert the concrete diagnostic, formatting delta, or emitted artifact that the lint rule guards.
• Keep authored reproducers scoped to the files and rules named in the brief.
• Avoid runtime behavior tests unless the lint issue changes executable output.

refactor

• Add characterization tests at the public seams around the files being reorganized before moving code.
• Favor assertions that pin stable behavior, serialized output, or cross-module contracts over internal implementation details.
• When target_files is set, place or select tests whose paths intersect those files or the immediate native test peer paths that exercise them.
• Split large refactor coverage into multiple small tests so each acceptance criterion stays traceable.

infra

• Author smoke tests or reproducers around startup scripts, CI glue, migrations, configuration rendering, or deployment entrypoints touched by the task.
• Assert exit codes, rendered config, artifact creation, or orchestrated command output instead of human inspection steps.
• Replace ambient environment assumptions with deterministic fixtures, temp directories, or mocked process inputs.
• Tie each test name to the operational invariant the infra change must preserve.

docs

• Convert executable examples, CLI snippets, schema examples, or config fragments in scope into doctest-style verifiers or reproducers.
• Assert the documented command output, parse result, validation result, or sample render that the docs promise.
• Keep tests tied to the documented surface in target_files when those paths are provided.
• Reject prose-only completion when the brief still defines a deterministic verifier.

security

• Author negative tests first: unauthorized access, malformed input, missing auth, unsafe output, or invariant-violation attempts.
• Assert the exact denial status, redaction behavior, audit event, or invariant hold that separates secure from insecure behavior.
• Use deterministic malicious payload fixtures; do not rely on live services or random fuzzing for the gating test.
• When expected_failure_mode is set, make the pre-change run show that specific vulnerable behavior or failure signature before coding starts.

Quality Gates

Before emitting done, all of these must pass:

1. Every acceptance_criteria id has >=1 generated test.
2. Every test has at least one concrete assertion; no assert true, no empty bodies.
3. Pre-change run captured; failure reason matches expected_failure_mode when set.
4. Test file paths intersect verification_spec.target_files when target_files is set.
5. Generated tests pass stop-slop anti-stub patterns.
6. test_plan.json validates against the schema in this skill.

Loop Contract addendum:

• Mandatory floor tests and required task-signal tests must be covered when loop_contract_path is active.
• Agent-selected tests are additive only. Do not remove or downgrade required tests to make the plan cheaper.
• Coverage must be machine-readable through coverage_tags and covers_required_tests; prose explanations do not satisfy bin/adlc loop-test-selection.
• For productionization, robust maturity requires executed required-test evidence in docs/schemas/loop-test-result.schema.json; tag-only plans are capped below robust.

Output Schema (test_plan.json)

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "ADLC Spec-to-Tests Plan",
  "type": "object",
  "additionalProperties": false,
  "required": [
    "brief_id",
    "task_id",
    "generated_tests",
    "pre_change_run_path",
    "verifier_target_intersection",
    "self_check"
  ],
  "properties": {
    "brief_id": {
      "type": "string",
      "minLength": 1
    },
    "task_id": {
      "type": "string",
      "minLength": 1
    },
    "generated_tests": {
      "type": "array",
      "minItems": 1,
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": [
          "ac_id",
          "test_path",
          "test_name",
          "expected_pre_change_failure_reason",
          "assertion_count"
        ],
        "properties": {
          "ac_id": {
            "type": "string",
            "pattern": "^AC-[A-Z0-9_-]+$"
          },
          "test_path": {
            "type": "string",
            "minLength": 1
          },
          "test_name": {
            "type": "string",
            "minLength": 1
          },
          "expected_pre_change_failure_reason": {
            "type": "string",
            "minLength": 1
          },
          "assertion_count": {
            "type": "integer",
            "minimum": 1
          },
          "coverage_tags": {
            "type": "array",
            "items": {
              "type": "string",
              "minLength": 1
            }
          },
          "covers_required_tests": {
            "type": "array",
            "items": {
              "type": "string",
              "minLength": 1
            }
          },
          "execution_evidence_refs": {
            "type": "array",
            "items": {
              "type": "string",
              "minLength": 1
            }
          }
        }
      }
    },
    "test_result_refs": {
      "type": "array",
      "items": {
        "type": "string",
        "minLength": 1
      }
    },
    "pre_change_run_path": {
      "type": "string",
      "const": ".adlc/pre_change_run.txt"
    },
    "verifier_target_intersection": {
      "type": "boolean"
    },
    "self_check": {
      "type": "object",
      "additionalProperties": false,
      "required": [
        "gate_1",
        "gate_2",
        "gate_3",
        "gate_4",
        "gate_5",
        "gate_6"
      ],
      "properties": {
        "gate_1": {
          "type": "string",
          "enum": ["pass", "fail"]
        },
        "gate_2": {
          "type": "string",
          "enum": ["pass", "fail"]
        },
        "gate_3": {
          "type": "string",
          "enum": ["pass", "fail"]
        },
        "gate_4": {
          "type": "string",
          "enum": ["pass", "fail"]
        },
        "gate_5": {
          "type": "string",
          "enum": ["pass", "fail"]
        },
        "gate_6": {
          "type": "string",
          "enum": ["pass", "fail"]
        }
      }
    }
  }
}

Failure Modes

• Missing structured acceptance criteria -> emit stuck.
• Cannot make the authored verifier fail pre-change -> emit stuck.
• Verifier coverage against target_files cannot be confirmed -> emit revise.

Anti-Slop

No TODO/FIXME/PLACEHOLDER. No stub functions. No commented-out code. Every function has a real implementation. Every import is used.

Banned in shipped code and generated tests:

• TODO, FIXME, PLACEHOLDER
• todo!(), unimplemented!(), panic!("not implemented")
• NotImplementedError, pass, empty placeholder bodies
• fake/mock placeholder logic in production code
• unwired entry points, dead handlers, unused providers, and unused config added "for later"
• assert true, empty it() or test() bodies, and snapshot-only tests with no concrete assertion

Examples

Feature Example

Input acceptance criterion:

{
  "id": "AC-BALANCE-ZERO",
  "given": "a newly created account with no ledger entries",
  "when": "the account summary is requested",
  "then": "the summary returns balance_cents set to 0",
  "measurable_post_condition": "response.json.balance_cents == 0"
}

Output test body:

def test_account_summary_returns_zero_balance_for_new_account(client, account_factory):
    account = account_factory(entries=[])

    response = client.get(f"/accounts/{account.id}/summary")

    assert response.status_code == 200
    assert response.json["balance_cents"] == 0

Expected pre-change failure reason:

AssertionError: assert None == 0

Bugfix Example

Input acceptance criterion:

{
  "id": "AC-CSV-TRAILING-NEWLINE",
  "given": "a CSV export with one data row",
  "when": "the exporter serializes the file",
  "then": "the export ends with exactly one trailing newline"
}

Output test body:

def test_export_csv_ends_with_exactly_one_trailing_newline(report_factory):
    report = report_factory(rows=[{"id": 1, "name": "Ada"}])

    output = export_csv(report)

    assert output.endswith("\n")
    assert not output.endswith("\n\n")
    assert output == "id,name\n1,Ada\n"

Expected pre-change failure reason:

AssertionError: assert not True