testing

name	testing
description	Testing philosophy, coverage workflow, and test generation for the Furst compiler
allowed-tools	Read, Edit, Write, Glob, Grep, Bash

Testing Skill

Philosophy

Boundary testing, not unit testing

Do not write per-function unit tests. They couple tests to implementation details and break when internals are refactored. Instead, identify the entrypoint functions for each directory/module — typically one or two public functions that everything else feeds through. These are the test boundaries. Test through them.

If a directory has parse, lower, emit as its public API, those are the three boundaries. Everything behind them is implementation detail.

Feature-driven, not function-driven

Every test should exercise a language feature, not a function. The test says "this feature works" by feeding representative source code through a boundary and asserting on the output. The set of language features the language supports is the test matrix.

Bad paths are first-class and separate

Bad-path tests live separately from happy-path tests. Ideally a separate test project; at minimum separate files in their own subdirectory. Keeping them apart prevents congestion — happy-path tests stay focused on "this feature works" and bad-path tests stay focused on "this invalid input is rejected correctly." Separation lets you run and reason about them independently.

Branch coverage as dead code detector

After tests cover all language features through all boundaries, any branch with zero coverage is suspect. Ask: can a language feature exercise this branch? If yes, write the test. If no, it's dead code — remove it. Coverage is not a vanity metric; it's a dead code finder.

Workflow

1. Define the feature matrix

Enumerate every feature the language supports (or will support). Group by category: expressions, bindings, functions, modules, types, structs, etc. Each feature is a row. Each boundary it should pass through is a column. The matrix tells you what's tested and what's not.

2. Write happy-path tests through boundaries

For each feature, write at least one test through the highest-level boundary that exercises it (prefer end-to-end roundtrip when possible). The test provides source code as input and asserts on structured output. No mocking, no internal state inspection.

3. Write bad-path tests for rejection

For each construct that should be invalid, write a test providing the bad source and asserting on the exact error type and message. Organize by error category: lex errors, parse errors, type errors, symbol errors.

Bad-path test pattern:

input:  invalid source string
assert: Result is Error with expected message/type

4. Run coverage and analyze

# run tests with coverage collection
# generate a summary report
# review per-module branch coverage

For each uncovered branch:

• Reachable by a feature? Write the test.
• Unreachable? Dead code. Remove it.

5. Exclude generated code

Autogenerated code (protobuf, codegen output) should be excluded from coverage targets. It inflates uncovered lines and isn't meaningful to test through feature boundaries.

Principles

• One test, one feature, one boundary. A test should prove a single feature works through a single boundary. Combining multiple features in one test makes failures ambiguous.
• Source in, structured data out. Tests take source text as input and assert on parsed/lowered/emitted structured output. No testing internal state.
• Errors are behavior. Bad-path tests are not second-class. A compiler that silently accepts invalid code is broken.
• Coverage without features is noise. Don't write tests to hit branches. Write tests to cover features. Coverage follows naturally. Residual uncovered branches after full feature coverage = dead code.
• No mocking. Each boundary is a pure-ish function from input to output. Test the real thing.
• Readable source inputs. Test source code must use triple-quoted strings (""") starting on a new line, with the source formatted naturally across multiple lines. This makes tests read like real Furst code, not compressed one-liners.