# tested
`tested` is a single-command Go test runner, coverage analyzer, and report
generator. It combines a `tparse`-style terminal summary with a self-contained
HTML test report, Go-authored annotated coverage HTML, machine-readable
summaries, and CI artifacts.
`tested` owns a stable `.coverage` artifact bundle containing human-readable
reports, machine-readable summaries, and the authoritative captured evidence.
A coherent, coverage-enabled live run normally produces:
```text
.coverage/
├── test_output.jsonl
├── stderr.log
├── run.json
├── coverage.out
├── coverage.html
├── test_output.html
├── summary.json
├── junit.xml
├── index.html
└── manifest.json
```
A successful generation omits the coverage pair with `--no-coverage`.
`coverage.html` is produced only from a valid profile; a nonempty invalid
`coverage.out` may remain as diagnostic evidence without HTML. Imported
offline bundles may omit unbound `stderr.log` or `coverage.out`.
`manifest.json` is published only for a coherent generation whose requested
reports completed; an ordinary test, build, or coverage-policy failure can
still be coherent and receive a manifest.
Unlike a `go test | tee` shell pipeline, `tested` retains the real child-process
exit status, understands both Go test and build events, preserves repeated test
occurrences, and cannot present stale coverage as a new successful run.
## Quick start
Install from source:
```bash
go install github.com/greenpau/tested@latest
```
For full build provenance, install from a Git checkout using Make:
```bash
git clone https://github.com/greenpau/tested.git
cd tested
make install
tested version
```
`make install` honors Go's `GOBIN` setting (or the first `GOPATH` entry's
`bin` directory) and records the version, full commit, branch, UTC build time,
and builder. Use `GOBIN=/your/bin make install` to select a destination.
`make build` records the same metadata in `bin/tested`.
A plain `go install ...@version` records the module version but usually omits
Git and build provenance. `tested version` uses the metadata embedded in the
executable and labels unavailable fields `not recorded`; it cannot recover
information that was never embedded.
Run every package in the current module:
```bash
tested
```
Pass exact arguments to `go test` after `--`:
```bash
tested run -- -race -count=1 ./...
tested run -- -run TestAuthentication ./...
tested run -C ../my-project -- -shuffle=on ./...
tested run --coverage-diff-base HEAD -- ./...
```
`--coverage-diff-base REV` adds an opt-in source comparison to
`coverage.html`. `REV` must resolve to exactly one commit already present in
the project repository. `tested` does not infer a base or fetch missing Git
objects. This option requires the local `git` executable; ordinary coverage
generation does not.
Regenerate derived reports without rerunning tests:
```bash
tested report
tested report -C ../my-project
tested report -C ../my-project --coverage-diff-base HEAD
tested report -C ../my-project \
--events /archive/test_output.jsonl \
--run-metadata /archive/run.json
```
External event, stderr, and profile evidence is copied byte-for-byte into
tested's canonical managed names before reporting. A matching external
`run.json` is strictly validated and canonically re-encoded; it can also select
bound sibling `stderr.log` and `coverage.out`. Stale managed companions that do
not belong to the selected bundle are removed. For a policy-free bundle,
`report --no-coverage` removes the profile binding while preserving the
authoritative child status and other bindings. A bundle with a recorded
coverage policy is incompatible with `report --no-coverage`; tested rejects
that request before changing the managed bundle.
Run `tested help` for the complete command reference.
## Reports
Each run produces primary evidence first and derives every presentation from
the same normalized result model.
| Artifact | Purpose |
| --- | --- |
| `test_output.jsonl` | Byte-faithful live stdout, or a byte-faithful imported event-log copy |
| `coverage.out` | Native Go coverage profile, live or imported when selected |
| `test_output.html` | Searchable, filterable, self-contained test report |
| `coverage.html` | Selected Go toolchain's annotated-source report with tested's fixed presentation layer |
| `summary.json` | Versioned, compact result for agents and CI |
| `junit.xml` | JUnit-compatible test result exchange |
| `stderr.log` | Byte-faithful live child stderr, or a selected imported copy |
| `run.json` | Strict `tested/run/v1` child status and raw-evidence bindings |
| `index.html` | Offline landing page linking only to managed reports that exist |
| `manifest.json` | Conditional inventory of completed artifacts, published last |
The three HTML entry points—`index.html`, `test_output.html`, and
`coverage.html`—share one compact, dependency-free operational theme. They are
responsive, honor the system dark-mode preference, expose visible keyboard
focus, retain semantic status colors, and print cleanly without fetching
remote assets. Their report-owned HTML, CSS, and JavaScript sources are
compiled into the binary with Go's `embed` package. The coverage page keeps the
selected Go toolchain's source annotation, coverage spans, file selector, and
script intact. An exactly removable head layer adds the viewport,
content-security policy, embedded presentation assets, package filtering,
uncovered-region focus, and expandable context. When
`--coverage-diff-base` is supplied, that layer also carries a bounded,
contextually escaped comparison model for genuine unified and split changes.
A changed-files switch then limits both package and file choices to modified,
added, renamed, and untracked covered sources.
Plain and Markdown console output stream test/package transitions, test and
build logs, child stderr, and report-generation stages. A status line appears
after roughly 10 seconds of silence. Live details are capped at 4 MiB, with
bounded previews; stages, heartbeats, and the final summary continue. Use
`--quiet` for the final summary only. JSON mode remains one final document.
With `--redact`, live log payloads are omitted to prevent secrets leaking across
chunk boundaries; completed reports retain their normal redaction behavior.
See [the console guide](USER_GUIDE.md#reading-the-console-summary) for limits.
Live raw evidence and imported evidence copies are never silently redacted.
Repeatable `--redact REGEXP` options apply to console output,
`test_output.html`, `summary.json`, `junit.xml`, and `index.html`. They do not
change `test_output.jsonl`, `stderr.log`, `coverage.out`, `coverage.html`,
`run.json`, or manifest hashes. `tested` accepts at most 32 expressions, 4096
bytes each and 32 KiB in total, and rejects expressions that match empty input.
A contextual zero-width match or a replacement that would grow a presentation
value beyond the larger of its original byte length and the fixed omission
marker omits that complete value and stops later redaction rules. This prevents
ordered expressions from amplifying output. The managed event-log copy remains
complete.
## Correctness model
- The `go test` process exit status is authoritative for a live run.
- Raw bytes are recorded before JSON interpretation.
- `run.json` is one strict, bounded `tested/run/v1` document. It records
child/cancellation state, monotonic `duration_ns`, capture integrity, issues,
exact coverage-policy evidence, and at most three unique size/SHA-256
bindings for required `test_output.jsonl` plus retained `stderr.log` and
`coverage.out`; duplicate JSON member names are rejected at every depth,
signal cancellation accepts only the canonical interrupt/terminated exit
mappings, and tested-written status encodes bindings in canonical name order.
- Offline mode verifies those bindings and never infers success from an
unbound event stream.
- Go `TestEvent` and `BuildEvent` records remain distinguishable and correlated.
- Go 1.25 test attributes and Go 1.26 artifact-directory records retain their
source sequence and package/test occurrence. Derived JSON, HTML, and JUnit
reports redact their keys, values, and paths; artifact paths remain
informational text and never become links.
- A test identity is package, full test name, and occurrence number. Runs from
`-count`, `-cpu`, or repeated benchmarks are not merged.
- Every started occurrence ends as passed, failed, skipped, benchmarked, or
explicitly incomplete.
- Global coverage is the statement-weighted profile total. Package percentages
printed by `go test` are shown separately and never averaged.
- Derived files are written through same-directory temporary files and renamed
into place. The manifest is the final coherence marker.
- On POSIX systems, a multiply-linked managed regular file is rejected before
tested changes its permissions or hashes it. External evidence imports are
copied to an independent managed file rather than chmodding the source.
- Package and test presentation order is deterministic even when execution is
parallel.
- Cancellation targets the owned process tree, waits up to the default
two-second graceful interval, then forces termination as needed while
draining output and reaping the child.
## Replacing the legacy Makefile pipeline
The following:
```make
test:
@go test -json -v -coverprofile=.coverage/coverage.out ./... \
| tee .coverage/test_output.jsonl
@go-test-report < .coverage/test_output.jsonl \
-o .coverage/test_output.html
@go tool cover -html=.coverage/coverage.out \
-o .coverage/coverage.html
@grep -q '"Action":"fail"' .coverage/test_output.jsonl && exit 1 || true
```
becomes:
```make
test:
@tested run -- -v ./...
```
There are no runtime dependencies on `tparse`, `go-test-report`, or a shell.
The selected Go toolchain is still used for test execution and canonical
coverage HTML.
## Coverage policy
Fail a passing test run when weighted statement coverage is below a threshold:
```bash
tested run --minimum-coverage 80 -- ./...
```
The threshold input is limited to 256 bytes and its grammar is the exact ASCII
plain-decimal form `DIGIT+("."DIGIT+)?` in the inclusive range `0` through
`100`. Signs, whitespace, exponents, percent suffixes, NaN, and infinities are
rejected. Comparison uses the exact weighted statement ratio; equality passes
and display rounding never decides the exit status.
Package coverage observed in Go test events and authoritative profile coverage
answer different questions. `tested` labels them separately.
## Resource bounds
`--max-event-bytes` defaults to 16 MiB per decoded event and also bounds
aggregate temporary assembly of one fragmented Go benchmark result line.
`--max-test-output-bytes` defaults to 1 MiB independently for each test
occurrence, package, build, and unattributed output scope in derived reports.
`--max-total-output-bytes` adds a 64 MiB run-wide retained-output budget, and
`--max-result-entries` limits normalized entities and retained chunks to
1,000,000. `--max-normalized-bytes` separately limits retained identity and
metadata strings to 128 MiB, preventing many small records from creating an
unbounded semantic projection. Zero explicitly disables any of these bounds;
nonzero byte limits must be at least 1024 bytes. These limits never truncate
the managed event log. Exhausting an entity or normalized-string budget marks
the derived result incomplete; clipping derived output alone is reported as
presentation loss because the raw log remains authoritative.
Fragmented benchmark assembly retains a fixed fragment-count safety limit even
when its byte bound is disabled.
Each decoded event additionally permits at most 64 fields unknown to its
selected schema and 64 KiB total decoded unknown-field names plus exact
retained JSON values. Exceeding either extension bound or repeating a decoded
JSON member name makes that record malformed while its raw bytes remain
preserved.
Coverage parsing independently caps one profile at 256 MiB, 100,000 unique
source files, and 1,000,000 unique blocks; these conservative parser limits
protect both live and imported evidence processing.
## Exit codes
| Code | Meaning |
| --- | --- |
| `0` | The command completed successfully under the requested policy |
| `1` | Tests or a package build failed and failures were not allowed |
| `2` | Usage, execution, unavailable required coverage, integrity, or reporting failed |
| `3` | A coherent successful child run was below the exact coverage minimum |
| `130` | SIGINT, deadline, or programmatic cancellation |
| `143` | SIGTERM cancellation |
If the Go command returns another nonzero status or terminates from a signal,
`tested` preserves its authoritative status instead of normalizing it to 1.
Cancellation has highest precedence. The same cancellation projection applies
while regenerating reports; a cancelled report does not publish a manifest.
When tests and report generation both fail, both are diagnosed and the
authoritative child test status wins.
## Security
Test output and annotated coverage source can contain credentials, private file
paths, request data, and implementation details. `tested` creates the artifact
directory with mode `0700` and files with mode `0600`, rejects symlink report
targets, and HTML-escapes all untrusted content. Review and redact reports
before uploading them to shared CI storage. With `--coverage-diff-base`, the
report can also embed source from the selected commit, including deleted lines
that no longer exist in the working tree; treat that baseline source as
equally sensitive.
See [USER_GUIDE.md](USER_GUIDE.md) for operating details and
[SECURITY.md](SECURITY.md) for the report threat model.
## Development
The module requires Go 1.25 or newer and uses only the Go standard library.
All development targets run with Go 1.25. On Go 1.26 or newer, `make e2e`
additionally verifies the version-specific artifact-directory and BuildEvent
records; Go 1.25 reports those assertions as explicitly skipped.
`make test` first runs direct formatting, vet, unit, race, and shuffled checks
as the bootstrap authority. The three direct test stages show each test and
subtest as it runs, with verbose output grouped by package. Packages run one
at a time so Go streams their output immediately; unit and race checks bypass
the result cache, and shuffled checks execute three repetitions.
It then builds the checked-out source as
`bin/tested` and uses that binary for one canonical `-count=1` self-test.
The successful self-test leaves a validated ten-file `.coverage` bundle at the
repository root. `make self-test` runs only that build-and-report stage;
`make coverage` is an alias for the complete `make test` contract. Use
`make clean` to remove the generated binary and reports.
```bash
make skills-check
make test
make self-test
make e2e
make e2e-release
make cross-build
make ci
make release-check
make release
```
Browser E2E tests open generated reports directly through `file://` in Chromium
and WebKit. They require Node.js 20 or newer and a separate, pinned development
dependency; the Go binary still has no browser or Node.js runtime dependency.
Install the tooling explicitly, then run the suite:
```bash
npm --prefix scripts/browser ci
npm --prefix scripts/browser exec -- playwright install chromium webkit
make e2e-browser
```
On Linux, add `--with-deps` to the browser installation command if system
libraries are missing. The suite builds the current binary, runs controlled
fixtures in private temporary directories, and checks hierarchy, filtering,
keyboard controls, print, dark mode, mobile layouts, and the no-JavaScript
fallback. Screenshots and failure traces are written under
`scripts/browser/test-results/`. The Go conformance CI job runs this suite
separately from `make ci`, which keeps its Go-only tool requirements.
Release versions always use major **1**: `1.<minor>.<patch>`, without leading
zeros, prerelease suffixes, or build metadata. Minor releases reset the patch
number to zero. `make version-check`, local builds/installs, and tagged CI
enforce the same version policy.
| Publication command | Version change | Local validation |
| --- | --- | --- |
| `make release` | `1.2.3` → `1.2.4` | Complete `release-check` |
| `make minor-release` | `1.2.3` → `1.3.0` | Complete `release-check` |
| `make fast-release` | `1.2.3` → `1.2.4` | Skipped explicitly |
| `make fast-minor-release` | `1.2.3` → `1.3.0` | Skipped explicitly |
These operator-controlled targets require a clean `main`, the pinned
`versioned` 1.0.36 tool, an up-to-date branch, and an absent next tag locally
and at `origin`'s push destination. Install the tool explicitly with
`go install github.com/greenpau/versioned/cmd/versioned@v1.0.36` if needed.
Checked releases run `release-check` once before updating `VERSION`; fast
releases skip only that local gate. All four create a version-only commit and
an annotated tag, then atomically push only the release branch and that tag,
even when `push.followTags` is enabled. **Tagged CI validation still runs for
fast releases** and must pass before GitHub publication.
`make release-git-check` checks patch-release prerequisites without changing
`VERSION`, committing, tagging, or pushing. It inspects the remote and fetches
the release branch. `RELEASE_BRANCH`, `RELEASE_REMOTE`, and `VERSIONED` can
select a different branch, configured remote, or installed tool path. Only one
push destination is supported; tagged CI still requires the release commit
to belong to `main`. Partial version-update/commit targets refuse to bypass
the complete workflow.
Preflight and local-gate failures stop before the release's version update.
If a commit, tag, or push fails later, inspect the local and remote state and
recover that release before trying another increment; local changes are
retained for review. Do not invoke publication targets from CI or while
unrelated work is present.
`make e2e-release` tests all four public targets with disposable repositories
and local bare remotes, including failed gates, stale branches, rejected
atomic pushes, and unrelated tags. Its controlled version-tool fixture keeps
ordinary `make e2e` and CI independent of an installed release tool. To also
qualify an installed pin, run:
```bash
versioned_path="$(command -v versioned)" && \
TESTED_RELEASE_VERSIONED="$versioned_path" \
go test -v -count=1 -tags=integration ./scripts/releaseversion \
-run '^TestReleaseWorkflowTargets$'
```
Repository engineering contracts begin in [AGENTS.md](AGENTS.md) and route to
the focused skills under `.codex/skills`.
## License
`tested` is licensed under the Apache License 2.0.