| name | testing |
| description | Use when: designing, reviewing, registering, or running cmake-toolset CTest cases, configure/module regressions, generated-tool checks, or dependency link/startup smoke tests. For port upgrades use port-upgrade; for CI failure diagnosis use ci-fix-port. |
Testing (cmake-toolset)
This repository's test/ tree primarily validates CMake configure behavior, dependency import/build/link integration,
generated tools, and executable startup. Do not describe all of it as unit coverage.
Read test design and acceptance when planning, writing, or reviewing tests.
It is not needed merely to run a known configured test.
Choose the correct layer
- For a reusable module/helper regression, prefer the smallest fixture project or
cmake -P-style check that invokes
the real function/macro and asserts its observable configure result, generated artifact, property, or diagnostic.
- For a port/import change, test the contract cmake-toolset owns: package discovery, target availability/properties,
generated-tool invocation, compile/link compatibility, and startup. Do not duplicate upstream library behavior tests.
- A trivial GTest/benchmark/main executable can prove that an imported target builds, links, and starts. It does not
prove GTest discovery, benchmark semantics, or unrelated cmake-toolset behavior unless the registered command executes
that target and asserts the claimed outcome.
- Cross-compiling paths may only build target artifacts when they cannot execute on the host. Report build-only evidence
separately from executed tests and use
port-upgrade for the affected platform/dependency matrix.
Register what you intend to run
- For every
add_test, verify NAME, COMMAND, target/file, arguments, working directory, configuration, platform
guard, and expected observable result against current test/CMakeLists.txt and source. Building a test executable does
not mean CTest executes it.
- Use
ctest --test-dir <BUILD_DIR> -N -V to inspect the resolved command before claiming coverage. Keep names in the
existing cmake-toolset.<scenario> style and make the scenario/result clear.
- A configure-failure case must fail for the intended contract violation and distinguish that diagnostic/result from an
unrelated missing tool, download, compiler, or environment failure.
- Keep generated files and nested fixture build trees under the selected test build directory. Reuse the current
generator, platform, toolset, cache options, and CI mode; do not write artifacts into the repository root.
Run the narrowest evidence first
Reuse an existing configured tree such as the current test/build_jobs_* directory. When a new tree is required, match
the relevant ci/do_ci.* configuration rather than inventing options:
cmake -S test -B <BUILD_DIR> <verified-options>
cmake --build <BUILD_DIR> --config <CONFIG> --parallel
ctest --test-dir <BUILD_DIR> -C <CONFIG> -R "^cmake-toolset\.<case>$" --output-on-failure
Omit -C only for a verified single-config generator. After the focused case, run the affected CTest set and the
platform/backend matrix justified by risk. Configuration or dependency downloads may be expensive and network-backed;
report unavailable prerequisites and unrun matrix entries rather than treating them as green.
Apply cmake-format to changed CMake test/fixture files as required by AGENTS.md. For port/test integration, also
follow port-upgrade/references/repository-maintenance-guidelines.md.
