| name | using-clj-mutate |
| description | Use when mutation-testing Clojure code, assessing test quality beyond coverage, or investigating surviving mutations to strengthen specs |
Using clj-mutate
Overview
Mutation testing for Clojure. Mutates source code systematically, runs your specs against each mutant, and reports which mutations survived (tests didn't catch) vs were killed (tests caught). Surviving mutations reveal gaps in your test suite that line coverage alone cannot detect.
Setup
clj-mutate auto-detects whether your project uses bb.edn (babashka) or deps.edn (JVM Clojure) and adjusts its spec runner and worker setup accordingly.
deps.edn projects
Add a :mutate alias to your project's deps.edn:
:mutate {:main-opts ["-m" "clj-mutate.core"]
:extra-deps {clj-mutate/clj-mutate {:local/root "/path/to/clj-mutate"}
org.clojure/tools.reader {:mvn/version "1.4.2"}}
:extra-paths ["spec"]}
Requires Speclj as test runner with a :spec alias that runs all specs.
Optional coverage integration (skips mutations on uncovered lines):
:cov {:main-opts ["-m" "speclj.cloverage" "--" "-p" "src" "-s" "spec" "--lcov"]
:extra-deps {cloverage/cloverage {:mvn/version "1.2.4"}
speclj/speclj {:mvn/version "3.12.2"}}
:extra-paths ["spec"]}
bb.edn projects
Add mutate and spec tasks to your project's bb.edn:
{:paths ["src" "spec"]
:deps {clj-mutate/clj-mutate {:local/root "/path/to/clj-mutate"}
org.clojure/tools.reader {:mvn/version "1.4.2"}
speclj/speclj {:mvn/version "3.12.2"}}
:tasks {spec {:doc "Run all specs"
:requires ([speclj.main :as speclj])
:task (speclj/-main "-c")}
mutate {:doc "Run mutation testing"
:requires ([clj-mutate.core :as mutate])
:task (apply mutate/-main *command-line-args*)}}}
Important: use speclj 3.12.2+. clj-mutate detects killed mutants by checking whether bb spec exits non-zero. Speclj 3.12.2 correctly propagates exit codes under babashka via {:babashka/exit N}. Earlier versions call System/exit which bb's task runner intercepts, causing bb to always exit 0 — making all mutants appear to survive.
Requires a spec task that runs all specs. Cloverage is not available under babashka; coverage-guided filtering is skipped (all lines are mutation-tested). If an lcov.info file is present from an external source, it will be used.
Usage
clj -M:mutate src/myapp/foo.cljc
clj -M:mutate src/myapp/foo.cljc --lines 45,67,89
bb mutate src/myapp/foo.cljc
bb mutate src/myapp/foo.cljc --lines 45,67,89
The tool automatically:
- Detects project type (
bb.edn or deps.edn) and uses the appropriate spec runner
- Runs a baseline test to verify all specs pass unmodified
- Applies each mutation, runs all specs with timeout (10x baseline)
- Restores original file after each mutation
- Stamps source with
;; mutation-tested: YYYY-MM-DD on full runs
- For deps.edn projects: runs coverage if
lcov.info is missing or stale
- For bb.edn projects: uses existing
lcov.info if present, otherwise tests all lines
Mutation Rules
| Category | Mutations |
|---|
| Arithmetic | + <-> -, * -> /, inc <-> dec |
| Comparison | > <-> >=, < <-> <= |
| Equality | = <-> not= |
| Boolean | true <-> false |
| Conditional | if <-> if-not, when <-> when-not |
| Constant | 0 <-> 1 |
Interpreting Results
| Result | Meaning | Action |
|---|
| Killed | Test caught the mutation | Tests are strong here |
| Survived | Tests passed with mutant code | Write a spec that would fail with the mutation |
| Timeout | Mutation caused infinite loop | Treated as killed (mutation had observable impact) |
| Equivalent | Mutation can't change behavior | Auto-suppressed; no action needed |
Equivalent Mutation Suppression
Known-equivalent mutations are auto-suppressed to reduce false survivors:
</<=/>/>= mutations on (rand) comparisons= mutations in rand-nth single-element guards- Constants inside
rand-nth [...] pools
> -> >= at subvec boundaries
Workflow
- Write specs using BDD/TDD
- Run mutation testing:
- deps.edn:
clj -M:mutate src/myapp/foo.cljc
- bb.edn:
bb mutate src/myapp/foo.cljc
- Review survivors -- each is a test gap
- Write specs to kill survivors
- Retest survivors:
- deps.edn:
clj -M:mutate src/myapp/foo.cljc --lines 45,67
- bb.edn:
bb mutate src/myapp/foo.cljc --lines 45,67
- Repeat until kill rate is satisfactory
Common Mistakes
- Specs not running: Ensure your
:spec alias (deps.edn) or spec task (bb.edn) runs all specs under spec/
- Specs fail at baseline: Fix your specs before mutation testing
- Chasing equivalent mutations: Some survivors are mathematically equivalent; suppress them rather than writing impossible tests
- Missing coverage in bb projects: Cloverage is JVM-only. Babashka projects test all lines by default, which is slower but thorough
- All mutants survive in bb projects: Requires speclj 3.12.2+ which propagates exit codes correctly under babashka. Earlier versions always exit 0 regardless of failures, so clj-mutate can't detect killed mutants.
