| name | wagoe-gen-tests |
| description | Generate a Kaocha test namespace for a Wagoe source file with `bb ai gen-tests`, put it in the right place, and run it. Use when asked to write tests for an existing namespace, to cover a function that has none, or to bootstrap a test file for freshly scaffolded code. Covers the test-type metadata Kaocha selects on, where the file belongs, and how to triage the failures a generated draft always has. |
Wagoe — generating tests
bb ai gen-tests <source-file>
bb ai gen-tests <source-file> --write
bb ai gen-tests <source-file> -o <file>
It needs an AI provider. Check first — the command is otherwise a slow way to
discover that:
bb doctor:env
The output is a draft, and the failures are the point
Measured on this repository's own libs/ai/src/wagoe/ai/core/context.clj: the
generated namespace loaded, ran, and reported
Ran 10 tests containing 84 assertions.
{:test 10, :pass 74, :fail 10, :error 0}
Ten failing assertions out of eighty-four is the normal result for a subtle
namespace, not a malfunction. The model infers what each function should
return from its name and shape; some of those guesses are wrong. A short module
of obvious pure functions fared much better in the same measurement — 10 of 10
— so expect the failure rate to track how much of the behaviour a reader could
guess from the source.
Every failure is either a wrong assertion or a real bug, and only you can say
which. Read the function, then fix whichever is wrong — never delete the test
to get green.
Treat the run as a review queue. This is why the skill runs the tests rather
than stopping at "file written".
Where the file goes, and what metadata it gets
Both are derived from the source path, in Clojure, not asked of the model:
| Source path contains | Kaocha metadata |
|---|
/core/ | ^:unit |
/adapters/ | ^:contract |
anything else (/shell/, …) | ^:integration |
libs/user/src/wagoe/user/core/validation.clj
-> libs/user/test/wagoe/user/core/validation_test.clj ^:unit
--write puts it there. A source file with no src/ path segment has no
convention to apply — --write refuses and asks for -o rather than guessing
a location Kaocha will not look in.
The metadata matters more than it looks: Kaocha selects suites on it. A
deftest without it is in the file and in no test run. bb ai gen-tests
stamps it now; before it merely asked the model, which supplied it on 0 of 15
measured deftests.
What the tool repairs, and what it will not
Three failures were reproducible enough to be fixed in bb ai gen-tests
itself, so you do not need to check for them:
-
Missing metadata — stamped from the path, as above.
-
A namespace used but not required. str/join in the body with no
[clojure.string :as str] fails at load with No such namespace: str;
clojure.set/subset? with no require fails with
ClassNotFoundException: clojure.set. Both are repaired for the standard
clojure.* aliases (str set walk edn io pprint). An alias outside that set
is left to fail, because the namespace behind it cannot be inferred.
-
An answer cut off mid-form. A long source file can exhaust the model's
output budget; the reply then ends inside a form and the file cannot be
read. That is now reported as an error instead of written to disk:
Error: The model's answer was cut off mid-form — 598 lines, delimiters
still open. Generate for a smaller source file, or raise :max-tokens.
If you hit it, generate for a smaller namespace. Splitting a 250-line source
file into two calls is more reliable than raising the budget.
It does not check that the tests pass, that the assertions are right, or
that coverage is meaningful. That is the step below.
Steps
bb doctor:env — confirm a provider. Without one, stop; there is no
offline fallback for this command.bb ai gen-tests <source-file> --write. If the destination already exists
it refuses rather than overwriting — with -o as well as --write. Write
elsewhere and merge by hand, or pass --force only when you mean to replace
an earlier draft.- Run just the new namespace and read every failure:
clojure -M:test --focus <the.test.namespace>
In this repository, add the library suite: clojure -M:test :ai --focus ….
- Triage each failure — wrong assertion, or real bug. Fix the right one.
bb check:placeholder-tests. Generated drafts have not produced (is true)
placeholders in practice, but the gate is cheap and it is a hard CI failure.bb check before committing.
What this does not cover
The generated namespace tests the public functions it can see in one file. It
does not know about the database, the Integrant system, or the ports a shell
namespace depends on, so ^:integration output for a shell/ file is usually
the weakest of the three — expect to supply the stubs yourself. For a service
that needs a repository, reify the port; the existing tests under
libs/*/test/**/shell/ show the pattern.
Nor does it replace scaffolding: bb scaffold already generates a test
skeleton for a new module. Use this for code that exists and has none.
