extend-commands-api

# Extend the Lettuce Commands API Implement a new Redis command (or extend an existing one) in Lettuce, following the conventions maintainers enforce in review. The workflow is evidence-first: prove the command's behavior on a real server before designing the Java API. Read [.agents/docs/architecture.md](../../../.agents/docs/architecture.md) first for the model behind this flow — especially that the sync/async/reactive/Kotlin command **interfaces are hand-edited source files kept in lockstep by the API consistency test suite** (see [.agents/docs/api-consistency.md](../../../.agents/docs/api-consistency.md)). > **Historical caveat when reading old PRs**: PRs merged before the generator removal > also edit `src/test/java/io/lettuce/core/api/<Group>Commands.java` — the old > template source files. Those files **no longer exist**; do not recreate them. The > flavor interfaces are now edited directly. ## Phase 0 — Gather evidence BEFORE planning Do all of the following before writing any plan or code: 1. **Ask the user for the HLD.** Interactively ask for the path to a markdown file containing the High-Level Design for the command(s) (or confirm none exists). If a path is given, read it fully — it is the primary source for syntax, semantics, reply shape per RESP2/RESP3, and edge cases. 2. **Find the server-side PR in the repo that owns the command.** Route the search by command family — module command families are developed in their owning repositories, not in `redis/redis`: | Command family | Repository | Syntax source | |----------------|------------|---------------| | Core commands, vector sets (`VADD`, …) | `redis/redis` | `src/commands/*.json` | | Search (`FT.*`) | `RediSearch/RediSearch` | PR diff + command docs (no `src/commands/*.json`) | | JSON (`JSON.*`) | `RedisJSON/RedisJSON` | PR diff + command docs | | Probabilistic (`BF.*`, `CF.*`, `CMS.*`, `TOPK.*`, `TDIGEST.*`) | `RedisBloom/RedisBloom` | PR diff + command docs | | Time series (`TS.*`) | `RedisTimeSeries/RedisTimeSeries` | PR diff + command docs | If the search comes up empty in the routed repo, fall back to `redis/redis` (and vice versa) before concluding there is no server PR. First verify that `gh` works in the current (sandboxed) environment: ```bash gh auth status ``` Sandboxes often block access to credential files, so `gh` may report unauthenticated here even though it works on the user's machine. If so, ask the user for permission to run these specific read-only `gh` commands outside the sandbox; only fall back to the unauthenticated GitHub REST API if they decline. Then search for the PR that adds/extends the command on the server: ```bash gh search prs --repo <owning-repo> "<COMMAND NAME>" --limit 10 gh pr view <num> --repo <owning-repo> gh pr diff <num> --repo <owning-repo> # in redis/redis: src/commands/*.json has exact syntax ``` Extract: exact wire syntax (argument order and optionality), reply type per **RESP2 and RESP3** (they can differ — this determines the `CommandOutput` and the reactive `Mono`/`Flux` mapping), error conditions, and the **first server version carrying the feature** (drives `@EnabledOnCommand` gating and the test env version). Note whether the server marks the feature *experimental/preview*. 3. **Verify the command exists on the "next" Redis OSS version.** Start the integration environment using the **highest** version the Makefile supports (check `SUPPORTED_TEST_ENV_VERSIONS` in the `Makefile`, or the `src/test/resources/docker-env/.env.vX.XX` files — pick numerically, `8.10` > `8.8`): ```bash grep SUPPORTED_TEST_ENV_VERSIONS Makefile make start version=8.10 ``` Then probe the running servers. Test connection defaults live in `src/test/java/io/lettuce/test/settings/TestSettings.java` — the main standalone node is `localhost:6479` (no auth); module commands (`FT.*`, `JSON.*`, `BF.*`, `VADD`, …) run on the stack node at `localhost:16379`: ```bash redis-cli -p 6479 INFO server | grep redis_version redis-cli -p 6479 COMMAND INFO <COMMAND> # non-empty → command exists redis-cli -p 6479 COMMAND DOCS <COMMAND> # arity/args — compare with the server PR ``` For a NEW command, `COMMAND INFO` must return a non-empty reply. For an EXTENDED command, additionally invoke it with the new syntax against a scratch key and confirm the server accepts it (no `ERR syntax error` / `ERR unknown argument`). **If the command/option is missing on the latest env version**, the image tag pinned in the `.env.vX.XX` file is too old. Ask the user for a `redislabs/client-libs-test` image tag that contains it (e.g. an RC/edge build — tags are listed at https://hub.docker.com/r/redislabs/client-libs-test/tags; you may check and suggest candidates), then: ```bash make stop CLIENT_LIBS_TEST_IMAGE_TAG=<tag> make start ``` If the PR should also move CI to that build, update the newest `.env.vX.XX` pin as part of the change — command PRs do this when they need a fresh server (cf. the hotkeys PR bumping `.env.v8.6`). If no tag carries the feature, report it and proceed: implementation continues, integration tests stay gated until an image ships the change. Keep the environment running for later test runs. 4. **Create redis-cli showcase test cases.** Once the command is available, derive a small set of redis-cli scenarios from the HLD and the server PR and run them. These serve two purposes: - **Smoke test**: prove the documented behavior holds — happy path, each new option/keyword, reply shape (run with `redis-cli -3` too when RESP3 replies differ), edge cases and error conditions (missing key, out-of-range args, conflicting options) — so the Java implementation is built against observed replies, not assumptions. - **Showcase**: each scenario should read as a mini use-case explaining WHY the command/option exists. Prefer realistic data over `foo`/`bar`. Save the scenarios as a commented script in a scratch file (one block per use-case: a "what this demonstrates" comment, the commands, the observed reply pasted back as comments). Carry this forward: it feeds the Phase 1 plan, the test assertions, and the PR description. If the command could not be made available on any image, still write the scenarios as *expected* transcripts and mark them unverified. 5. **Read the testing and consistency docs**: [.agents/docs/integration-testing.md](../../../.agents/docs/integration-testing.md) (environment, `*UnitTests` vs `*IntegrationTests` naming, base/overload test structure, running one test) and [.agents/docs/api-consistency.md](../../../.agents/docs/api-consistency.md) (the flavors and their return-type mapping rules). 6. **Trace one analogous existing command** end-to-end (same command group, similar reply shape) so the plan mirrors real code, not guesswork. Good reference PRs, validated against git history: | Commit | What it exemplifies | |--------|---------------------| | `7ca3e9cc0` (CLIENT NO-TOUCH #3776) | Minimal new command: all 6 flavors + builder + 2 dispatch layers + Kotlin impl + builder unit test + integration test | | `312ecc7b4` (INCREX #3746) | New command with args: self-typed abstract args base, new value type, new `CommandOutput`s, per-type args/output unit tests, **creating a missing RESP2 overload class** | | `b009df37b` (XNACK #3728) | New command with an enum-valued argument | | `9a0899875` (BITOP extensions #3334) | New operations/overloads on an existing command | | `c603c8120` (HSCAN NOVALUES #2816) | New overload rippling into helpers (`ScanIterator`, `ScanStream`, `ScanFlow`) | | `aa7b4b0be` (stream idempotency #3637) | Option added to an existing `*Args` class flows through with **no interface change** (the `XAddArgs` part) | | `a209fba70` (CAS/CAD #3512) | Read-only command registered in `ReadOnlyCommands` + count test update | | `838a4d39a` (HOTKEYS #3638) | Map-shaped reply via `ComplexOutput` + a `*ReplyParser`, cluster interface additions, full integration overload set, `.env` image bump | | `6567f2d5e` (RediSearch #3375) | Whole new command area: own `Redis<Area>CommandBuilder`, `arguments`/reply-parser packages | **Conventions beat precedent.** Traced reference commands can predate the current conventions; when the reference code conflicts with a written rule in this skill or the linked docs, the rule wins. Example: `sintercard(K...)` ships without a single-key overload because it predates the one-overload-per-varargs rule — a new command mirroring it must still add the single-argument overload. 7. **Determine the `@since` version** — recipe owned by [.agents/docs/javadoc.md](../../../.agents/docs/javadoc.md): ```bash mvn help:evaluate -Dexpression=project.version -q -DforceStdout ``` Drop `-SNAPSHOT` and the patch digit (`7.7.0-SNAPSHOT` → `@since 7.7`). **The evidence is your working context, not a deliverable.** Hold it in mind to drive the work — do not paste it back to the maintainer or pause for "spec approval." The only stop for the maintainer is the Phase 1 plan approval; surface genuinely ambiguous design choices *there*, with the proposed sync signatures, rather than interrupting earlier. ## Phase 1 — Plan mode, then explicit approval Enter **plan mode**. Using the evidence, classify the change with the decision tree below and enumerate the exact file-by-file touch list, the test matrix, and the gating annotations. The plan must contain: - A short "what this feature enables" section built from the showcase scenarios (Phase 0 step 4), including one or two representative command/reply transcripts. - **The proposed sync interface signature(s) with their Javadoc.** The sync interface is the human-authored **API contract**; every other flavor is derived from it and it is costly to change once mirrored. Plan approval is the maintainer's sign-off on that contract — if the signatures must deviate later during implementation, stop and re-confirm before mirroring. - **The complete overload set, enumerated.** For every varargs parameter, list the matching single-argument overload; for a multi-key command with an `*Args` object, use the `List<K>` shape (no varargs) and list the fixed-arity convenience overloads instead — both house rules live in "Types & args conventions". Justify any overload you omit so the maintainer signs off on the exception. An overload discovered missing in review means reworking all six flavors. Present the plan and **explicitly ask permission to execute** before implementing. Do not start editing files until the user approves. Once approved, copy this checklist into your working notes and tick items off: ``` Extend-commands progress: - [ ] 0. Evidence: HLD, server PR, live probe + showcase, RESP2/RESP3 replies, @since - [ ] 1. Plan approved — incl. the sync signature(s), the API contract - [ ] 2. Types: argument & response types (they must exist before any interface edit) - [ ] 3. Sync interface: full overload set (varargs → single-arg too) + Javadoc (@param constraints + @throws for builder-validated preconditions) - [ ] 4. Mirror: async, reactive, Kotlin, NodeSelection×2 — consistency tests pass - [ ] 5. Implementations: CommandType/Keyword, builder, async, reactive, Kotlin impl - [ ] 6. Tests: args/builder/output unit tests + integration base/overloads - [ ] 7. Docs: entry in the current-release section of docs/new-features.md - [ ] 8. Verify: mvn clean test + a single integration test run; then make stop ``` ## Decision tree — what kind of change is this? **A. Extension of an existing command that fits an existing `*Args` class** (new option token / new field): - Touch ONLY the `*Args` class (+ `CommandKeyword` for new tokens) + tests. Do NOT touch the command interfaces, builder signatures, or dispatch layers — the existing `args.build(commandArgs)` delegation carries the new option through automatically (cf. the `XAddArgs` part of the stream-idempotency PR). - Add a fluent setter returning `this`; register new tokens in `src/main/java/io/lettuce/core/protocol/CommandKeyword.java`. - If the reply shape grows, extend the response model/output backward-compatibly. **B. New command(s) in an existing group — core or module area** — the FULL matrix, in this order (types first — every flavor references them, so they must exist to compile). For a command joining an **existing module area** (a new `FT.*` method in the Search group, a new `JSON.*` method, …) the same matrix applies with the area substitutions: the group is the area's flavor interfaces, the builder is the area's `Redis<Area>CommandBuilder` (+ its `Redis<Area>CommandBuilderUnitTests`), argument/reply types go in the area package, and gating/tests follow the module rules in D (capability probe, stack node). The dispatch layers are the same `AbstractRedisAsyncCommands` / `AbstractRedisReactiveCommands` and Kotlin `*Impl.kt` as for core commands. 1. **Argument/response types** — see "Types & args conventions" below. 2. **Sync interface** `src/main/java/io/lettuce/core/api/sync/<Group>Commands.java` — pick the group by command family (STRING → `RedisStringCommands`, HASH → `RedisHashCommands`, generic-key → `RedisKeyCommands`, …). This is the reference signature + Javadoc all flavors mirror (see the [writing-javadoc](../writing-javadoc/SKILL.md) skill; `@since` mandatory): ```java /** * Returns the length of the string value stored at {@code key}. * * @param key the key, must not be {@code null}. * @return the length of the string at {@code key}, or {@code 0} when {@code key} does * not exist. * @since 7.7 */ Long strlen(K key); ``` Three contract rules to apply while designing the signatures and their Javadoc: - **Multi-key commands that also take an `*Args` object take the keys as `List<K>`, not varargs.** A varargs parameter must come last, so a trailing options object can only follow it via an awkward leading placement (cf. the older `sintercard(long limit, K... keys)`). For new commands, take the keys as `List<K>` so the `*Args` argument comes last — `sunioncard(List<K> keys, SUnionCardArgs args)` — and add fixed-arity convenience overloads (`(K key1, K key2)` and their `*Args` variants) instead of a varargs form. Since there is no varargs parameter here, the single-argument-overload rule below does not apply; add only the overloads that are meaningful — e.g. **no single-key `sunioncard`/`sdiffcard`**, whose one-key union/difference is just `SCARD`. - **Every varargs parameter gets a single-argument overload** — `foo(K key, V value)` alongside `foo(K key, V... values)`. This is a hard convention for new commands and overrides any traced precedent that lacks it (older commands predate the rule). Does not apply to the `List<K>`-shaped multi-key-with-`*Args` commands above, which take no varargs. - **Validated preconditions are contract.** Any constraint the builder will enforce (null checks, non-empty varargs) must be stated in the `@param` text with the house phrases (`must not be {@code null}.`, `must not be empty.`) *and* documented with a matching `@throws IllegalArgumentException if …` tag — forms owned by [.agents/docs/javadoc.md](../../../.agents/docs/javadoc.md). Both are then mirrored to every flavor. 3. **Mirror to every flavor**: async, reactive, Kotlin coroutines, and the two cluster node-selection interfaces (`NodeSelection<Group>Commands` / `NodeSelection<Group>AsyncCommands`), applying the per-flavor return-type mapping rules from [.agents/docs/api-consistency.md](../../../.agents/docs/api-consistency.md). 4. **Protocol enums** — command name in `src/main/java/io/lettuce/core/protocol/CommandType.java` (wire bytes derive from the enum name); subcommand tokens in `CommandKeyword`. **Never add a `CommandKeyword` that duplicates a name already in `CommandType`** (e.g. `SET`, `DISCARD`) — the builder static-imports both enums and the bare name becomes ambiguous; reference `CommandType.<NAME>` directly instead. 5. **Builder** — `RedisCommandBuilder.java` (or the area builder, see D): null checks via `LettuceAssert`/`notNullKey`, `CommandArgs` in **wire order**, the `CommandOutput` chosen from the observed RESP2/RESP3 replies: ```java public Command<K, V, Long> strlen(K key) { notNullKey(key); return createCommand(STRLEN, new IntegerOutput<>(codec), key); }