kms-adversarial-test

# Eviden KMS — PR Review & Final-Layer Testing Agent ## 0 — Prerequisites **KMS server**: must be running on port `9998` before any test. Check first; start if needed. ```bash # non-FIPS (most common — all algorithms enabled) pnpm -C ui build && cargo run -p cosmian_kms_server --features non-fips -- -c test_data/configs/server/no_auth.toml # FIPS only pnpm -C ui build && cargo run -p cosmian_kms_server -- -c test_data/configs/server/no_auth.toml ``` Adapt config to the PR: TLS → `tls_auth_non_fips.toml`, JWT → `jwt_auth.toml`, PostgreSQL → start DB with `docker compose up -d` first. Full config reference: `reference/kmip-ops-and-configs.md`. **Health check:** ```bash curl -s http://127.0.0.1:9998/health | jq . # {"status":"UP","latency_ms":...,"dependencies":{"database":{"name":"sqlite","status":"UP"}}} curl -s http://127.0.0.1:9998/server-info | jq . # {"version":"...","fips_mode":false,...} ``` **Chrome DevTools MCP** (for UI testing): install `npx -y chrome-devtools-mcp@latest`. Skill is designed for VS Code but works in any MCP-capable IDE. See `reference/ui-routes.md` for fully-qualified tool names. --- ## 1 — Test Channels Identify which channel(s) the PR affects and test accordingly. ### 1.1 — CLI (`ckms` binary) **When**: PR touches `crate/clients/clap/`, `crate/clients/client/`, `crate/crypto/`, `crate/server/src/core/operations/`, or any KMIP-visible behavior. ```bash cargo run -p ckms --features non-fips -- <subcommand> [args] ``` Full command tree, roundtrip examples → **`reference/cli-commands.md`** ### 1.2 — HTTP endpoints (`curl`) **When**: PR touches `crate/server/src/routes/` (any of: KMIP, access, JOSE REST, JWKS, enterprise integrations, tokenize, middleware). Full endpoint map, curl examples, JOSE algorithm table, JWKS eligibility rules → **`reference/endpoints.md`** ### 1.2.1 — AI token consumption Token names and billing vary by provider. Use provider telemetry; token counts are not word or character counts. |Type|What is counted|Usage control| |---|---|---| |Input|Instructions, history, files, tool results included in a model request|Scope reads; avoid repeated large context/logs| |Cached input|Reused prompt prefix when provider caching applies|Eligibility and pricing vary by provider| |Output|Generated response text and tool-call arguments|Keep responses concise; avoid duplicate reports| |Reasoning|Internal reasoning usage when separately exposed|Provider-specific; use controls only when suitable for task risk| |Multimodal input|Screenshots, images, audio/video sent to capable models|Capture only relevant frames/regions; avoid repeated full-screen payloads| |Tool execution/results|Runtime may be billed separately; returned text may re-enter a later prompt as input|Request only relevant files/logs; preserve necessary error context| Context-window size is capacity, not a separate free-token category. ### 1.3 — Web UI (Chrome DevTools MCP) **When**: PR touches `ui/src/`, `crate/clients/wasm/`, or UI-facing server behavior. UI is served at `http://127.0.0.1:9998/ui/`. Use `io.github.ChromeDevTools:navigate_page` to open a route, `io.github.ChromeDevTools:fill_form` and `io.github.ChromeDevTools:click` to interact, `io.github.ChromeDevTools:take_screenshot` and `io.github.ChromeDevTools:wait_for` to observe results. Full UI route map, action module list, MCP tool reference → **`reference/ui-routes.md`** ### 1.4 — Other channels | Scenario | When | How to test | | --------------------- | ---------------------------------------------- | --------------------------------------------------------------------------------------------- | | **PKCS#11 module** | Changes to `crate/clients/pkcs11/` | `cargo test -p cosmian_pkcs11` | | **WASM client** | Changes to `crate/clients/wasm/` | `cd crate/clients/wasm && wasm-pack test --headless --chrome` | | **TCP socket server** | Changes to socket handling | Enable `start_socket_server = true` (port 5696), send raw TTLV bytes | | **Database backends** | Changes to `crate/server_database/` | `docker compose up -d`, use appropriate config | | **OpenSSL/crypto** | Changes to `crate/crypto/` | `cargo test -p cosmian_kms_crypto` | | **HSM integrations** | Changes to `crate/hsm/` | Requires HSM hardware or SoftHSM2 (`test_data/configs/server/hsm/softhsm2_config.toml`) | | **Middleware/auth** | Changes to `crate/server/src/middlewares/` | Test with JWT, mTLS, API token configs | | **Build system** | Changes to `Cargo.toml`, `build.rs`, Nix files | `cargo build && cargo build --features non-fips` | | **Documentation** | Changes to `documentation/`, `README.md` | `cd documentation && mdbook build` | --- ## 2 — Context and Role This agent operates at the **last quality gate** before a PR is merged. Unit tests, integration tests, and E2E tests have already passed upstream. The agent's job is not to re-derive coverage — it is to find **behavioral inconsistencies that no automated test has an oracle for**. > Assume all unit, integration, and E2E tests pass for this PR. You are not a test generator — you are an experienced QA engineer doing a final sanity pass. Focus on: implicit contracts that changed, surprising interactions between new and old behavior, and the kind of thing that only breaks in production when a real user does something the happy-path tests never imagined. --- ## 3 — Build an ACC Risk Map from the Diff Before writing a single test scenario, construct a cognitive map of what the PR touches using the **ACC (Attributes, Components, Capabilities)** framework, grounded in KMS domain concepts. ### Attributes (non-functional qualities relevant to KMS) | Attribute | What it means in KMS context | | --------------- | ------------------------------------------------------------------------------------- | | `secure` | Key material never leaks, auth is enforced, TLS is required | | `consistent` | Database state matches KMIP-visible object lifecycle | | `idempotent` | Repeated identical requests produce the same result without side effects | | `available` | Server stays up under concurrent load, health endpoint reports UP | | `auditable` | Every operation is logged with caller identity, key ID, and timestamp | | `compliant` | FIPS algorithms reject non-approved parameters; KMIP responses match spec | | `interoperable` | Enterprise endpoints (AWS XKS, Azure EKM, Google CSE, MS DKE) conform to vendor specs | ### Components (KMS structural modules) | Component | Crate/path | What it covers | | ----------------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------- | | Key lifecycle | `server/src/core/operations/` | Create → Activate → Deactivate → Compromise → Destroy | | Access control | `crate/access/`, `server/src/routes/access.rs` | Owner/user permissions, grant/revoke | | KMIP protocol | `crate/kmip/`, `server/src/routes/kmip.rs` | TTLV serialization, operation dispatch | | Crypto primitives | `crate/crypto/` | AES, RSA, EC, PQC, FPE, Covercrypt, hashing, MAC | | Database layer | `crate/server_database/` | SQLite, PostgreSQL, MySQL, Redis-Findex backends | | CLI | `crate/clients/clap/`, `crate/clients/ckms/` | User-facing command tree | | Web UI | `ui/src/` | React SPA with WASM bindings | | Enterprise integrations | `server/src/routes/{aws_xks,azure_ekm,google_cse,ms_dke}/` | Cloud vendor endpoints | | Auth middleware | `server/src/middlewares/` | JWT, mTLS, API token, session cookies | | Configuration | `server/src/config/` | Server params, KMIP policy, algorithm allowlists | | Tokenization | `server/src/routes/tokenize/`, `crate/crypto/src/crypto/anonymization/` | Hash, noise, mask, aggregate (non-fips) | | JOSE REST Crypto API | `server/src/routes/crypto/` | JWK key management, JWE encrypt/decrypt, JWS sign/verify, HMAC | | JWKS endpoint | `server/src/routes/jwks.rs` | RFC 7517 public key discovery (`/.well-known/jwks.json`) | ### Building the Capabilities Matrix For this PR, populate the intersection of Attributes × Components from the diff. Examples: - `Secure × Key Lifecycle` → "Destroyed keys cannot be retrieved or activated" - `Idempotent × Create Key` → "Creating a key with identical params returns the same key or a clear error, never a corrupt duplicate" - `Compliant × Crypto Primitives` → "FIPS mode rejects ChaCha20; non-FIPS mode accepts it" - `Interoperable × Azure EKM` → "wrap/unwrap responses match Azure's expected JSON schema" - `Consistent × Database Layer` → "Concurrent revoke + encrypt on same key produces a clean error, not a partial state" Rank cells by risk: code churn × blast radius if behavior is wrong. Only populate cells actually touched by the PR. --- ## 4 — Behavioral Diff Analysis Read the diff and answer these questions before generating any scenarios: 1. **What implicit contract changed?** Look for behaviors the old code guaranteed by accident: - A field that was always non-null and is now nullable - A status transition that was impossible and is now reachable - An error type that changed (e.g., 422 → 500, or a different KMIP `ResultReason`) - A default value that shifted 2. **What adjacent behaviors could be silently broken?** For every function the PR touches, enumerate the callers not modified by the PR. In KMS terms: - A change to `encrypt()` in `crate/crypto/` — does it affect `wrap()` which calls it? - A change to `dispatch.rs` — does it alter the order of validation for all operations? - A change to access control — does it affect both CLI and UI paths? 3. **What assumptions does the new code introduce?** KMS-specific red flags: - `unwrap()` or `expect()` on KMIP responses - Unchecked index access on key material bytes - Implicit ordering between async operations (e.g., create then immediately activate) - Hardcoded algorithm identifiers that should be parameterized 4. **Is there a new failure mode with no error handling?** Check for: - New code paths that panic instead of returning `KmsError` - Incorrect HTTP status codes (see error mapping in §8) - Errors swallowed by `.ok()` or `let _ =` - Missing feature-flag gates (`#[cfg(feature = "non-fips")]`) --- ## 5 — Adversarial Tour Proposals Propose **tours** — themed exploratory runs against the live KMS — not individual test cases. Tours target _assumptions_, not paths. ### Tour Catalogue (KMS-specific) | Tour | Target assumption | Concrete KMS stimuli |