scout-api-testing

# Scout API Testing ## Core rules (API) - API specs live in `<module-root>/test/scout*/api/{tests,parallel_tests}/**/*.spec.ts` (examples: `test/scout/api/...`, `test/scout_uiam_local/api/...`). - Use the Scout package that matches the module root: - `src/platform/**` or `x-pack/platform/**` -> `@kbn/scout` - `x-pack/solutions/observability/**` -> `@kbn/scout-oblt` - `x-pack/solutions/search/**` -> `@kbn/scout-search` - `x-pack/solutions/security/**` -> `@kbn/scout-security` - Prefer a single top-level `apiTest.describe(...)` per file and avoid nested `describe` blocks; multiple top-level `describe`s are supported, but files get hard to read quickly. - Tags: add `{ tag: ... }` on the suite (or individual tests) so CI/discovery can select the right test target. For **solution** modules, prefer explicit targets (e.g. `[...tags.stateful.classic, ...tags.serverless.observability.complete]` in Observability); reserve `tags.deploymentAgnostic` mainly for **platform** specs that truly need every deployment-agnostic target (see `scout-migrate-from-ftr`). Unlike UI tests, API tests don’t currently validate tags at runtime. - **No `@` in test titles**: Playwright treats `@word` in test/describe titles as tags. Do not use `@` followed by word characters in titles (e.g., `@timestamp`, `@elastic`). Rephrase the title instead (e.g., use `timestamp field` instead of `@timestamp`). - If the module provides Scout fixtures, import `apiTest` from `<module-root>/test/scout*/api/fixtures` to get module-specific extensions. Importing directly from the module’s Scout package is also fine when you don’t need extensions. - Browser fixtures are disabled for `apiTest` (no `page`, `browserAuth`, `pageObjects`). ## Imports - Test framework + tags: `import { apiTest, tags } from '@kbn/scout';` (or the module's Scout package, e.g. `@kbn/scout-oblt`) - Assertions: `import { expect } from '@kbn/scout/api';` (or `@kbn/scout-oblt/api`, etc.) — **not** from the main entry - Types: `import type { RoleApiCredentials } from '@kbn/scout';` - `expect` is **not** exported from the main `@kbn/scout` entry. Use the `/api` subpath for API tests. ## Auth: pick based on endpoint - `api/*` endpoints: use **API keys** via `requestAuth` (`getApiKey`, `getApiKeyForCustomRole`). - `internal/*` endpoints: use **cookies** via `samlAuth.asInteractiveUser(...)`. ## Recommended test shape 1. **Prepare** environment (optional): `apiServices`/`kbnClient`/`esArchiver` in `beforeAll`. 2. **Authenticate** (least privilege): generate credentials in `beforeAll` and reuse. 3. **Request**: call the endpoint with `apiClient` and the right headers. 4. **Assert**: verify `statusCode` and response body; verify side effects via `apiServices`/`kbnClient` when needed. Important: `apiServices`/`kbnClient` run with elevated privileges. Don’t use them to validate the endpoint under test (use `apiClient` + scoped auth). Header reminders: - State-changing requests usually need `kbn-xsrf`. - Prefer sending `x-elastic-internal-origin: kibana` for Kibana APIs. - Include `elastic-api-version` for versioned public APIs (e.g. `'2023-10-31'`) or internal APIs (e.g. `'1'`). ## Assertions - `apiClient` methods (`get`, `post`, `put`, `delete`, `patch`, `head`) return `{ statusCode, body, headers }`. - Use the custom matchers from `@kbn/scout/api`: - `expect(response).toHaveStatusCode(200)` - `expect(response).toHaveStatusText('OK')` - `expect(response).toHaveHeaders({ 'content-type': 'application/json' })` - Standard matchers (`toBe`, `toStrictEqual`, `toMatchObject`, etc.) and asymmetric matchers (`expect.objectContaining(...)`, `expect.any(String)`) are also available. ## API services - Put reusable server-side helpers behind `apiServices` (no UI interactions). Use it for setup/teardown and verifying side effects, not for RBAC validation. - **Module-local service**: create it under `<module-root>/test/scout*/api/services/<service>_api_service.ts` (or similar). Register it by extending the module's `apiServices` fixture in `<module-root>/test/scout*/api/fixtures/index.ts` (prefer `{ scope: 'worker' }` when the helper doesn't need per-test state). - **Shared service** (reused across modules): consider contributing it to the Scout packages under `src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/`. ## Extending fixtures When tests need custom auth helpers or API services, extend `apiTest` in the module's `fixtures/index.ts`: ```ts import { apiTest as base } from '@kbn/scout'; // or the module's Scout package import type { RequestAuthFixture } from '@kbn/scout'; interface MyApiFixtures { requestAuth: RequestAuthFixture & { getMyPluginApiKey: () => Promise<RoleApiCredentials> }; } export const apiTest = base.extend<MyApiFixtures>({ requestAuth: async ({ requestAuth }, use) => { const getMyPluginApiKey = async () => requestAuth.getApiKeyForCustomRole({ kibana: [{ base: [], feature: { myPlugin: ['all'] }, spaces: ['*'] }], }); await use({ ...requestAuth, getMyPluginApiKey }); }, }); ``` Tests then import `apiTest` from the local fixtures: `import { apiTest } from '../fixtures';` ## Parallelism - Treat Scout API tests as sequential by default. Parallel API runs require manual isolation (spaces, indices, saved objects) and are uncommon. ## Run / debug quickly - Use either `--config` or `--testFiles` (they are mutually exclusive). - Run by config: `node scripts/scout.js run-tests --arch stateful --domain classic --config <module-root>/test/scout*/api/playwright.config.ts` (or `.../api/parallel.playwright.config.ts` for parallel API runs) - Run by file/dir (Scout derives the right `playwright.config.ts` vs `parallel.playwright.config.ts`): `node scripts/scout.js run-tests --arch stateful --domain classic --testFiles <module-root>/test/scout*/api/tests/my.spec.ts` - For faster iteration, start servers once in another terminal: `node scripts/scout.js start-server --arch stateful --domain classic [--serverConfigSet <configSet>]`, then run Playwright directly: `node scripts/playwright test --config <...> --project local --grep <tag>`. - `run-tests` auto-detects custom config sets from `.../test/scout_<name>/...` paths. - `start-server` has no Playwright config to inspect, so pass `--serverConfigSet <name>` when your tests require a custom config set. - Debug: `SCOUT_LOG_LEVEL=debug` ## CI enablement - Scout tests run in CI only for modules listed under `plugins.enabled` / `packages.enabled` in `.buildkite/scout_ci_config.yml`. - `node scripts/scout.js generate` registers the module under `enabled` so the new configs run in CI. ## References Open only what you need: - requestAuth vs samlAuth, headers, and least-privilege auth tips: `references/scout-api-auth.md` - Creating and registering `apiServices` helpers (kbnClient + retries + logging): `references/scout-api-services.md`