ci-scenario-authoring

name

ci-scenario-authoring

description

Add or modify CI integration-test scenarios — new persistence layers, scenario entries in the composable registry (test/ci/registry/), features, shortnames, and lifecycle hooks (pre-install fixtures/scripts, post-deploy, pre-upgrade) with the TestLifecycleFixtures contract. Use when adding a new test scenario or persistence backend, wiring pre-install prerequisites like TLS secrets or a CloudNativePG cluster, or adding upgrade-flow cleanup hooks.

Adding New Persistence Layers and Scenarios

New Persistence Layer

A persistence layer is a values file at charts/<version>/test/integration/scenarios/chart-full-setup/values/persistence/<name>.yaml. To add one:

1. Create the YAML file with the values needed for the data backend.

No code change required — the name is discovered automatically from the filesystem.

New CI Test Scenario

Scenarios live in the composable registry charts/<version>/test/ci/registry/ (ci-test-config.yaml only defines the unit-test matrix). The lone exception is 8.6, which predates the registry and has no active CI — its legacy inline ci-test-config.yaml scenarios run only via manual workflow_dispatch.

Adding a scenario takes two files:

1. An entry in test/ci/registry/manifest.yaml under integration.scenarios:

- id: elasticsearch-self-signed      # matches scenarios/<id>.yaml
  shortname: esss                    # 4-char, used in namespace generation
  tier: 2                            # omit for untiered
  enabled: false                     # set true when ready for CI

2. A scenario definition at test/ci/registry/scenarios/<id>.yaml:

name: elasticsearch-self-signed
auth: keycloak
flows: [upgrade-minor]
identity: keycloak
persistence: elasticsearch-self-signed
features: [migrator]                 # includes values/features/migrator.yaml
platforms: [gke]
infra-type:
  gke: distroci
dependencies: [keycloak, elasticsearch]
pre-install: elasticsearch-self-signed   # optional — hook name, see below

E2E legs

The after-install Playwright job is a matrix of legs — one per Playwright project — each with its own blocking behaviour. Every scenario gets a blocking smoke leg; three optional fields change that:

e2e-full-suite: true            # add a second leg running the full-suite project
e2e-smoke-blocking: false       # default true  — smoke failures fail the run
e2e-full-suite-blocking: true   # default false — full-suite failures are informational

The two blocking fields are tri-state: omit them to keep the default, since e2e-smoke-blocking defaults to true and an absent key must not read as false.

test-integration-template.yaml resolves the legs from the registry via deploy-camunda ci e2e-matrix and passes them to test-integration-runner.yaml as the JSON e2e-matrix input, so they apply to scenarios invoked directly by an external caller (the AlwaysGreen gate) as well as to the generated PR matrix. Resolution keys on shortname, not scenario name — several registry files share a name: (8.8 has three name: elasticsearch) and only shortname is unique. The full leg gets the long-running runner and a 60-minute timeout; legs are serialized (max-parallel: 1) because smoke-tests.spec.js matches the full-suite glob too and both legs share one deployment. skip-e2e: true yields no legs at all (a skipped job) and combining it with any of these fields is rejected by the registry validator, as is e2e-full-suite-blocking without e2e-full-suite.

The features array maps to values/features/<name>.yaml. The migrator feature enables identity and data migration jobs during upgrades — use it for any upgrade-minor scenario. Note: the automatic needsMigrator() function in scenarios.go only activates when ChartVersion starts with "13", but the matrix runner does not set ChartVersion, so always use features: [migrator] explicitly.

Pre-Install Hooks (Scenario-Specific)

When a scenario needs prerequisites in the namespace before helm install (e.g., a CloudNativePG cluster, TLS secrets), declare a hook file at test/ci/registry/hooks/<hook-name>.yaml and reference it by name from the scenario file:

# scenarios/rdbms.yaml
pre-install: cnpg

# hooks/cnpg.yaml
fixtures:
  - postgresql-cluster.yaml
description: |
  Provisions a CloudNativePG `Cluster` plus auth Secret in the scenario
  namespace; required before helm install.

There are two modes — pick exactly one per hook:

• fixtures: [...] — names YAML manifests under charts/<version>/test/integration/scenarios/common/resources/. The matrix runner applies them via Go server-side apply (kube.Client.ApplyManifest, Force: true, idempotent), substituting $NAMESPACE, $RELEASE_NAME, plus the env-var passthrough listed in lifecycleVarPassthrough (RDBMS_POSTGRESQL_USERNAME, RDBMS_POSTGRESQL_PASSWORD, GITHUB_WORKFLOW_JOB_ID, POSTGRESQL_JDBC_URL). Prefer this mode for trivial kubectl-apply cases.
• script: <filename> — names a shell script under charts/<version>/test/integration/scenarios/pre-setup-scripts/. The matrix runner runs it via bash -x with TEST_NAMESPACE, KUBE_CONTEXT, and the same env-var passthrough. Use only when the work cannot be expressed as a manifest (cert generation, JKS keystores, conditional kubectl ops). Example: pre-install-elasticsearch-self-signed.sh runs openssl + keytool, packages JKS, and creates three Secrets.

description is required and must explain the effect — reviewers must understand from the hook-file diff alone what a fixture does.

TestLifecycleFixtures (matrix package) walks every chart version's config and asserts: every script reference resolves on disk, every fixture reference resolves under common/resources/, every description is non-empty, exactly one of fixtures/script is set per hook, and every script in pre-setup-scripts/ is referenced (orphan check). Files in preSetupScriptAllowlist (pre-install-upgrade.sh sed-marker, create-elasticsearch-tls-secrets.sh helper) are exempt.

Post-Deploy Hooks (Scenario-Specific)

For resources whose CRDs are only registered by the chart itself (e.g., the Gateway API ProxySettingsPolicy on gateway-keycloak), reference a hook via post-deploy: <hook-name> in the scenario file, same hook-file shape as pre-install. Runs after helm upgrade/install returns successfully and before the deploy result is reported. Example:

# scenarios/gateway-keycloak.yaml
post-deploy: gateway-keycloak

# hooks/gateway-keycloak.yaml
fixtures: [gateway-proxy-settings.yaml]
description: |
  Applies the NGINX ProxySettingsPolicy that bumps gateway buffer sizes.
  Runs after helm install because the Gateway API CRD is only registered
  by the chart itself.

Pre-Upgrade Hooks (Flow-Specific)

For cleanup between Step 1 (old version) and Step 2 (new version) of an upgrade flow, declare on the target version's integration.flows.<flow>.pre-upgrade block in test/ci/registry/manifest.yaml:

integration:
  flows:
    upgrade-patch:
      pre-upgrade:
        script: pre-upgrade-patch.sh
        description: |
          Deletes orchestration StatefulSets and the postgresql-web-modeler
          StatefulSet + PVC before the patch upgrade (PSQL 15→14 rollback).

Same fixtures / script / description shape as scenario-level pre-install:. The hook runs after Step 1 completes and before Step 2's helm upgrade, scoped to the target version (the version being upgraded to).

Verifying a new or changed scenario

Registry snapshot (charts/<v>/test/ci/registry-snapshot.yaml) is the compiled view of the composable registry — use it as a diff target when editing scenarios/hooks/dependencies; regenerate with make go.update-registry-golden (see AGENTS.md → Generated Artifacts). To run the scenario locally before merge, see the rfr-validation skill.