Write tests for Apache Iggy connectors - sinks, sources, transforms, SDK, runtime. Covers BDD unit-test naming, the canonical source state round-trip tests, and the real-infra integration test layout under `core/integration/tests/connectors/` with `testcontainers-modules` + `#[iggy_harness]`. Load when writing or reviewing tests in `core/connectors/` or `core/integration/tests/connectors/`. Use for connector test authoring. NOT for runtime internals or non-connector test patterns.
Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.
Write tests for Apache Iggy connectors - sinks, sources, transforms, SDK, runtime. Covers BDD unit-test naming, the canonical source state round-trip tests, and the real-infra integration test layout under `core/integration/tests/connectors/` with `testcontainers-modules` + `#[iggy_harness]`. Load when writing or reviewing tests in `core/connectors/` or `core/integration/tests/connectors/`. Use for connector test authoring. NOT for runtime internals or non-connector test patterns.
Testing Apache Iggy Connectors
Universal connector rules live in
connectors-overview. Repo-wide testing rules (#[iggy_harness], BDD naming, harness layout) live in AGENTS.md. This skill covers connector-specific test conventions.
You may omit one part when the test is small (given_X_should_Y instead of full given_X_when_Y_should_Z), but stay consistent within a file. Imperative test_foo / does_bar names are not the convention here.
test_config() helper
Every plugin's tests start with a small helper returning a tuned-down version of the production config. New tests extend it. don't construct from scratch each time.
The underlying client lib (sqlx, reqwest, mongodb). Tests would exercise the mock, not the code.
The Iggy server.
The runtime.
If you need a fake backend, use wiremock (a real local HTTP server) - see core/integration/tests/connectors/http/.
Integration tests (real infra via Docker)
The canonical pattern lives in core/integration/tests/connectors/. Plugins that integrate with external services (Postgres, Elasticsearch, Iceberg, MongoDB, InfluxDB, Quickwit, HTTP/wiremock) have a paired fixture + test file there.
Every test container MUST be named with the iggy-test- prefix so a single
docker ps -aqf 'name=^iggy-test-' sweep reaps them (just clean-test-containers,
alias ctc; the test/nextest recipes also reap on exit). New fixtures get
this for free by naming through fixtures::unique_container_name("<svc>"), which
appends a uuid for parallel-safe ephemeral containers. Reuse fixtures use a fixed
iggy-test-<svc> literal instead, because the stable name is what lets later
nextest processes attach to the same container via ReuseDirective::Always. Do
not hand-roll a different prefix - the sweep only finds iggy-test-*.
As of now only elasticsearch and doris share a container. Both have a slow
boot (JVM / Doris FE+BE, tens of seconds), so reusing one across the whole
regression beats spinning a fresh one per test. Every other fixture boots a
fresh, ephemeral container per test - cheap enough that the isolation is worth
more than the reuse.
The #[iggy_harness] proc macro
Each integration test is annotated with this macro from the integration crate. It boots an in-process Iggy server (and optionally the connectors runtime) for the duration of the test, runs the seeds, then injects a &TestHarness and your fixture.
use integration::harness::seeds;
use integration::iggy_harness;
#[iggy_harness(
server(connectors_runtime(config_path = "tests/connectors/postgres/sink.toml")),
seed = seeds::connector_stream
)]asyncfnjson_messages_sink_stores_as_bytea(
harness: &TestHarness,
fixture: PostgresSinkFixture,
) {
letclient = harness.root_client().await.unwrap();
letpool = fixture.create_pool().await.expect("Failed to create pool");
fixture.wait_for_table(&pool, "iggy_messages").await;
// ... send messages, assert on rows
}
Key pieces:
server(connectors_runtime(config_path = ...)) boots the connectors runtime against the given TOML.
seed = seeds::connector_stream runs the named seed to create the test stream/topic.
fixture: PostgresSinkFixture - the test harness calls PostgresSinkFixture::setup() (which spins up the testcontainer) and injects the result.
harness: &TestHarness (defined in core/integration/src/harness/orchestrator/harness.rs) exposes .root_client().await, .connectors_runtime().expect("...").http_url(), etc. - connectors_runtime() returns Option, so .expect() first.
Fixture pattern (TestFixture trait)
Each fixture implements integration::harness::TestFixture. Pattern:
The fixture itself wraps the container and implements TestFixture::setup() to spin it up and inject env vars that the runtime picks up.
Env-var injection via ConfigEnv
The runtime's config structs derive ConfigEnv (configs_derive::ConfigEnv, applied in runtime/src/configs/connectors.rs to SinkConfig, SourceConfig, StreamConsumerConfig, StreamProducerConfig). Fields are addressable by env var with a path-built prefix. Three concrete forms observed in core/integration/tests/connectors/fixtures/postgres/container.rs:
// Plugin-config fields (your sink/source's own config struct):// IGGY_CONNECTORS_<TYPE>_<KEY>_PLUGIN_CONFIG_<FIELD>pubconst ENV_SINK_CONNECTION_STRING: &str =
"IGGY_CONNECTORS_SINK_POSTGRES_PLUGIN_CONFIG_CONNECTION_STRING";
// Indexed nested fields (per-stream entries are a Vec):// IGGY_CONNECTORS_<TYPE>_<KEY>_STREAMS_<INDEX>_<FIELD>pubconst ENV_SINK_STREAMS_0_STREAM: &str =
"IGGY_CONNECTORS_SINK_POSTGRES_STREAMS_0_STREAM";
// Top-level scalar fields on SinkConfig/SourceConfig itself:// IGGY_CONNECTORS_<TYPE>_<KEY>_<FIELD>pubconst ENV_SINK_PATH: &str =
"IGGY_CONNECTORS_SINK_POSTGRES_PATH";
<TYPE> is SINK or SOURCE. <KEY> is the connector key (TOML key field, uppercased). PLUGIN_CONFIG is the literal segment used when reaching into your plugin's nested config. Fields marked #[config_env(skip)] (e.g., transforms, plugin_config as a whole) are NOT env-addressable. primitive leaves marked #[config_env(leaf)] are.
The fixture sets these in setup() before the runtime reads its config. This is how a dynamic container port reaches a static TOML.
Test naming for integration tests
Integration tests use a descriptive declarative style, not the unit-test given_should form. Match the codebase:
Pattern: <subject>_<action>_<observation>. Reads like a sentence describing what the test proves.
Polling vs sleeping
Integration tests poll until an expected condition - never sleep(big_duration). Constants POLL_ATTEMPTS and POLL_INTERVAL_MS defined per-backend mod.rs. Fixture helpers like PostgresSinkFixture::fetch_rows_as and PostgresSinkFixture::wait_for_table encapsulate the pattern.
When to add an integration test
New sink/source plugin → at minimum one happy-path test per supported Schema and per supported plugin mode. Concrete coverage in core/integration/tests/connectors/postgres/: 3 sink tests (json_messages_sink_stores_as_bytea, binary_messages_sink_stores_as_bytea, json_messages_sink_stores_as_jsonb) backed by 3 sink fixtures, plus 5 source fixtures (Json, Jsonb, Bytea, Delete, Mark) and a dedicated restart.rs for state-survival.
Bug fix in a plugin → regression test reproducing the bug.
Behavior that crosses the FFI boundary (state restart, transform chain, schema decode) → integration test, not unit test.
When NOT to add an integration test
Pure logic (query builder, payload converter) - that's a unit test.
Config validation - unit test.
Code paths already covered by an equivalent test for another backend, when the new backend has identical behavior.
Runtime tests live under core/integration/tests/connectors/runtime/ and core/integration/tests/connectors/api/. They exercise the runtime alone (no plugin-specific backend) - error isolation, missing plugin, invalid config, HTTP API contract.
For integration tests, the relevant package is integration. There is no --test connectors target - the integration crate has no [[test]] entries and tests/ is structured as a single binary entry through tests/mod.rs with connectors/, server/, cli/, etc. as nested modules. Run via name filter:
cargo test -p integration -- connectors::postgres::postgres_sink
cargo test -p integration -- connectors::postgres:: # all postgres tests
cargo test -p integration -- connectors:: # all connector tests
Integration tests assume Docker is available locally and pull images from a public registry on first run.
Common pitfalls
Skipping integration tests for a real-infra plugin. Unit tests don't catch FFI/lifecycle/encoding regressions.
Forgetting the four canonical source state tests. This is the most common review comment.
Mocking the client library instead of using testcontainers - test exercises the mock.
#[tokio::test] in src/lib.rs - pulls tokio test macros into prod build. Use Runtime::new() instead. (#[tokio::test] is fine in tests/ directories, which compile separately.)
unwrap() in test assertions - prefer assert_eq! / expect("descriptive") so failures tell you which step blew up.
Tests depending on system time - inject the clock or use tokio::time::pause().
Hidden coupling via shared globals (static AtomicU64, env vars) - tests must be independent and parallelizable.
Asserting on Display of an error - use matches!(err, Error::Variant(_)). assert_eq!(err.to_string(), ...) breaks on message tweaks.
Sleeping a fixed duration instead of polling - causes flakes when CI is slow.
