Some failure modes are invisible at Tier 1 unit and Tier 2 integration because each step in a multi-step pipeline works in isolation — the break is at the seam between steps. A pipeline like km.train(df) → km.register(result, name=...) can have 100% unit coverage on km.train AND 100% integration coverage on km.register while the chain fails because result is missing a field the next step needs (the "fake-integration" failure mode).
Defense: A release-blocking regression tier above Tier 3 — a test that executes the full pipeline against real infrastructure AND asserts a deterministic fingerprint over the output. Any change that alters observable behavior flips the fingerprint and blocks release.
Pattern
# DO — release-blocking regression with pinned SHA-256 fingerprint@pytest.mark.regression@pytest.mark.release_blockingasyncdeftest_readme_quick_start_end_to_end(real_conn):
# 1. Execute the README Quick Start verbatim
result = await km.train(df, target="churned")
registered = await km.register(result, name="demo")
# 2. Compute fingerprint over deterministic output
fingerprint = hashlib.sha256(
json.dumps(registered.artifact_uris, sort_keys=True).encode()
).hexdigest()
# 3. Assert against pinned value (pinned in spec)assert fingerprint == "c962060cf467cc732df355ec9e1212cfb0d7534a3eed4480b511adad5a9ceb00"# DO NOT — rely on unit + integration alonedeftest_km_train_unit(): ... # passes, km.train works in isolationdeftest_km_register_integration(): ... # passes, km.register works given a valid result# Chain still breaks because result is missing `.trainable` back-reference.
When to apply
Public multi-step pipelines documented in README / spec Quick Start sections
Any chain where A() → B(A's output) → C(B's output) is the user's primary ergonomic
