port-moto-tests

name	port-moto-tests
description	Port moto's Python test cases for a given AWS service to winterbaume Rust integration tests. Reads moto tests from the moto GitHub repository, translates to aws-sdk-rust + MockAws style, and appends to existing integration_test.rs.
argument-hint	<service-name> [test-file-filter...]
user_invocable	true

Port Moto Tests

Read moto's Python test suite for a given AWS service and translate the test cases into winterbaume Rust integration tests, appending them to the service crate's tests/integration_test.rs.

Arguments

• $0 — Service name matching the winterbaume crate suffix (e.g., ecs, dynamodb, s3, kms)
• $1... — (optional) Filter to specific moto test file(s) (e.g., test_ecs.py, test_s3_lifecycle.py). If omitted, port from the main test file (test_{moto_name}.py).

---

Step 0: Resolve service names and locate files

0a. Winterbaume crate

ls crates/winterbaume-{service}/tests/integration_test.rs

If the crate or test file doesn't exist, abort — the service must already be implemented via the add-service skill first.

0b. Moto test directory

The moto test directory is tests/test_{moto_name}/ in the moto repository. The moto name usually matches the crate suffix but there are exceptions. Use this mapping:

Crate suffix	Moto test dir
cognitoidp	test_cognitoidp
cognitoidentity	test_cognitoidentity
config	test_config
elbv2	test_elbv2
events	test_events
logs	test_logs
secretsmanager	test_secretsmanager
lambda	test_awslambda
organizations	test_organizations
route53	test_route53
costexplorer	test_ce
dynamodbstreams	test_dynamodb (subset)

For anything not listed, try test_{service} first. If not found:

Use WebFetch to browse https://github.com/getmoto/moto/tree/master/tests and search for the service name.

0c. List moto test files

Use WebFetch to list files at https://github.com/getmoto/moto/tree/master/tests/test_{moto_name}/

If the user specified file filters, use those. Otherwise select the main test file (usually the largest test_{moto_name}.py).

---

Step 1: Inventory existing winterbaume tests

Read the existing integration test file to identify:

1. Already-ported test function names — collect all async fn test_* names
2. Implemented operations — which SDK operations are exercised
3. Helper functions — make_client(), resource setup helpers, constants like ACCOUNT_ID
4. Import patterns — which SDK types are imported

grep 'async fn test_' crates/winterbaume-{service}/tests/integration_test.rs

Also read the handlers to know which operations are actually implemented:

grep -E '(handle_|=> self\.)' crates/winterbaume-{service}/src/handlers.rs

Build a set of implemented operations — only port tests for operations that winterbaume actually handles.

---

Step 1b: Scan for handler fixes from E2E test sessions

Other agent sessions running terraform E2E tests (write-e2e-tests skill) discover and fix handler bugs, marking each fix with a // FIX(terraform-e2e): comment. These fixes reveal behaviors that existing tests didn't cover. Scan for them to ensure the ported test suite covers these fixes too.

1b-i. Find all fix markers in the service crate

grep -rn "FIX(terraform-e2e)" crates/winterbaume-{service}/src/

1b-ii. For each fix, check if moto tests already cover it

Cross-reference each fix with the moto test suite. If moto has a test exercising the same behaviour (e.g., testing that ContentBasedDeduplication is stored and returned for FIFO queues), prioritise porting that test.

1b-iii. If no moto test covers the fix, write a targeted test

When a FIX(terraform-e2e) marker addresses a behaviour that moto doesn't test (because it's terraform-specific, like response shape requirements), write a new test specifically for the fixed behaviour. Common patterns:

Fix category	Test to add
Missing response field	Call Get/Describe, assert the field is present with the correct default value
Missing default attribute	Create a resource, call Get/Describe, assert defaults are populated
Non-deterministic response	Call Describe twice, assert the field value is identical
New operation added	Call the operation, verify success and that it persists
Attribute parsing on create	Create with the attribute set, Get and verify it matches input
ARN normalization	Call with variant ARN forms, verify correct behaviour
Conditional response shape	Create with alternate config, Describe and assert expected fields

Mark these tests with a comment:

// Covers FIX(terraform-e2e): {brief description of the fix}
#[tokio::test]
async fn test_{fix_scenario}() { ... }

1b-iv. Append fix-coverage tests to the report

In the JOURNAL.md entry (Step 6), add a sub-section:

- FIX(terraform-e2e) coverage: K tests added for handler fixes (list scenarios)

---

Step 2: Read and analyse moto tests

Read the target moto test file(s). For each def test_* function, extract:

1. Test name — e.g., test_create_cluster
2. Operations called — e.g., client.create_cluster(...), client.describe_clusters(...)
3. Setup dependencies — which other operations must succeed first
4. Assertions — response field checks, error expectations
5. Skip conditions — @pytest.mark.parametrize, if settings.TEST_SERVER_MODE, SkipTest → skip these

Classification

For each moto test, classify:

• SKIP: Already ported (test name matches or equivalent logic exists in winterbaume tests)
• SKIP: Uses operations not implemented in winterbaume
• SKIP: Tests moto-specific internals (moto_api, state_manager, CloudFormation integration)
• SKIP: Tests server-mode-only features
• PORT: All operations used are implemented in winterbaume, and no equivalent test exists

Ordering

Port tests in dependency order:

1. Basic CRUD (create, describe/get, list, delete)
2. Update/modify operations
3. Error cases (not-found, already-exists, validation)
4. Tag operations
5. Complex scenarios (multi-resource interactions)

---

Step 3: Translate Python tests to Rust

For each test marked PORT, translate using these patterns:

3a. Test function skeleton

// Ported from moto: test_{moto_name}.py::test_function_name
#[tokio::test]
async fn test_function_name() {
    let client = make_client().await;
    // ... translated logic ...
}

3b. Python-to-Rust translation rules

Python (boto3)	Rust (aws-sdk)
client = boto3.client("svc", region_name="us-east-1")	let client = make_client().await; (reuse existing helper)
client.create_thing(Name="foo")	client.create_thing().name("foo").send().await.unwrap()
response["Thing"]["Name"]	response.thing().unwrap().name().unwrap() or response.thing().unwrap().name()
assert response["X"] == "Y"	assert_eq!(response.x(), Some("Y")) or assert_eq!(response.x(), "Y")
with pytest.raises(ClientError) as exc:	let err = client.op().send().await.unwrap_err();
exc.value.response["Error"]["Code"] == "NotFound"	Check error variant: assert!(format!("{:?}", err).contains("NotFound")) or use .into_service_error()
len(response["Items"])	response.items().len()
response["Items"][0]["Name"]	response.items()[0].name().unwrap()
assert ... is None	assert!(response.field().is_none())
datetime comparisons	Generally skip exact timestamp assertions; assert is_some() instead
uuid.UUID(response["Id"])	assert!(!response.id().unwrap().is_empty()) (verify non-empty)
@pytest.mark.parametrize(...)	Separate test functions, or a helper called with different args
for item in items: / list comprehension	.iter(), .map(), .collect::<Vec<_>>()

3c. SDK type nuances

• Required string fields: response.name() returns &str — compare with assert_eq!(r.name(), "expected")
• Optional string fields: response.name() returns Option<&str> — compare with assert_eq!(r.name(), Some("expected"))
• Enum fields: Use aws_sdk_xxx::types::EnumType::Variant — e.g., assert_eq!(status, &aws_sdk_ecs::types::ClusterStatus::Active) . Tip: it's often easier to compare the string form, since winterbaume may return raw strings
• Integer fields: Usually i32 or i64, not Option — assert_eq!(r.count(), 5)
• Blob/Bytes fields: r.data().unwrap().as_ref() to get &[u8]. Input blob bodies use ByteStream::from_static(...) ( or Blob::new(...) for non-streaming blobs ).
• @httpPayload blob bodies are opaque bytes: if the service has a blob @httpPayload operation ( upload / invoke / put-object body ) and the moto suite only exercises it with text, add a targeted test that sends a non-UTF-8 body ( e.g. gzip magic &[0x1f, 0x8b, …, 0xff, 0xfe] ) and asserts acceptance — this guards the wire deserialiser against UTF-8-validating an opaque body ( issue #12 ). See the write-tests skill's "Binary @httpPayload body" row for framing ( wire-opacity, not "AWS accepts arbitrary binary" for format-constrained ops ).
• Nested builders: Use .builder()...build().unwrap() for complex input types

3d. Error assertions

For error path tests, the simplest approach:

let err = client
    .some_operation()
    .some_field("bad-input")
    .send()
    .await
    .unwrap_err();

// Check error type through display/debug format:
let err_str = format!("{:?}", err);
assert!(
    err_str.contains("ResourceNotFoundException") || err_str.contains("NotFound"),
    "Expected not-found error, got: {err_str}"
);

Or if the SDK provides typed errors:

let service_err = err.into_service_error();
assert!(service_err.is_resource_not_found_exception());

3e. Setup helpers

If multiple tests need the same resource setup (e.g., "create a table before testing put_item"), create or reuse helper functions:

async fn create_test_cluster(client: &aws_sdk_ecs::Client) -> String {
    let resp = client
        .create_cluster()
        .cluster_name("test-cluster")
        .send()
        .await
        .unwrap();
    resp.cluster().unwrap().cluster_arn().unwrap().to_string()
}

Always check if an equivalent helper already exists in the test file before adding a new one.

3f. What to skip in translation

• Moto-internal assertions: state_manager, moto_api._internal, mock reset
• CloudFormation integration tests: test_*_cloudformation — these test CFN stack behaviour
• Server-mode tests: if settings.TEST_SERVER_MODE / @mock_aws(config={"core": {"mock_credentials": False}})
• Pagination tokens: If winterbaume doesn't implement pagination, skip pagination-specific tests. If it does, port them.
• Account/credential-specific behaviour: mock.patch.dict(os.environ, ...) — simplify or skip
• Exact timestamp assertions: Replace with is_some() checks
• File I/O tests: Tests that read/write actual files — adapt to use in-memory data

---

Step 4: Append tests to integration_test.rs

4a. Add section header

Group ported tests under a clear comment block:

// ============================================================================
// Ported from moto: test_{moto_name}.py
// ============================================================================

4b. Add necessary imports

Check if new imports are needed (new SDK types, etc.) and add them at the top of the file. Merge with existing imports — don't duplicate.

4c. Append test functions

Append the translated tests after the existing tests. Maintain the existing code style:

• Same indentation (4 spaces)
• Same assertion style (assert_eq! / assert!)
• Same error handling style (.unwrap() vs .expect("message"))
• Same comment style

4c-ii. Append FIX(terraform-e2e) coverage tests

If Step 1b found fix markers with no corresponding moto test coverage, append the targeted tests under a separate section header:

// ============================================================================
// Coverage for FIX(terraform-e2e) handler fixes
// ============================================================================

4d. Add helper functions

If new setup helpers are needed, add them near the existing helpers (after make_client(), before test functions), not at the bottom.

---

Prior Art and Lessons Learned

This skill continues work documented in JOURNAL.md (2026-03-22). Key findings from porting 816 tests across 32 services:

Behavioral fixes are the main value

Many "working" handlers had subtly wrong behaviour that smoke tests couldn't catch. When ported moto tests fail, fix the winterbaume handler/state code to match moto's behaviour rather than weakening assertions. Common categories:

1. Wrong defaults — ACM status ISSUED vs PENDING_VALIDATION; ECS container cpu default 0 not 256; OpenSearch engine 2.5 not 2.11
2. Missing response fields — Lambda missing Architectures/EphemeralStorage/TracingConfig; KMS missing EncryptionAlgorithms
3. Idempotency semantics — SES/SNS/StepFunctions/CodeBuild delete operations should be idempotent (no error on missing resource)
4. Error code/message mismatches — IAM EntityAlreadyExists message format; MemoryDB type-specific fault names
5. Missing validation — CodeBuild name regex/length; SES sender identity; Secrets Manager recovery window range
6. Data computation — Lambda CodeSha256 must be real SHA-256; SQS message MD5 must match

Moto tests are the authoritative behavioural spec

They encode AWS-specific quirks (validation patterns, default values, idempotency) that can't be derived from the Smithy model. Prefer fixing winterbaume to match moto over adjusting assertions.

Wire module serialisers

New handlers should use wire::serialize_*_response() from the generated wire module rather than hand-building JSON/XML. Check existing wire.rs for available serialisers and regenerate if needed.

---

Step 5: Build, lint, and test

./.agents/bin/cargo.sh build -p winterbaume-{service}
./.agents/bin/cargo.sh clippy -p winterbaume-{service} --all-targets --all-features -- -D warnings
./.agents/bin/cargo.sh fmt -p winterbaume-{service} -- --check
./.agents/bin/cargo.sh test -p winterbaume-{service} -- --test-threads=1

Clippy and fmt --check are mandatory before reporting; if they fail, fix the violations and re-run. New test code is the most common source of unnecessary_sort_by, collapsible_match, and similar lints — running clippy here prevents those from reaching CI.

--maxfail is pytest, not cargo. --maxfail is a pytest flag, not a Rust libtest flag. For Rust crate verification, use focused filters ( cargo test <pattern> ), --test-threads=N, --no-fail-fast, and per-crate gates instead. Do not pass --maxfail to cargo test.

If tests fail:

1. Compilation errors — Fix type mismatches, missing imports, wrong method names. Check the SDK docs by reading the generated model types in the aws-sdk crate.
2. Assertion failures — The winterbaume implementation may differ from moto's behaviour. Possible fixes:
   • Adjust the assertion to match winterbaume's current behaviour (if it's a valid alternative)
   • Fix the winterbaume handler/state to match the expected AWS behaviour
   • Add a // TODO: winterbaume doesn't yet support X comment and weaken the assertion
3. 501 Not Implemented — The test uses an operation that's a stub. Remove the test or mark it with a // TODO and #[ignore].

Run the tests iteratively — fix one test at a time:

cargo test -p winterbaume-{service} -- test_function_name

---

Step 6: Report

After all tests pass, summarize:

1. Tests ported: Count and list of new test functions
2. Tests skipped: Count and brief reason for each skip category
3. Issues found: Any bugs discovered in winterbaume during porting (with or without fixes)
4. Missing operations: Operations that moto tests exercise but winterbaume hasn't implemented

Append to JOURNAL.md

Add a brief entry to .agents/docs/JOURNAL.md:

### Port moto tests: {service} (YYYY-MM-DD)

Ported N tests from moto's `tests/test_{moto_name}/test_{file}.py`.
- Tests added: N (list key scenarios)
- Tests skipped: M (operations not implemented / moto-internal / already covered)
- FIX(terraform-e2e) coverage: K tests added for handler fixes (list scenarios)
- Issues: (any bugs found and fixed)

---

Reference: Service Name Mismatches

Common cases where the winterbaume crate name, moto test directory name, and SDK crate name don't trivially match:

winterbaume crate	moto test dir	aws-sdk crate	Notes
winterbaume-lambda	test_awslambda	aws-sdk-lambda	moto uses awslambda to avoid Python keyword
winterbaume-costexplorer	test_ce	aws-sdk-costexplorer	moto uses ce
winterbaume-cognitoidentityprovider	test_cognitoidp	aws-sdk-cognitoidentityprovider	moto uses cognitoidp
winterbaume-elasticloadbalancingv2	test_elbv2	aws-sdk-elasticloadbalancingv2	moto uses elbv2
winterbaume-eventbridge	test_events	aws-sdk-eventbridge	moto uses events
winterbaume-cloudwatchlogs	test_logs	aws-sdk-cloudwatchlogs	moto uses logs
winterbaume-config	test_config	aws-sdk-configservice	SDK uses configservice
winterbaume-directory	test_ds	aws-sdk-directoryservice	moto uses ds
winterbaume-route53	test_route53	aws-sdk-route53	Matching
winterbaume-secretsmanager	test_secretsmanager	aws-sdk-secretsmanager	Matching