middleware-testing-patterns

# Middleware Testing Patterns <default_to_action> When testing enterprise middleware, ESBs, or message-driven systems: 1. VALIDATE message routing logic (content-based, header-based, recipient list) 2. TEST message transformations end-to-end (input format -> output format) 3. VERIFY dead letter queue handling (poison messages, retry exhaustion) 4. ASSERT message ordering and sequencing with correlation IDs 5. EXERCISE error handling and compensation patterns 6. TEST protocol mediation (SOAP-to-REST, sync-to-async) 7. VALIDATE EIP patterns (splitter, aggregator, content enricher, normalizer) **Quick Pattern Selection:** - Message routing issues -> Content-based router tests - Transformation failures -> Schema-in/schema-out validation - Lost messages -> DLQ and retry pattern tests - Protocol bridging -> Mediation round-trip tests - Complex flows -> Correlation ID tracing tests **Critical Success Factors:** - Middleware is invisible when it works; test the invisible - Always validate both the happy path AND the error channel (DLQ) - Correlation IDs are your lifeline for tracing multi-hop messages </default_to_action> ## Quick Reference Card ### When to Use - Testing ESB message flows (IBM IIB/ACE, MuleSoft, WSO2) - Validating message transformations (XSLT, JSON-to-XML, flat-file parsing) - Testing message broker routing (MQ, Kafka, RabbitMQ) - Verifying dead letter queue behavior - Testing protocol mediation (SOAP/REST bridging) - Validating Enterprise Integration Patterns (EIP) ### Testing Levels | Level | Purpose | Dependencies | Speed | |-------|---------|--------------|-------| | Unit Transform | Single mapping correctness | None | Fast | | Route Logic | Routing decision accuracy | Mocked endpoints | Fast | | Integration | End-to-end message flow | Broker + endpoints | Medium | | DLQ/Error | Error handling and recovery | Full middleware stack | Slower | ### Critical Test Scenarios | Scenario | Must Test | Example | |----------|----------|---------| | Routing | Correct destination selection | Order type A -> Queue A, type B -> Queue B | | Transformation | Schema compliance after mapping | XML -> JSON field mapping accuracy | | DLQ | Poison message handling | Malformed XML lands in DLQ, not lost | | Ordering | Sequence preservation | Messages 1-2-3 arrive in order | | Correlation | Multi-hop tracing | Request-reply matched by correlation ID | | Retry | Transient failure recovery | 3 retries with backoff, then DLQ | | Mediation | Protocol bridging fidelity | SOAP request produces correct REST call | ### Tools - **Message Brokers**: IBM MQ, RabbitMQ, Apache Kafka, ActiveMQ - **ESBs**: IBM IIB/ACE, MuleSoft, WSO2, Apache Camel - **Testing**: SoapUI, Postman, custom harnesses - **Virtualization**: Mountebank, WireMock, HoverFly - **Monitoring**: Splunk, ELK, Datadog ### Agent Coordination - `qe-middleware-validator`: Validate routing rules, transformation accuracy, EIP patterns - `qe-message-broker-tester`: Test broker connectivity, DLQ behavior, message ordering - `qe-soap-tester`: SOAP/WSDL validation, WS-Security, protocol mediation --- ## Message Routing Pattern Testing ### Content-Based Router ```javascript describe('Content-Based Router - Order Type', () => { it('routes standard orders to fulfillment queue', async () => { const message = { correlationId: uuid(), body: { orderType: 'STANDARD', orderId: 'ORD-001', items: [{ sku: 'A1', qty: 2 }] } }; await broker.publish('orders.inbound', message); const routed = await broker.consume('orders.fulfillment', { timeout: 5000 }); expect(routed.body.orderId).toBe('ORD-001'); expect(routed.correlationId).toBe(message.correlationId); const dlq = await broker.tryConsume('orders.dlq', { timeout: 1000 }); expect(dlq).toBeNull(); // Nothing in DLQ }); it('routes express orders to priority queue', async () => { const message = { correlationId: uuid(), body: { orderType: 'EXPRESS', orderId: 'ORD-002', items: [{ sku: 'B1', qty: 1 }] } }; await broker.publish('orders.inbound', message); const routed = await broker.consume('orders.priority', { timeout: 5000 }); expect(routed.body.orderId).toBe('ORD-002'); }); it('sends unrecognized order types to DLQ', async () => { const message = { correlationId: uuid(), body: { orderType: 'UNKNOWN', orderId: 'ORD-003' } }; await broker.publish('orders.inbound', message); const dlqMessage = await broker.consume('orders.dlq', { timeout: 5000 }); expect(dlqMessage.body.orderId).toBe('ORD-003'); expect(dlqMessage.headers['x-error-reason']).toContain('Unrecognized orderType'); }); }); ``` ### Header-Based Router ```javascript describe('Header-Based Router - Region', () => { it('routes by x-region header to regional queues', async () => { const regions = ['US', 'EU', 'APAC']; for (const region of regions) { const message = { headers: { 'x-region': region }, body: { orderId: `ORD-${region}` } }; await broker.publish('orders.global', message); const routed = await broker.consume(`orders.${region.toLowerCase()}`, { timeout: 5000 }); expect(routed.body.orderId).toBe(`ORD-${region}`); } }); it('routes messages without region header to default queue', async () => { await broker.publish('orders.global', { body: { orderId: 'ORD-NO-REGION' } }); const routed = await broker.consume('orders.default', { timeout: 5000 }); expect(routed.body.orderId).toBe('ORD-NO-REGION'); }); }); ``` ### Recipient List ```javascript describe('Recipient List - Order Notifications', () => { it('fans out order to all registered recipients', async () => { const message = { body: { orderId: 'ORD-100', total: 250.00 } }; await broker.publish('orders.confirmed', message); // All three systems should receive the message const [warehouse, billing, crm] = await Promise.all([ broker.consume('notify.warehouse', { timeout: 5000 }), broker.consume('notify.billing', { timeout: 5000 }), broker.consume('notify.crm', { timeout: 5000 }) ]); expect(warehouse.body.orderId).toBe('ORD-100'); expect(billing.body.orderId).toBe('ORD-100'); expect(crm.body.orderId).toBe('ORD-100'); }); }); ``` --- ## Message Transformation Testing ### JSON-to-XML Transformation ```javascript describe('JSON to XML Transformation', () => { it('transforms order JSON to SAP XML format', async () => { const jsonInput = { orderId: 'ORD-500', customer: { id: 'CUST-01', name: 'Acme Corp' }, items: [ { sku: 'MAT-100', quantity: 10, unitPrice: 25.00 } ] }; await broker.publish('transform.json-in', { body: jsonInput }); const xmlOutput = await broker.consume('transform.xml-out', { timeout: 5000 }); const parsed = parseXml(xmlOutput.body); expect(parsed.SalesOrder.OrderNumber).toBe('ORD-500'); expect(parsed.SalesOrder.SoldToParty.CustomerID).toBe('CUST-01'); expect(parsed.SalesOrder.Items.Item[0].MaterialNumber).toBe('MAT-100'); expect(parsed.SalesOrder.Items.Item[0].Quantity).toBe('10'); }); it('handles missing optional fields gracefully', async () => { const jsonInput = { orderId: 'ORD-501', customer: { id: 'CUST-02' }, // name is missing items: [{ sku: 'MAT-200', quantity: 5 }] // unitPrice missing }; await broker.publish('transform.json-in', { body: jsonInput }); const xmlOutput = await broker.consume('transform.xml-out', { timeout: 5000 }); const parsed = parseXml(xmlOutput.body); expect(parsed.SalesOrder.SoldToParty.Name).toBe(''); // Default empty expect(parsed.SalesOrder.Items.Item[0].UnitPrice).toBe('0.00'); }); it('rejects invalid input and sends to DLQ with error details', async () => { const invalidInput = { items: [] }; // Missing required orderId and customer await broker.publish('transform.json-in', { body: invalidInput }); const dlq = await broker.consume('transform.dlq', { timeout: 5000 }); expect(dlq.headers['x-error-reason']).toContain('orderId is required'); }); }); ``` ### Flat-File to XML Transformation ```javascript describe('Flat-File to XML Transformation', () => { it('parses fixed-width flat file into structured XML', async () => { // Fixed-width: positions 1-10 = orderId, 11-20 = sku, 21-25 = qty const flatFile = 'ORD-001 MAT-10000010\nORD-001 MAT-20000005\n'; await broker.publish('transform.flatfile-in', { body: flatFile }); const xmlOutput = await broker.consume('transform.flatfile-xml-out', { timeout: 5000 }); const parsed = parseXml(xmlOutput.body); expect(parsed.Orders.Order).toHaveLength(1); expect(parsed.Orders.Order[0].Items.Item).toHaveLength(2); expect(parsed.Orders.Order[0].Items.Item[0].Quantity).toBe('10'); }); it('handles lines with trailing whitespace correctly', async () => { const flatFile = 'ORD-002 MAT-30000001 \n'; await broker.publish('transform.flatfile-in', { body: flatFile }); const xmlOutput = await broker.consume('transform.flatfile-xml-out', { timeout: 5000 }); const parsed = parseXml(xmlOutput.body); expect(parsed.Orders.Order[0].OrderId).toBe('ORD-002'); expect(parsed.Orders.Order[0].Items.Item[0].Quantity).toBe('1'); }); }); ``` --- ## Dead Letter Queue (DLQ) Testing ```javascript describe('Dead Letter Queue Strategy', () => { it('moves message to DLQ after max retries', async () => { // Configure endpoint to always fail mockEndpoint.respondWith(500, 'Service Unavailable'); await broker.publish('orders.process', { body: { orderId: 'ORD-FAIL' }, headers: { 'x-correlation-id': 'corr-123' } }); // Wait for retries to exhaust (3 retries with backoff) const dlqMessage = await broker.consume('orders.dlq', { timeout: 30000 }); expect(dlqMessage.body.orderId).toBe('ORD-FAIL'); expect(dlqMessage.headers['x-retry-count']).toBe('3'); expect(dlqMessage.headers['x-original-queue']).toBe('orders.process'); expect(dlqMessage.headers['x-correlation-id']).toBe('corr-123'); expect(dlqMessage.headers['x-error-reason']).toContain('Service Unavailable'); expect(dlqMessage.headers['x-first-failure-timestamp']).toBeDefined(); }); it('preserves original message body in DLQ', async () => { const originalBody = { orderId: 'ORD-PRESERVE', items: [{ sku: 'A', qty: 1 }, { sku: 'B', qty: 2 }] }; mockEndpoint.respondWith(500, 'Timeout'); await broker.publish('orders.process', { body: originalBody }); const dlqMessage = await broker.consume('orders.dlq', { timeout: 30000 }); expect(dlqMessage.body).toEqual(originalBody); // Body unchanged });