contract-test-framework

name	contract-test-framework
description	Consumer-driven contract testing for SDK-API compatibility. Generate Pact consumer tests, verify provider contracts, configure Pact broker, and implement can-i-deploy checks.
allowed-tools	Bash(*) Read Write Edit Glob Grep WebFetch
metadata	{"author":"babysitter-sdk","version":"1.0.0","category":"sdk-testing","backlog-id":"SK-SDK-003"}

contract-test-framework

You are contract-test-framework - a specialized skill for consumer-driven contract testing between SDKs and APIs, ensuring compatibility and preventing breaking changes through automated verification.

Overview

This skill enables AI-powered contract testing including:

• Generating Pact consumer contracts from SDK usage
• Configuring Pact Broker for contract management
• Provider verification against consumer contracts
• Can-i-deploy safety checks before releases
• Breaking change detection and alerting
• Webhook integration for automated verification
• Support for bidirectional contract testing

Prerequisites

• Node.js 18+ or Python 3.8+
• Pact library for your SDK language
• Pact Broker (PactFlow recommended) or self-hosted
• CI/CD pipeline access
• Consumer SDK and provider API access

Capabilities

1. Consumer Contract Generation for SDKs

Generate contracts from SDK tests:

// tests/contracts/user-api.pact.ts
import { PactV3, MatchersV3 } from '@pact-foundation/pact';
import { MyServiceSDK } from '@company/myservice-sdk';

const { like, eachLike, regex, uuid, datetime, integer } = MatchersV3;

const provider = new PactV3({
  consumer: 'myservice-typescript-sdk',
  provider: 'myservice-api',
  logLevel: 'info'
});

describe('MyService SDK Contracts', () => {
  describe('Users API', () => {
    it('should get user by ID', async () => {
      const expectedUser = {
        id: uuid(),
        email: like('[e-mail]'),
        name: like('John Doe'),
        createdAt: datetime("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"),
        status: regex(/active|inactive|pending/, 'active')
      };

      await provider
        .given('a user with ID exists', { :  })
        .()
        .({
          : ,
          : ,
          : {
            : ,
            : (, )
          }
        })
        .({
          : ,
          : { :  },
          : expectedUser
        });

       provider.( (mockServer) => {
         sdk =  ({
          : mockServer.,
          : 
        });

         user =  sdk..();

        (user).();
        (user.).();
      });
    });

    (,  () => {
       provider
        .()
        .()
        .({
          : ,
          : ,
          : {
            : ,
            : 
          }
        })
        .({
          : ,
          : {
            : ({
              : (),
              : (),
              : ()
            }),
            : {
              : (),
              : (),
              : (),
              : ()
            }
          }
        });

       provider.( (mockServer) => {
         sdk =  ({ : mockServer. });
         response =  sdk..({ : , :  });

        (response.).();
        (response..).();
      });
    });

    (,  () => {
       provider
        .()
        .()
        .({
          : ,
          : ,
          : {
            : ,
            : (, )
          },
          : {
            : (),
            : (),
            : ()
          }
        })
        .({
          : ,
          : {
            : (),
            : (),
            : (),
            : ()
          }
        });

       provider.( (mockServer) => {
         sdk =  ({
          : mockServer.,
          : 
        });

         user =  sdk..({
          : ,
          : ,
          : 
        });

        (user.).();
      });
    });

    (,  () => {
       provider
        .(, { :  })
        .()
        .({
          : ,
          : 
        })
        .({
          : ,
          : {
            : {
              : (),
              : ()
            }
          }
        });

       provider.( (mockServer) => {
         sdk =  ({ : mockServer. });

         (sdk..())
          .
          .();
      });
    });
  });
});

2. Multi-SDK Contract Testing

Test contracts for multiple SDK implementations:

# pact-config.yaml
consumers:
  - name: myservice-typescript-sdk
    language: typescript
    version: ${GIT_COMMIT}
    branch: ${GIT_BRANCH}

  - name: myservice-python-sdk
    language: python
    version: ${GIT_COMMIT}
    branch: ${GIT_BRANCH}

  - name: myservice-java-sdk
    language: java
    version: ${GIT_COMMIT}
    branch: ${GIT_BRANCH}

provider:
  name: myservice-api
  baseUrl: http://localhost:3000

broker:
  url: https://your-broker.pactflow.io
  token: ${PACT_BROKER_TOKEN}
  publishResults: true

verification:
  enablePending: true
  wipPactsSince: '2024-01-01'
  consumerVersionSelectors:
    - matchingBranch: true
    - mainBranch: true
    -  

3. Provider Verification

Verify API against all SDK contracts:

// tests/contracts/provider-verification.ts
import { Verifier } from '@pact-foundation/pact';
import { startServer, stopServer, resetDatabase } from '../test-utils';

describe('Provider Verification', () => {
  beforeAll(async () => {
    await startServer();
  });

  afterAll(async () => {
    await stopServer();
  });

  it('should verify all SDK contracts', async () => {
    const verifier = new Verifier({
      provider: 'myservice-api',
      providerBaseUrl: 'http://localhost:3000',

      // Pact Broker configuration
      pactBrokerUrl: process.env.PACT_BROKER_URL,
      pactBrokerToken: process.env.PACT_BROKER_TOKEN,

      // Provider version
      providerVersion: process.env.GIT_COMMIT || '1.0.0',
      providerVersionBranch: process.env.GIT_BRANCH || 'main',

      // Consumer selection
      consumerVersionSelectors: [
        { :  },
        { :  },
        { :  }
      ],

      
      : {
        :  (params) => {
           ();
           db..({
            : params.,
            : ,
            : 
          });
        },

        :  () => {
           ();
           db..([
            { : , : , :  },
            { : , : , :  }
          ]);
        },

        :  (params) => {
           ();
          
           db..(params.).( {});
        },

        :  () => {
           ();
        }
      },

      
      :  {
        
         (!req..) {
          req.. = ;
        }
        ();
      },

      
      : ,
      : ,
      : 
    });

     verifier.();
  });
});

4. CI/CD Pipeline Integration

Complete GitHub Actions workflow:

name: SDK Contract Testing

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

env:
  PACT_BROKER_URL: https://your-broker.pactflow.io
  PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}

jobs:
  # Consumer SDK contract tests
  sdk-contracts:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        sdk: [typescript, python, java]

    steps:
      - uses: actions/checkout@v4

      - name: Setup SDK environment
        uses: ./.github/actions/setup-${{ matrix.sdk }}

      - name: Install dependencies
        run: |
          cd sdks/${{ matrix.sdk }}
          ${{ matrix.sdk == 'typescript' && 'npm ci' || matrix.sdk == 'python' && 'pip install -e .[dev]' || 'mvn install -DskipTests' }}

      - name: Run  
         
         
           
         
  
  
     
     

    
        
        
           

         
         
        
           

         
          

         
            

          
           

         
           
        
             
             

  
  
     
     [, ]
       

    
      
        
           
           
           
           

    
          
         
  
  
     
     
       

    
      
        
           
           
           
           

    
         
         

5. Webhook Configuration

Set up automated verification webhooks:

# Create webhook for SDK changes
pact-broker create-webhook \
  'https://api.github.com/repos/your-org/myservice-api/dispatches' \
  --request=POST \
  --header 'Accept: application/vnd.github.v3+json' \
  --header 'Authorization: Bearer ${GITHUB_TOKEN}' \
  --data '{
    "event_type": "contract_requiring_verification",
    "client_payload": {
      "pact_url": "${pactbroker.pactUrl}",
      "consumer_name": "${pactbroker.consumerName}",
      "provider_name": "${pactbroker.providerName}"
    }
  }' \
  --description "Trigger API verification on SDK contract change" \
  --contract-content-changed \
  --provider myservice-api \
  --broker-base-url https://your-broker.pactflow.io \
  --broker-token $PACT_BROKER_TOKEN

# Create webhook for verification results
pact-broker create-webhook \
  'https://api.github.com/repos/your-org/myservice-sdk/statuses/${pactbroker.consumerVersionNumber}' \
  --request=POST \
  --header 'Authorization: Bearer ${GITHUB_TOKEN}' \
  --data '{
    "state": "${pactbroker.verificationResultSuccess ? \"success\" : \"failure\"}",
    "description": "Contract verification ${pactbroker.verificationResultSuccess ? \"passed\" : \"failed\"}",
    "context": "pact/provider-verification"
  }' \
  --description "Update SDK commit status on verification" \
  --provider-verification-published \
  --broker-base-url https://your-broker.pactflow.io \
  --broker-token $PACT_BROKER_TOKEN

6. Breaking Change Detection

Detect and handle breaking changes:

// scripts/check-breaking-changes.ts
import { PactBrokerClient } from '@pact-foundation/pact';

async function checkBreakingChanges(
  provider: string,
  newVersion: string
): Promise<BreakingChangeReport> {
  const client = new PactBrokerClient({
    brokerBaseUrl: process.env.PACT_BROKER_URL!,
    token: process.env.PACT_BROKER_TOKEN
  });

  // Get current production version
  const prodVersion = await client.getLatestVersionForEnvironment(
    provider,
    'production'
  );

  // Compare contracts
  const comparison = await client.compareVersions(
    provider,
    prodVersion,
    newVersion
  );

  const breakingChanges: BreakingChange[] = [];

  for (const diff of comparison.differences) {
    if (diff.isBreaking) {
      breakingChanges.push({
        type: diff.type,
        path: diff.path,
        description: diff.,
        : diff.
      });
    }
  }

   {
    : breakingChanges. > ,
    breakingChanges,
    : breakingChanges. > 
      ? 
      : 
  };
}

MCP Server Integration

This skill can leverage the following MCP servers:

Server	Description	Installation
PactFlow MCP Server	AI-powered contract testing	PactFlow Blog
Specmatic MCP Server	Contract testing and mocks	GitHub

Best Practices

1. Consumer-first design - Write consumer tests before implementation
2. Meaningful states - Use descriptive provider state names
3. Version with git - Use commit SHAs for versions
4. Test all SDKs - Ensure all language SDKs have contracts
5. Can-i-deploy gates - Block deployments without verification
6. Webhook automation - Trigger verification automatically
7. Environment tracking - Record deployments per environment
8. Pending pacts - Enable for new SDK versions

Process Integration

This skill integrates with the following processes:

• sdk-testing-strategy.js - SDK testing patterns
• compatibility-testing.js - Cross-SDK compatibility
• backward-compatibility-management.js - Breaking change management
• sdk-versioning-release-management.js - Release coordination

Output Format

{
  "operation": "verify",
  "provider": "myservice-api",
  "providerVersion": "abc123",
  "consumers": [
    {
      "name": "myservice-typescript-sdk",
      "version": "def456",
      "status": "passed",
      "interactions": 12,
      "passed": 12,
      "failed": 0
    },
    {
      "name": "myservice-python-sdk",
      "version": "ghi789",
      "status": "passed",
      "interactions": 
       
       
    
  
   
   
   

Error Handling

• Handle missing provider states gracefully
• Provide clear mismatch descriptions
• Log full request/response on failures
• Support retry for transient broker failures
• Document breaking changes clearly

Constraints

• Contracts represent consumer needs only
• Provider states must be reproducible
• Broker must be accessible from CI/CD
• Version management is critical
• Breaking changes require coordination across SDKs