contract-testing

name

contract-testing

description

Use this skill when writing, reviewing, or planning contract and schema tests. Covers API response schema validation with Ajv, OpenAPI/Swagger contract testing, consumer-driven contract testing with Pact, GraphQL schema testing, versioning and backward compatibility, and schema-first development. Trigger when the user mentions contract testing, schema validation, API contracts, OpenAPI, Pact, or consumer-driven testing.

Contract & Schema Testing

A reference for GitHub Copilot to generate API contract tests, schema validation, and consumer-driven contract checks.

---

API Response Schema Validation

Every API endpoint should validate that responses match a defined schema — not just status codes.

What to check:

• Response body matches the expected JSON schema (field names, types, required fields)
• Extra fields don't leak (no internal IDs, server paths, or debug info in production responses)
• Nested objects and arrays have consistent structure
• Null vs missing — a field set to null is different from a field omitted entirely
• Pagination metadata is consistent (page, pageSize, total, hasNext)

Example test pattern:

import Ajv from 'ajv';

const ajv = new Ajv();
const userSchema = {
  type: 'object',
  required: ['id', 'email', 'name', 'createdAt'],
  properties: {
    id: { type: 'string', format: 'uuid' },
    email: { type: 'string', format: 'email' },
    name: { type: 'string', minLength: 1 },
    createdAt: { type: 'string', format: 'date-time' },
  },
  additionalProperties: false, // Catch unintended field leaks
};

test('GET /users/:id matches schema', async () => {
  const response = await api.get('/users/123');
  const validate = ajv.compile(userSchema);
  expect(validate(response.body)).toBe(true);
  expect(validate.errors).toBeNull();
});

---

OpenAPI / Swagger Contract Testing

If the project has an OpenAPI spec, tests should validate that the implementation matches the spec.

What to check:

• Every documented endpoint returns the documented status codes
• Request body validation matches the spec's required fields and types
• Response shapes match the spec's schema definitions
• Error responses follow the documented error format
• Query parameters, headers, and path parameters match the spec
• The spec itself is valid (use a linter like @stoplight/spectral)

Tools:

• openapi-backend — route-level validation against spec
• prism — mock server from OpenAPI spec for consumer testing
• schemathesis — auto-generated property-based tests from OpenAPI spec

---

Consumer-Driven Contract Testing

When multiple services depend on an API, contracts ensure changes don't break consumers.

Pattern:

1. Consumer defines a contract: "I call GET /users/123 and expect { id, name, email }"
2. Provider verifies it: "My endpoint returns at least these fields with these types"
3. CI fails if the provider changes break any consumer's contract

What to check:

• Consumer tests define the minimum fields they need (not the full response)
• Provider tests verify all consumer contracts are satisfied
• New fields can be added without breaking existing contracts
• Removed or renamed fields trigger contract failures
• Contract tests run in both consumer and provider CI pipelines

Tools:

• Pact — the most widely used contract testing framework
• Manual schema comparison — define shared schemas in a package both sides import

---

GraphQL Schema Testing

What to check:

• Schema changes don't remove or rename fields used by existing queries
• New required arguments on existing queries/mutations are flagged as breaking
• Deprecated fields have a migration path documented
• Resolver return types match the schema definition
• Null/non-null annotations are correct (String! vs String)

Example test:

import { buildSchema, introspectionFromSchema } from 'graphql';

test('schema has not changed unexpectedly', () => {
  const schema = buildSchema(schemaString);
  const introspection = introspectionFromSchema(schema);
  expect(introspection).toMatchSnapshot();
});

test('breaking changes are detected', () => {
  const changes = findBreakingChanges(oldSchema, newSchema);
  expect(changes).toHaveLength(0);
});

---

Versioning & Backward Compatibility

What to check:

• API versioning strategy is consistent (URL path /v2/, header, query param)
• Deprecated endpoints still work for the documented sunset period
• New fields added to responses don't break existing consumers (additive changes)
• Removed fields are documented in changelog and trigger contract test failures
• Error response format is consistent across API versions

---

Best Practices

• Schema-first development — define the contract before implementing the endpoint
• Validate both directions — test that requests are validated AND responses match the schema
• Fail on extra fields — use additionalProperties: false to catch accidental data leaks
• Automate in CI — contract tests should block merges, not run after deploy
• Version your schemas — store schema definitions in version control alongside the code
• Test error contracts too — error responses need schemas just as much as success responses