mutation-strategies

name	mutation-strategies
description	Mutation testing strategies and test-improvement patterns for TypeScript projects using Stryker Mutator with a Vitest runner. Provides guidance on interpreting mutation reports and improving tests to achieve higher kill rates. Use when analyzing mutation testing results.

Mutation Testing Strategies

This skill provides guidance on mutation testing with Stryker Mutator for JavaScript and TypeScript, interpreting results, and improving tests to achieve higher kill rates.

Stryker is optional — the QA pipeline only runs it when mutation_available: true in .state/config.json or when the user opts in with --with-mutation.

Understanding Mutation Testing

What is Mutation Testing?

Mutation testing evaluates test quality by introducing small changes (mutations) to your code and checking if tests detect them. A mutation that no test catches "survives" — indicating a gap in coverage or assertion strength. A mutation that breaks a test is "killed."

Requirements

Stryker for Vitest:

npm install -D @stryker-mutator/core @stryker-mutator/vitest-runner @stryker-mutator/typescript-checker

Minimal stryker.config.json at the repo root:

{
  "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
  "packageManager": "npm",
  "testRunner": "vitest",
  "checkers": ["typescript"],
  "tsconfigFile": "tsconfig.base.json",
  "reporters": ["progress", "json", "html"],
  "coverageAnalysis": "perTest",
  "mutate": [
    "backend/src/**/*.ts",
    "frontend/src/**/*.ts",
    "shared/src/**/*.ts",
    "!**/*.test.ts",
    "!**/__test-helpers/**",
    "!**/test/setup.ts"
  ]
}

Key Metrics

Metric	Target	Description
Mutation Score	Diagnostic	Higher is generally better, but diminishing returns above ~70%
Killed	Maximize	Mutations detected by tests
Survived	Minimize	Mutations NOT detected (test gaps)
NoCoverage	0	Code never executed by any test
Timeout	Investigate	Mutation caused infinite loop / hang — usually points to a missing termination test

Running Stryker

# Full run (slow — minutes to tens of minutes)
npx stryker run

# Mutate only one workspace
npx stryker run --mutate "backend/src/**/*.ts"

# Mutate a specific file/dir
npx stryker run --mutate "backend/src/scanner/**"

# Limit concurrency (default = cpu count)
npx stryker run --concurrency 2

Common Mutation Types

1. Arithmetic mutations

Mutation: changes arithmetic operators (+ → -, * → /, etc.).

// Original
const total = price * quantity;

// Mutant (survives if tests don't verify exact amount)
const total = price + quantity;

Kill strategy: assert exact calculated values.

// BAD (doesn't kill mutant)
expect(total).toBeGreaterThan(0);

// GOOD (kills mutant)
expect(total).toBe(150); // $50 * 3 = $150

2. Comparison / equality mutations

Mutation: changes comparison operators (> → >=, === → !==, etc.).

// Original
if (balance > 0) return 'positive';

// Mutant
if (balance >= 0) return 'positive';

Kill strategy: test boundary conditions explicitly.

it('returns neutral for zero balance', () => {
  expect(status(0)).toBe('neutral');
});

it('returns positive for the smallest positive balance', () => {
  expect(status(0.01)).toBe('positive');
});

3. Boolean mutations

Mutation: changes boolean values (true → false, negates conditions).

// Original
const isActive = true;

// Mutant
const isActive = false;

Kill strategy: assert both states explicitly.

it('is active by default', () => {
  expect(user.isActive).toBe(true);
});

it('can be deactivated', () => {
  user.deactivate();
  expect(user.isActive).toBe(false);
});

4. Return / block-statement mutations

Mutation: returns undefined, empty array, or removes a statement.

// Original
return collection.filter(predicate);

// Mutant
return [];

Kill strategy: assert on return structure AND contents.

// BAD
expect(result).toBeDefined();

// GOOD
expect(result).toHaveLength(3);
expect(result[0]?.name).toBe('Expected Name');

5. String-literal mutations

Mutation: replaces string literals with "" or a related variant.

// Original
res.status(403).json({ error: 'extension-not-allowed' });

// Mutant
res.status(403).json({ error: '' });

Kill strategy: assert the exact error code/string in tests.

const res = await request(app).put('/api/file').send({ path: 'x.sh' });
expect(res.status).toBe(403);
expect(res.body.error).toBe('extension-not-allowed');

6. Method-call mutations

Mutation: removes method calls or swaps them.

// Original
logger.info('User created', { id: user.id });

// Mutant (removed)

Kill strategy: mock and verify the call (only when the side effect is meaningful — log lines are usually noise).

const spy = vi.spyOn(logger, 'info');
service.createUser(data);
expect(spy).toHaveBeenCalledWith('User created', expect.objectContaining({ id: expect.any(String) }));

7. Increment / Update operator mutations

Mutation: ++ → --, += → -=.

// Original
count++;

// Mutant
count--;

Kill strategy: assert exact counts before and after.

const before = store.count;
store.increment();
expect(store.count).toBe(before + 1);

8. Optional-chaining / nullish-coalescing mutations

Mutation: ?? → ||, ?. → ..

// Original
const tool = entry.tool ?? 'claude';

// Mutant
const tool = entry.tool || 'claude';

Kill strategy: test with '' or 0 to expose the difference between ?? and ||.

it('falls back to "claude" only when tool is null/undefined', () => {
  expect(resolveTool({ tool: '' })).toBe(''); // killed: || would return 'claude'
  expect(resolveTool({ tool: undefined })).toBe('claude');
});

Surviving-Mutant Analysis

Step 1: identify the survivor

From the Stryker JSON / HTML report:

Mutant 47 [SURVIVED]:
  File: backend/src/scanner/scanner.ts:128
  Mutator: EqualityOperator
  Original:  if (entries.length === 0) {
  Mutated:   if (entries.length !== 0) {

Step 2: analyze why it survived

Reason	Solution
No assertion on the affected value	Add a specific value assertion
Assertion too loose (e.g., toBeDefined)	Make assertion specific to value/structure
Test only walks the happy path	Add tests for exact return / branch
Missing edge case	Add boundary tests (0, empty, max)
Mocking hides the real behavior	Reduce mocking; favor integration with withTmpRoot

Step 3: write the killing test

// Before (doesn't kill mutant)
it('scans a directory', async () => {
  const result = await scan(root, patterns);
  expect(result).toBeDefined();
});

// After (kills mutant — distinguishes empty vs non-empty)
it('returns an empty entries array when no files match', async () => {
  const result = await scan(emptyRoot, patterns);
  expect(result.entries).toEqual([]);
});

it('returns at least one entry when matches exist', async () => {
  const result = await scan(rootWithFiles, patterns);
  expect(result.entries.length).toBeGreaterThan(0);
});

Test-Improvement Patterns

Pattern 1: strengthen assertions

// Weak
expect(res.status).toBe(200);

// Strong
expect(res.status).toBe(200);
expect(res.body).toMatchObject({
  data: { count: 3, items: expect.arrayContaining([expect.objectContaining({ id: expected.id })]) },
});

Pattern 2: test boundaries

describe('size cap', () => {
  it('accepts a write at exactly the 1 MB limit', async () => { /* ... */ });
  it('rejects a write one byte over the limit', async () => { /* ... */ });
});

Pattern 3: test both branches

it('returns the entry when found', () => {
  expect(findById('a1')).toEqual(expect.objectContaining({ id: 'a1' }));
});

it('returns undefined when not found', () => {
  expect(findById('does-not-exist')).toBeUndefined();
});

Pattern 4: verify side effects

import { vi } from 'vitest';

it('emits harness:update after a debounced burst', async () => {
  const emitSpy = vi.fn();
  // ... wire emitter, write files in quick succession, advance fake timers
  expect(emitSpy).toHaveBeenCalledTimes(1); // coalesced
});

Pattern 5: assert calculated values exactly

it('derives an 8-char hex id from sha256(scope::path::layer)', () => {
  expect(deriveEntryId({ scope: 'project', path: 'a.md', layer: 'skill' }))
    .toBe('3f9a7b2e');
});

Workflow

1. Run Stryker locally: npx stryker run --mutate "backend/src/scanner/**" (start narrow).
2. Review surviving mutants from reports/mutation/html/index.html.
3. Categorize each survivor:
   • Real gap — a missing behavioral test for actual logic.
   • Noise — log messages, debug branches, cosmetic strings.
   • Acceptable risk — impractical to test, low business impact.
4. For real gaps: add a test to the appropriate behavioral test file (e.g., scanner.test.ts), not a separate "mutation killers" file.
5. For noise: add the line/file to Stryker's ignorePatterns or use // Stryker disable next-line all annotation.
6. Mutation score improves naturally as gaps are addressed — never chase a number.

Disabling mutations for non-behavior code

// Stryker disable next-line all
console.log('debug:', payload);

/* Stryker disable next-line StringLiteral */
const errorMessage = 'Unknown error';

Quick Reference: mutation → what it reveals

Mutation type	What it reveals
+ → -	Arithmetic result not asserted with specific values
> → >=	Boundary not tested
=== → !==	Equality branch not exercised both ways
true → false	Boolean state not explicitly verified in both directions
return x → return undefined	Return value not asserted by any test
Method-call removed	Side effect not verified (often noise if it's logging)
?? → ||	Falsy-vs-nullish distinction not tested with "" or 0
++ → --	Counter direction not asserted with exact values
&& → ||	Condition combinations not exercised
Block statement removed	Side-effect-only code not verified

Troubleshooting

"Stryker timed out on N mutants"

Stryker kills tests that exceed the per-test timeout. Causes:

• A real infinite-loop bug (e.g., recursion without termination) — these are valuable signals, investigate.
• A test using setTimeout / real timers — switch to vi.useFakeTimers().
• A flaky test — fix the test first, then re-run Stryker.

Slow mutation testing

# Mutate only the directory you changed
npx stryker run --mutate "backend/src/scanner/**"

# Reduce concurrency if the box is starving
npx stryker run --concurrency 4

# Cap mutants per file
# (in stryker.config.json: "thresholds": { "high": 80, "low": 60, "break": null })

High memory usage

# Stryker forks workers — set NODE_OPTIONS or reduce concurrency
NODE_OPTIONS='--max-old-space-size=4096' npx stryker run --concurrency 2

When NOT to run mutation testing

Skip Stryker for:

• CI pipelines (it's slow and produces results that need human judgment)
• Pre-commit hooks (use ESLint / typecheck / Vitest instead)
• Files dominated by side effects (logging, telemetry, console output)
• Generated code (zod-inferred types, build artifacts)

Mutation testing is a local diagnostic for tightening test quality on critical modules — not an automated gate.