test-coverage-audit

name	test-coverage-audit
description	Systematic audit of test coverage in Node.js/TypeScript projects using Vitest/Jest. Discovers test structure, runs tests, analyzes coverage gaps, and produces actionable recommendations. Trigger keywords: audit test coverage, check test health, testing gaps, vitest coverage, jest coverage, test infrastructure audit.
license	MIT
metadata	{"version":"1.0.0","hermes":{"tags":["Testing","Coverage","Audit","Vitest","Jest","Node.js","TypeScript"],"related_skills":["github-code-review"]}}

Test Coverage Audit

Systematically audit test coverage in a Node.js/TypeScript project.

Prerequisites

• Node.js project with a test framework (Vitest, Jest, or similar)
• Access to run npm/npx commands
• shell.nix or node_modules available

Execution Steps

1. Discover Test Setup

Identify the testing framework and configuration:

# Look for config files
ls vitest.config.* jest.config.* 2>/dev/null

# Check package.json for test scripts
jq '.scripts | with_entries(select(.key | contains("test")))' package.json

# Find test directories
find . -type d -name "*test*" -o -name "*spec*" 2>/dev/null | head -20

2. Install Dependencies (if needed)

# Check if node_modules exists
ls node_modules 2>/dev/null || nix-shell --run "npm install"

3. Run Tests and Capture Baseline

# Run with verbose output to see all tests
nix-shell --run "npx vitest run --reporter=verbose" 2>&1 | head -100

# Capture full results
nix-shell --run "npx vitest run" 2>&1 | tail -60

Watch for:

• Tests that pass but have limited coverage
• Tests that fail due to missing env vars (DATABASE_URL, API keys)
• Tests that are skipped or not running at all

4. Map Source to Test Files

Count and correlate:

# Source files (excluding tests)
find server -type f \( -name "*.ts" -o -name "*.tsx" \) ! -name "*.test.ts" | wc -l

# Test files
find tests -type f -name "*.test.ts" 2>/dev/null | wc -l
find server -type f -name "*.test.ts" | wc -l

Create a mapping:

• Which source files have corresponding .test.ts files?
• Which are tested indirectly via integration tests?
• Which have no test coverage?

5. Analyze Route/API Coverage (for backend projects)

Extract route definitions from the main routes file:

import re

with open("server/routes.ts", "r") as f:
    content = f.read()

# Find route definitions
routes = re.findall(r"\.(get|post|put|patch|delete)\s*\(\s*['\"]([^'\"]+)['\"]", content)

Categorize:

• Routes with dedicated integration tests
• Routes tested indirectly
• Routes with no test coverage

6. Identify Critical Gaps

Look for:

1. Infrastructure gaps

   • Missing test script in package.json
   • No coverage reporting
   • Tests require env vars that aren't documented

2. Functional gaps

   • Admin/management routes untested
   • DELETE operations untested
   • Error handling paths not covered

3. Service layer gaps

   • External API integrations not mocked
   • Database operations not tested
   • Background jobs untested

7. Generate Report

Structure the findings:

SUMMARY
- Total tests passing
- Test files passing/failing
- Coverage percentage (if available)

TEST BREAKDOWN
- By directory/module
- Count of tests per file

ROUTE COVERAGE
- List covered routes
- List uncovered routes
- Coverage ratio

CRITICAL GAPS
- Infrastructure issues
- Functional gaps
- Missing test scenarios

RECOMMENDATIONS
- High priority fixes
- Medium priority additions
- Nice-to-have improvements

Common Issues and Fixes

Issue: DATABASE_URL Required

Server tests fail because they import db.ts which requires DATABASE_URL.

Fix: Create a test setup that mocks the database or uses an in-memory/test database.

// tests/setup.ts or vitest.config.ts
import { vi } from 'vitest';

vi.mock('./db', () => ({
  db: mockDb
}));

Issue: Some Test Files Pass, Others Fail (Env Var Pattern)

You may see a pattern where tests in tests/ pass but tests in server/ fail with:

Error: DATABASE_URL must be set. Did you forget to provision a database?

Diagnosis:

• Integration tests in tests/ mock the database or use supertest without importing db.ts directly
• Unit tests in server/ import modules that import db.ts at the top level
• The server/*.test.ts files fail immediately on import, before any tests run

Investigation:

# Check which test files require DATABASE_URL
head -20 server/*.test.ts | grep -A5 "import.*db"

# Check if tests/ setup mocks the database
cat tests/setup.ts 2>/dev/null | head -30

Fix Options:

1. Add a test database URL to the test environment:

   export DATABASE_URL="postgresql://user@localhost:5432/test_db"
   

2. Exclude failing tests from the default run:

   // vitest.config.ts
   exclude: ["node_modules", "dist", "server/*.test.ts"]
   

3. Create a separate test configuration for integration vs unit tests

Document this in your report:

SERVER TESTS (server/*.test.ts): Cannot run - require DATABASE_URL
  - These would test storage layer and routes directly
  - Currently 0 tests running from this directory
  - Recommendation: Add test database or mock setup

Issue: No Test Script

package.json lacks a test script.

Fix: Add to package.json:

{
  "scripts": {
    "test": "vitest run",
    "test:watch": "vitest",
    "test:coverage": "vitest run --coverage"
  }
}

Issue: Tests Pass but Coverage is Low

Tests exist but don't exercise all code paths.

Indicators:

• Few assertions per test
• No error case testing
• Missing integration tests for full request/response cycles

Fix: Add integration tests that hit actual endpoints with various inputs.

Output Template

======================================================================
[PROJECT] TEST COVERAGE AUDIT REPORT
======================================================================

SUMMARY
----------------------------------------------------------------------
  Total Tests Passing:     X
  Test Files Passing:      X
  Test Files Failing:      X (reason)
  Coverage:                X% (if available)

TEST BREAKDOWN
----------------------------------------------------------------------
  [Directory Structure with test counts]

ROUTE COVERAGE ANALYSIS
----------------------------------------------------------------------
  COVERED ROUTES:
    ✓ [route]

  UNCOVERED ROUTES:
    ✗ [route]

  COVERAGE RATIO: X%

CRITICAL GAPS
----------------------------------------------------------------------
  1. [Issue description]
  2. [Issue description]

RECOMMENDATIONS
----------------------------------------------------------------------
  HIGH PRIORITY:
    - [action item]

  MEDIUM PRIORITY:
    - [action item]

  LOW PRIORITY:
    - [action item]

Verification

After making fixes, re-run:

nix-shell --run "npx vitest run"

Confirm:

• Previously failing tests now pass
• New tests are discovered and run
• Coverage report generates (if configured)