helmet

name

helmet

description

Full repo onboarding — bootstraps test infrastructure (Phase A), wires the CI/CD pipeline (Phase B), and generates a project CLAUDE.md (Phase C). Use when onboarding a new repo, setting up tests + CI from scratch, adding Codecov/pinact/SBOM/security scanning, auditing pipeline completeness, fixing CI failures, deploying pipeline changes across multiple repos, or generating/refreshing a repo's CLAUDE.md. Replaces ci-pipeline-setup and test-setup.

Repo Pipeline Setup

Three-phase repo onboarding: Phase A bootstraps test infrastructure (language detection, framework detection, test runner + coverage, smoke tests, gold-standard templates). Phase B wires the CI/CD pipeline (Codecov, SHA pinning, SBOM, vulnerability scanning, security backstop, Dependabot, commit signing, OpenSSF Scorecard, CodeScene, GitGuardian). Phase C generates a project CLAUDE.md by analyzing the repo's tech stack, structure, commands, conventions, and CI configuration — so every new Claude Code session starts with full project context.

When to Use

Phase A (Test Infrastructure):

• Onboarding an existing repo that has application code but no test suite
• Starting a new project and want test infrastructure from the start
• CI audit found missing test infrastructure (e.g., Codecov marked N/A)

Phase B (CI/CD Pipeline):

• Onboarding a new repo into the CI pipeline
• Adding or fixing Codecov, pinact, or GitGuardian for existing repos
• Deploying pipeline changes across multiple repos at once
• Fixing cross-platform CI failures (lightningcss, npm ci, vitest coverage, Swift iOS-only)
• Auditing CI pipeline completeness across the portfolio
• Adding SBOM generation or build provenance attestations
• Setting up SSH commit signing or troubleshooting signature issues
• Configuring Dependabot security alerts or version updates
• Deploying OpenSSF Scorecard or SECURITY.md
• Setting up CodeScene behavioral code analysis on PRs
• Adding security scanning CI backstop (Semgrep, Checkov, Zizmor) for defense-in-depth

Phase C (CLAUDE.md):

• Onboarding a new repo — auto-runs after Phase B completes
• Repo has no .claude/CLAUDE.md or it contains only boilerplate
• User asks to generate or refresh a project's CLAUDE.md
• Significant infrastructure changes (new test framework, CI additions) made CLAUDE.md stale

---

Phase A: Test Infrastructure

Bootstrap test infrastructure for repos with testable code but no tests. Detects language and framework, installs the test runner + coverage provider, creates config, and generates a smoke test + gold-standard template test.

When to Use

• Onboarding an existing repo that has application code but no test suite
• Starting a new project and want test infrastructure from the start
• CI audit found missing test infrastructure (e.g., Codecov marked N/A)

A0. Precondition Check

Before running detection, verify the repo has testable application code.

A repo is "testable" when BOTH conditions are met:

1. At least one language config file exists: package.json, go.mod, Cargo.toml, pyproject.toml, setup.py, requirements.txt, Package.swift, or a *.xcodeproj directory
2. At least one non-test source file exists in that language (.ts, .tsx, .js, .jsx, .py, .go, .rs, .swift)

Exclude from source file count: node_modules/, vendor/, .git/, dist/, build/, generated files.

If NEITHER condition is met (no config file AND no source files), stop and report:

"This repo has no testable application code. Test infrastructure is not applicable. Consider shellcheck for shell scripts or JSON schema validation for config files."

A1. Detection

A1a. Language Detection

Detect languages in order of confidence. Check config files first (highest signal), then fall back to file extension counts.

Primary signal — config files:

Config File	Language
package.json	TypeScript/JavaScript
go.mod	Go
Cargo.toml	Rust
pyproject.toml, setup.py, requirements.txt	Python
Package.swift, *.xcodeproj (directory, not file)	Swift

Fallback — file extension count (when no config file found for a language):

Extensions	Language
.ts, .tsx, .js, .jsx	TypeScript/JavaScript
.go	Go
.rs	Rust
.py	Python
.swift	Swift

Mixed repos: Detect ALL languages present. Scope each language's setup to its root directory:

• Find the nearest config file (package.json, go.mod, etc.) and treat that directory as the language root.
• Example: package.json at repo root + go.mod in services/api/ -> run TS setup at root, Go setup scoped to services/api/.
• Each language gets independent detection, installation, and output. They do not share test directories or configs.

A1b. Framework Detection

After detecting the language, inspect dependency declarations for framework-specific packages. The detected framework determines which test patterns the template test will demonstrate.

TypeScript/JavaScript (check dependencies + devDependencies in package.json):

Dependency	Framework	Template test approach
express	Express	supertest route tests
next	Next.js	Route handler tests, API route tests
hono	Hono	Hono test client
fastify	Fastify	app.inject() tests
None matched	Generic	Export/function-level unit tests

Python (check pyproject.toml [project.dependencies] or requirements.txt):

Dependency	Framework	Template test approach
fastapi	FastAPI	TestClient, dependency overrides
django	Django	TestCase, Client, model tests
flask	Flask	Test client, route tests
typer	Typer (CLI)	CliRunner, exit codes, output assertions
click	Click (CLI)	CliRunner, exit codes, output assertions
None matched	Generic	Module/function-level tests

Go (check require block in go.mod):

Dependency	Framework	Template test approach
github.com/gin-gonic/gin	Gin	httptest + gin test context
github.com/go-chi/chi	Chi	httptest + chi router
net/http imports in .go source files (not in go.mod — stdlib packages don't appear there)	Stdlib	httptest handler tests
None matched	Generic	Table-driven function tests

Rust (check [dependencies] in Cargo.toml):

Dependency	Framework	Template test approach
actix-web	Actix	actix_web::test, TestRequest
axum	Axum	Tower service tests
None matched	Generic	#[cfg(test)] module tests

Swift (check Package.swift dependencies or project structure):

Signal	Framework	Template test approach
import Testing in source files (Xcode 16+ / Swift 6)	Swift Testing	@Test functions, #expect assertions (note: Phase 2/3 templates use XCTest as fallback until Swift Testing templates are added)
SwiftUI imports + *.xcodeproj dir	SwiftUI app	ViewInspector, @Observable state tests
Package.swift (library)	Swift package	XCTest module tests
Vapor in dependencies	Vapor	XCTVapor request tests

A1c. Existing Test Detection

Before installing, check if test infrastructure already exists for each detected language. Skip or fill gaps as needed.

Signals to check:

Signal	Means
Test directories (__tests__/, tests/, test/, *_test.go files)	Tests may exist
Test config files (vitest.config.*, jest.config.*, pytest.ini, pyproject.toml with [tool.pytest])	Test framework configured
Test scripts in package.json ("test", "test:coverage") or Makefile (test: target)	Test runner registered
Coverage config (.coveragerc, .nycrc, codecov.yml)	Coverage already set up

Decision rules:

Config exists	Test dir exists	Test script exists	Action
Yes	Yes	Yes	Skip -- fully set up
Yes	No	--	Create directory only, keep existing config
No	Yes	--	Create config only, keep existing directory
--	--	No (but config + dir exist)	Add script only
No	No	No	Full setup

Proceed automatically in all cases (no user prompt). Report what was created vs. what was skipped.

A2. Installation

Install the test framework and coverage provider for each detected language. If installation fails (network, permissions, version conflict), stop and report the error -- do not proceed to Phase 3.

Package Manager Detection (TypeScript/JavaScript)

Detect the package manager from the lock file. Fall back to npm.

Lock File	Package Manager	Install Command
bun.lockb or bun.lock	bun	bun add -D vitest @vitest/coverage-v8
pnpm-lock.yaml	pnpm	pnpm add -D vitest @vitest/coverage-v8
yarn.lock	yarn	yarn add -D vitest @vitest/coverage-v8
package-lock.json or none	npm	npm install -D vitest @vitest/coverage-v8

Per-Language Installation

TypeScript/JavaScript

1. Install vitest + coverage provider via detected package manager
2. Install framework-specific test helpers based on detected framework:

Framework	Additional dev dependency
Express	supertest
Hono	(built-in test client, no extra dep)
Fastify	(built-in app.inject(), no extra dep)
Next.js	(no extra dep for route handler tests)
Generic	(no extra dep)

3. Create vitest.config.ts:

import { defineConfig } from 'vitest/config'

export default defineConfig({
  test: {
    coverage: {
      provider: 'v8',
      reporter: ['text', 'lcov'],  // lcov for Codecov compatibility
      exclude: ['node_modules/', 'dist/', '**/*.config.*'],
    },
  },
})

4. Add to package.json scripts:
   • "test": "vitest run"
   • "test:coverage": "vitest run --coverage"
5. Create __tests__/ directory

Python

1. Determine installation method:
   • uv.lock present -> add pytest and pytest-cov to dev dependencies, run uv sync --dev or uv pip install -e ".[dev]"
   • pyproject.toml with PEP 621 [project] section -> add pytest and pytest-cov to [project.optional-dependencies] dev group, run pip install -e ".[dev]"
   • pyproject.toml with Poetry ([tool.poetry]), PDM, or other non-PEP-621 format -> fall back to requirements-dev.txt approach
   • No pyproject.toml -> create requirements-dev.txt with pytest and pytest-cov, run pip install -r requirements-dev.txt
2. If $VIRTUAL_ENV is unset and no uv.lock, warn: "No virtual environment detected. Consider python -m venv .venv first." Proceed anyway.
3. Add pytest config to pyproject.toml (create or append):

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--cov=<package> --cov-report=xml --cov-report=term"  # Replace <package> with actual package name (e.g., src, app)

4. Create tests/ directory with __init__.py and conftest.py

Go

1. No installation needed (testing is built-in)
2. If Makefile exists, add targets:

test:
	go test ./...

test-coverage:
	go test -coverprofile=coverage.out ./... && go tool cover -html=coverage.out -o coverage.html

3. No separate test directory -- Go test files go alongside source files (*_test.go)

Rust

1. No test framework installation needed (built-in #[test])
2. Attempt coverage tool install:

   cargo install cargo-llvm-cov
   

   If install fails, warn: "cargo-llvm-cov not installed. Tests will work but coverage reports require it. Install manually or use CI-only coverage." Continue with setup.
3. Create tests/ directory for integration tests

Swift

1. XCTest is built-in -- no installation needed
2. For Package.swift projects: add test target if missing:

   .testTarget(name: "AppTests", dependencies: ["App"])
   

3. For Xcode projects: verify test target exists, warn if missing (cannot auto-create Xcode test targets reliably)
4. Create Tests/AppTests/ directory structure

A3. Output Files

Generate up to three test files per detected language:

1. Smoke test (always) — proves the app can be imported.
2. Gold-standard template test (always) — heavily commented pattern for example-based tests.
3. Property test template (opt-in) — ask the user first: "Does this repo have parsers, validators, serializers, crypto, state machines, or financial logic?" If yes, generate; if no, skip and don't install the framework.

A. Smoke Test

Generate one real, runnable test that proves the app can be imported without crashing.

Scope: Import-only. Does NOT start servers, connect to databases, or trigger side effects. If the app performs side effects on import (e.g., mongoose.connect() at module level), the smoke test will fail -- report this with the suggestion: "Your app performs side effects on import. Consider wrapping startup logic in a function."

File placement:

Language	Smoke test file
TypeScript/JS	__tests__/smoke.test.ts
Python	tests/test_smoke.py
Go	smoke_test.go (in root package)
Rust	tests/smoke.rs
Swift	Tests/AppTests/SmokeTests.swift

Templates:

TypeScript/JavaScript

import { describe, it, expect } from 'vitest'

describe('smoke', () => {
  it('main module imports without error', async () => {
    const mod = await import('../src/index')
    expect(mod).toBeDefined()
  })
})

Adjust the import path (../src/index) to match the actual entry point found in package.json "main" or "exports" field.

Python

def test_smoke():
    """Verify the main package can be imported."""
    import app  # noqa: F401

Adjust import app to match the actual package name (the top-level directory containing __init__.py, or the module name from pyproject.toml).

Go

package main

import "testing"

func TestSmoke(t *testing.T) {
    // Verify the package compiles and main symbols are accessible.
    // If this test fails, the package has a build error.
    t.Log("smoke test: package compiles successfully")
}

Place in the root package directory. Adjust package main to match the actual package name if different.

Rust

#[test]
fn smoke() {
    // Verify the crate compiles and can be used as a dependency.
    // If this fails, there is a build error in the main crate.
    assert!(true, "crate compiles successfully");
}

Place as tests/smoke.rs (integration test). The crate name is auto-resolved from Cargo.toml.

Swift

import XCTest
@testable import App

final class SmokeTests: XCTestCase {
    func testSmoke() {
        // Verify the module can be imported without error.
        XCTAssertTrue(true, "Module imports successfully")
    }
}

Adjust @testable import App to match the actual module/target name from Package.swift or the Xcode project.

B. Gold-Standard Template Test

Generate one heavily commented test file showing the right patterns for the detected language+framework. Contains 2-3 real implemented tests (not TODOs) demonstrating:

1. Happy path -- basic operation with expected input
2. Error case -- how to test error handling
3. Framework pattern -- one idiomatic framework-specific test (e.g., authenticated route, middleware)

Comments explain: import conventions, test structure, mocking approach, and where to find more patterns.

File placement:

Language	Template test file	Naming rationale
TypeScript/JS	__tests__/_template.test.ts	Underscore sorts first
Python	tests/test_template.py	Follows pytest test_ convention
Go	template_test.go (root package)	Matches template naming in other languages (example_test.go is reserved for godoc examples)
Rust	tests/template.rs	Integration test in tests/
Swift	Tests/AppTests/TemplateTests.swift	XCTest naming convention

Generate the template based on the detected framework. Use the framework detection from Phase 1b to select the right test patterns. The template must use the actual framework's test helpers (e.g., supertest for Express, TestClient for FastAPI, httptest for Go stdlib).

References to include in template comments:

• busdriver:tdd -- for generating tests for specific modules
• Language-specific testing skill -- busdriver:golang-testing, busdriver:python-testing, busdriver:rust-testing, etc.

TypeScript/JavaScript -- Express example

/**
 * TEMPLATE TEST -- Copy this file as a starting point for new test files.
 *
 * Pattern: supertest + vitest for Express route testing.
 * Run: npm test
 * Coverage: npm run test:coverage
 *
 * For full TDD workflow, use `busdriver:tdd` to generate tests for specific modules.
 * For more patterns, see `busdriver:tdd`.
 */
import { describe, it, expect } from 'vitest'
import request from 'supertest'
import { app } from '../src/app'

describe('GET /health', () => {
  // Happy path: verify the endpoint returns expected shape
  it('returns 200 with status ok', async () => {
    const res = await request(app).get('/health')
    expect(res.status).toBe(200)
    expect(res.body).toEqual({ status: 'ok' })
  })

  // Error case: verify proper error response format
  it('returns 404 for unknown routes', async () => {
    const res = await request(app).get('/nonexistent')
    expect(res.status).toBe(404)
  })

  // Framework pattern: testing with auth header
  it('authenticated route returns 401 without token', async () => {
    const res = await request(app).get('/api/protected')
    expect(res.status).toBe(401)
  })
})

TypeScript/JavaScript -- Generic (no framework)

/**
 * TEMPLATE TEST -- Copy this file as a starting point for new test files.
 *
 * Pattern: vitest for unit testing exported functions.
 * Run: npm test
 * Coverage: npm run test:coverage
 *
 * For full TDD workflow, use `busdriver:tdd`.
 */
import { describe, it, expect } from 'vitest'
// import { yourFunction } from '../src/utils'

describe('yourFunction', () => {
  // Happy path
  it.todo('returns expected result for valid input')
  // it('returns expected result for valid input', () => {
  //   const result = yourFunction('valid')
  //   expect(result).toBe(expected)
  // })

  // Error case
  it.todo('throws on invalid input')
  // it('throws on invalid input', () => {
  //   expect(() => yourFunction(null)).toThrow()
  // })
})

Python -- FastAPI example

"""
TEMPLATE TEST -- Copy this file as a starting point for new test files.

Pattern: pytest + TestClient for FastAPI endpoint testing.
Run: pytest
Coverage: pytest --cov

For full TDD workflow, use `busdriver:tdd`.
For more patterns, see `busdriver:python-testing`.
"""
from fastapi.testclient import TestClient
from app.main import app

client = TestClient(app)


# Happy path: verify endpoint returns expected shape
def test_health_returns_ok():
    response = client.get("/health")
    assert response.status_code == 200
    assert response.json() == {"status": "ok"}


# Error case: verify proper error response
def test_unknown_route_returns_404():
    response = client.get("/nonexistent")
    assert response.status_code == 404


# Framework pattern: dependency override for testing
def test_with_dependency_override():
    """Example of overriding a FastAPI dependency for testing."""
    # from app.dependencies import get_db
    # def mock_db():
    #     return FakeDB()
    # app.dependency_overrides[get_db] = mock_db
    # response = client.get("/items")
    # app.dependency_overrides.clear()
    pass  # Replace with real test

Python -- Generic (no framework)

"""
TEMPLATE TEST -- Copy this file as a starting point for new test files.

Pattern: pytest for unit testing functions and classes.
Run: pytest
Coverage: pytest --cov

For full TDD workflow, use `busdriver:tdd`.
For more patterns, see `busdriver:python-testing`.
"""
# from your_module import your_function


# Happy path: verify function returns expected result
def test_happy_path():
    # result = your_function("valid input")
    # assert result == expected
    pass  # Replace with real test


# Error case: verify error handling
def test_error_case():
    # with pytest.raises(ValueError):
    #     your_function(None)
    pass  # Replace with real test

Python -- Typer CLI example

"""
TEMPLATE TEST -- Copy this file as a starting point for new test files.

Pattern: pytest + Typer's CliRunner for CLI command testing.
Run: pytest
Coverage: pytest --cov

For full TDD workflow, use `busdriver:tdd`.
For more patterns, see `busdriver:python-testing`.
"""
from typer.testing import CliRunner
from app.main import app  # Adjust to your Typer app import

runner = CliRunner()


# Happy path: verify command runs and produces expected output
def test_command_succeeds():
    result = runner.invoke(app, ["--help"])
    assert result.exit_code == 0
    assert "Usage" in result.output


# Error case: verify proper error exit on bad input
def test_command_bad_input():
    result = runner.invoke(app, ["nonexistent-command"])
    assert result.exit_code != 0


# Framework pattern: test a subcommand with arguments
def test_subcommand_with_args(tmp_path):
    """Use tmp_path for any file I/O to keep tests isolated."""
    # result = runner.invoke(app, ["process", "--input", str(tmp_path / "data.csv")])
    # assert result.exit_code == 0
    # assert "Processed" in result.output
    pass  # Replace with real test

Python -- Click CLI example

"""
TEMPLATE TEST -- Copy this file as a starting point for new test files.

Pattern: pytest + Click's CliRunner for CLI command testing.
Run: pytest
Coverage: pytest --cov

For full TDD workflow, use `busdriver:tdd`.
For more patterns, see `busdriver:python-testing`.
"""
from click.testing import CliRunner
from app.main import cli  # Adjust to your Click group/command import

runner = CliRunner()


# Happy path: verify command runs and produces expected output
def test_command_succeeds():
    result = runner.invoke(cli, ["--help"])
    assert result.exit_code == 0
    assert "Usage" in result.output


# Error case: verify proper error exit on bad input
def test_command_missing_required():
    result = runner.invoke(cli, ["process"])  # Missing required arg
    assert result.exit_code != 0
    assert "Error" in result.output or "Missing" in result.output


# Framework pattern: test with isolated filesystem
def test_command_with_files(tmp_path):
    """Use tmp_path for file I/O; use runner.isolated_filesystem() for CWD isolation."""
    # with runner.isolated_filesystem(temp_dir=tmp_path):
    #     result = runner.invoke(cli, ["init"])
    #     assert result.exit_code == 0
    pass  # Replace with real test

Go -- net/http example

// Template test -- copy this file as a starting point for new test files.
//
// Pattern: table-driven tests with httptest for HTTP handler testing.
// Run: make test (or go test ./...)
// Coverage: make test-coverage
//
// For full TDD workflow, use `busdriver:tdd`.
// For more patterns, see `busdriver:golang-testing`.

package main

import (
	"net/http"
	"net/http/httptest"
	"testing"
)

// Happy path: verify handler returns expected status.
func TestHealthHandler(t *testing.T) {
	req := httptest.NewRequest(http.MethodGet, "/health", nil)
	w := httptest.NewRecorder()
	healthHandler(w, req)

	if w.Code != http.StatusOK {
		t.Errorf("expected 200, got %d", w.Code)
	}
}

// Error case: table-driven test pattern for multiple inputs.
func TestHealthHandler_EdgeCases(t *testing.T) {
	tests := []struct {
		name   string
		method string
		want   int
	}{
		{"GET returns 200", http.MethodGet, http.StatusOK},
		{"POST returns 405", http.MethodPost, http.StatusMethodNotAllowed},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			req := httptest.NewRequest(tt.method, "/health", nil)
			w := httptest.NewRecorder()
			healthHandler(w, req)
			if w.Code != tt.want {
				t.Errorf("expected %d, got %d", tt.want, w.Code)
			}
		})
	}
}

Go -- Generic (no framework)

// Template test -- copy this file as a starting point for new test files.
//
// Pattern: table-driven tests for pure functions.
// Run: go test ./...
// Coverage: go test -coverprofile=coverage.out ./...
//
// For full TDD workflow, use `busdriver:tdd`.
// For more patterns, see `busdriver:golang-testing`.

package main

import "testing"

// Happy path: verify function returns expected result.
func TestYourFunction(t *testing.T) {
	// result := YourFunction("valid input")
	// if result != expected {
	//     t.Errorf("expected %v, got %v", expected, result)
	// }
	t.Log("Replace with real test")
}

// Error case: table-driven test pattern.
func TestYourFunction_EdgeCases(t *testing.T) {
	tests := []struct {
		name    string
		input   string
		wantErr bool
	}{
		{"valid input", "hello", false},
		{"empty input", "", true},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			// _, err := YourFunction(tt.input)
			// if (err != nil) != tt.wantErr {
			//     t.Errorf("wantErr=%v, got err=%v", tt.wantErr, err)
			// }
			t.Log("Replace with real test")
		})
	}
}

Rust -- Generic example

//! TEMPLATE TEST -- Copy this file as a starting point for new integration tests.
//!
//! Pattern: integration test in tests/ directory.
//! Run: cargo test
//! Coverage: cargo llvm-cov
//!
//! For full TDD workflow, use `busdriver:tdd`.
//! For more patterns, see `busdriver:rust-testing`.

// use your_crate::your_function;

// Happy path: verify function returns expected result
#[test]
fn test_happy_path() {
    // let result = your_function("valid input");
    // assert_eq!(result, expected);
}

// Error case: verify error handling
#[test]
fn test_error_case() {
    // let result = your_function("");
    // assert!(result.is_err());
}

Swift -- XCTest example

/// TEMPLATE TEST -- Copy this file as a starting point for new test files.
///
/// Pattern: XCTest for unit testing.
/// Run: swift test (SPM) or xcodebuild test (Xcode)
///
/// For full TDD workflow, use `busdriver:tdd`.
import XCTest
@testable import App

final class TemplateTests: XCTestCase {

    // Happy path: verify function returns expected result
    func testHappyPath() throws {
        // let result = yourFunction("valid")
        // XCTAssertEqual(result, expected)
        throw XCTSkip("Template — replace with real test")
    }

    // Error case: verify error handling
    func testErrorCase() throws {
        // XCTAssertThrowsError(try yourFunction(nil))
        throw XCTSkip("Template — replace with real test")
    }
}

Adapt all import paths and function names to match the actual codebase. The template is a starting point -- the tests should compile and pass as-is, so the developer can immediately see the pattern and replace with real tests.

Placeholder tests: Avoid always-passing no-op assertions (assert True, expect(true).toBe(true), XCTAssertTrue(true)) in template tests -- they inflate pass counts and trigger automated reviewer warnings. Instead:

• Python: Use pass for placeholder bodies
• TypeScript/JS: Use it.todo('description') (vitest/jest mark them as pending, not passing)
• Swift: Use throw XCTSkip("Template — replace with real test")
• Go: t.Log(...) is fine (informational, not a false assertion)
• Rust: Commented-out assertions are fine (no placeholder needed)

For tests that demonstrate a real pattern (e.g., importing the app, hitting a real endpoint), use actual assertions -- only use placeholders for commented-out examples the developer hasn't wired up yet.

C. Property-Based Test Template (Optional)

Property-based testing complements example-based testing by generating random inputs and checking that invariants hold across all of them. It catches edge cases that example tests miss (empty strings, unicode boundaries, integer overflow, concurrent-operation orderings).

When to use property tests:

Good fit	Poor fit
Parsers / serializers — round-trip invariants (parse(serialize(x)) == x)	CRUD endpoints / HTTP glue
Validators — rejected inputs stay rejected after canonicalization	UI rendering logic
Crypto / hashing — output length, determinism, collision properties	Database migrations
State machines — sequences of operations preserve invariants	Configuration loaders
Financial calculators — commutative / associative / zero-sum properties	Pure plumbing / pass-throughs
Sort / search / data structures — algorithmic invariants	Pure presentation-layer code

If a repo has no modules matching "good fit," skip this template — adding property tests to CRUD handlers is noise, not signal.

Three gold-standard property patterns each template demonstrates:

1. Invariant — a property that always holds (reverse(reverse(x)) == x, parse(serialize(x)) == x)
2. Oracle — compare new implementation against a simple reference (my_sort(xs) == sorted(xs))
3. Model/state-machine — a sequence of operations preserves a higher-level invariant (push then pop on stack returns same element)

File placement:

Language	Property test file	Framework	Install
Python	tests/test_properties_template.py	Hypothesis	pip install hypothesis
TypeScript/JS	__tests__/properties.test.ts	fast-check	npm install -D fast-check
Go	properties_test.go	rapid (preferred — has shrinking) or testing/quick (stdlib, no shrinking)	go get -t pgregory.net/rapid
Rust	tests/properties.rs	proptest	cargo add --dev proptest
Swift	Tests/AppTests/PropertyTests.swift	Pragmatic mix — see Swift template notes	See notes

Opt-in flow: During Phase A, after detecting language + modules, ask the user:

"Does this repo have parsers, validators, serializers, crypto, state machines, or financial logic? (y/n)"

If yes → generate property test template. If no → skip; add no framework dependency.

Default is no — don't install framework dependencies speculatively.

Python -- Hypothesis

"""
PROPERTY TEST TEMPLATE -- Copy this file as a starting point.

When to use: parsers, serializers, validators, crypto, state machines,
financial calculators. NOT for CRUD, HTTP glue, or pure plumbing.

Pattern: Hypothesis generates random inputs; you assert invariants.
Run: pytest tests/test_properties_template.py
Install: pip install hypothesis

For more patterns, see https://hypothesis.readthedocs.io/en/latest/quickstart.html
"""
import pytest
from hypothesis import given, strategies as st

# from your_module import serialize, deserialize, validate, canonicalize


# 1. INVARIANT -- round-trip property: parse(serialize(x)) == x
@given(st.dictionaries(st.text(), st.integers()))
def test_serialize_roundtrip_is_identity(data):
    """Serializing then parsing should always return the original."""
    pytest.skip("Template — replace body with real serialize/deserialize round-trip")
    # assert deserialize(serialize(data)) == data


# 2. ORACLE -- compare implementation to a known-correct reference
@given(st.lists(st.integers()))
def test_custom_sort_matches_python_sorted(xs):
    """Your sort should match Python's built-in sorted()."""
    pytest.skip("Template — replace body with real my_sort vs sorted() comparison")
    # assert my_sort(list(xs)) == sorted(xs)


# 3. MODEL/STATE-MACHINE -- sequences of ops preserve an invariant
@given(st.lists(st.integers()))
def test_push_then_pop_returns_same_element(items):
    """Stack push/pop is a two-way mapping for each element."""
    pytest.skip("Template — replace body with real stack LIFO check")
    # stack = Stack()
    # for item in items:
    #     stack.push(item)
    #     assert stack.pop() == item


# Shrinking example -- Hypothesis will minimize a failing input.
# Uncomment to see: the test will "fail" with a minimal counterexample.
# @given(st.lists(st.integers()))
# def test_intentionally_fails_to_show_shrinking(xs):
#     assert sum(xs) < 1_000_000  # Shrinker finds [1_000_000] as minimal fail

TypeScript/JavaScript -- fast-check

/**
 * PROPERTY TEST TEMPLATE -- Copy this file as a starting point.
 *
 * When to use: parsers, serializers, validators, crypto, state machines,
 * financial calculators. NOT for CRUD, HTTP glue, or pure plumbing.
 *
 * Pattern: fast-check generates random inputs; you assert invariants.
 * Run: npm test -- properties
 * Install: npm install -D fast-check
 *
 * For more patterns, see https://fast-check.dev/docs/tutorials/quick-start/
 */
import { describe, it } from 'vitest'
// import fc from 'fast-check'
// import { serialize, deserialize, mySort, Stack } from '../src/module'

describe('property tests', () => {
  // 1. INVARIANT -- round-trip property
  it.todo('serialize + deserialize is identity')
  // it('serialize + deserialize is identity', () => {
  //   fc.assert(
  //     fc.property(fc.dictionary(fc.string(), fc.integer()), (data) => {
  //       expect(deserialize(serialize(data))).toEqual(data)
  //     })
  //   )
  // })

  // 2. ORACLE -- compare to a reference implementation
  it.todo('custom sort matches Array.prototype.sort')
  // it('custom sort matches Array.prototype.sort', () => {
  //   fc.assert(
  //     fc.property(fc.array(fc.integer()), (xs) => {
  //       const mine = mySort([...xs])
  //       const reference = [...xs].sort((a, b) => a - b)
  //       expect(mine).toEqual(reference)
  //     })
  //   )
  // })

  // 3. MODEL/STATE-MACHINE -- sequence of ops preserves invariant
  it.todo('stack push/pop preserves last-in-first-out')
  // it('stack push/pop preserves last-in-first-out', () => {
  //   fc.assert(
  //     fc.property(fc.array(fc.integer()), (items) => {
  //       const stack = new Stack<number>()
  //       for (const item of items) {
  //         stack.push(item)
  //         expect(stack.pop()).toBe(item)
  //       }
  //     })
  //   )
  // })

  // Shrinking example -- fast-check minimizes a failing input.
  // Uncomment to see: failure report will show the minimal counterexample.
  // it('intentionally fails to demonstrate shrinking', () => {
  //   fc.assert(
  //     fc.property(fc.array(fc.integer()), (xs) => {
  //       expect(xs.reduce((a, b) => a + b, 0)).toBeLessThan(1_000_000)
  //     })
  //   )
  // })
})

Go -- rapid (preferred) or testing/quick (stdlib fallback)

Recommended: pgregory.net/rapid. It provides shrinking (automatic minimization of failing inputs), stateful testing, and rich generators. testing/quick works but lacks shrinking — debugging failures on nested structs becomes painful.

// Package myapp_test contains property-based tests using pgregory.net/rapid.
//
// When to use: parsers, serializers, validators, crypto, state machines,
// financial calculators. NOT for CRUD, HTTP glue, or pure plumbing.
//
// Pattern: rapid generates random inputs and SHRINKS failures to minimal cases.
// Run: go test -run TestProperty ./...
// Install: go get -t pgregory.net/rapid
//
// For more patterns, see https://pkg.go.dev/pgregory.net/rapid
// Fallback (stdlib, no shrinking): use testing/quick — see commented section below.
package myapp_test

import (
	"testing"
	// Uncomment when you replace t.Skip() bodies with real property checks:
	// "sort"
	// "pgregory.net/rapid"
)

// 1. INVARIANT -- round-trip property
func TestPropertySerializeRoundtrip(t *testing.T) {
	t.Skip("template — replace body with real serialize/deserialize round-trip check")
	// rapid.Check(t, func(t *rapid.T) {
	//     data := rapid.MapOf(rapid.String(), rapid.Int()).Draw(t, "data")
	//     encoded := serialize(data)
	//     decoded := deserialize(encoded)
	//     if !reflect.DeepEqual(decoded, data) {
	//         t.Fatalf("roundtrip mismatch: got %v, want %v", decoded, data)
	//     }
	// })
}

// 2. ORACLE -- compare custom impl against sort.Ints
func TestPropertyCustomSortMatchesStdlib(t *testing.T) {
	t.Skip("template — replace body with real mySort vs sort.Ints comparison")
	// rapid.Check(t, func(t *rapid.T) {
	//     xs := rapid.SliceOf(rapid.Int()).Draw(t, "xs")
	//     mine := mySort(append([]int{}, xs...))
	//     reference := append([]int{}, xs...)
	//     sort.Ints(reference)
	//     if !slices.Equal(mine, reference) {
	//         t.Fatalf("sort mismatch: got %v, want %v", mine, reference)
	//     }
	// })
}

// 3. MODEL/STATE-MACHINE -- stack push/pop invariant
func TestPropertyStackPushPopLIFO(t *testing.T) {
	t.Skip("template — replace body with real stack LIFO invariant check")
	// rapid.Check(t, func(t *rapid.T) {
	//     items := rapid.SliceOf(rapid.Int()).Draw(t, "items")
	//     stack := NewStack[int]()
	//     for _, item := range items {
	//         stack.Push(item)
	//         if got := stack.Pop(); got != item {
	//             t.Fatalf("LIFO violated: push %d then pop %d", item, got)
	//         }
	//     }
	// })
}

// ── Stdlib fallback (no shrinking) ───────────────────────────────────────────
// If you cannot add rapid as a dependency, testing/quick from the stdlib works.
// Failure reports show raw generated inputs (no minimization), so debugging is
// harder — especially for maps and nested structs.
//
// import "testing/quick"
//
// func TestPropertyQuickSerializeRoundtrip(t *testing.T) {
//     f := func(data map[string]int) bool {
//         return reflect.DeepEqual(deserialize(serialize(data)), data)
//     }
//     if err := quick.Check(f, nil); err != nil {
//         t.Error(err)
//     }
// }

Rust -- proptest

//! PROPERTY TEST TEMPLATE -- Copy this file as a starting point.
//!
//! When to use: parsers, serializers, validators, crypto, state machines,
//! financial calculators. NOT for CRUD, HTTP glue, or pure plumbing.
//!
//! Pattern: proptest generates random inputs; you assert invariants.
//! Run: cargo test --test properties
//! Install: cargo add --dev proptest
//!
//! For more patterns, see https://proptest-rs.github.io/proptest/

use proptest::prelude::*;

// use std::collections::HashMap;  // uncomment when you wire up the serialize test
// use my_crate::{serialize, deserialize, my_sort};

proptest! {
    // 1. INVARIANT -- round-trip property
    // Remove #[ignore] once you wire up real serialize/deserialize.
    #[test]
    #[ignore = "template — replace body with real round-trip check"]
    fn serialize_roundtrip_is_identity(_data in prop::collection::hash_map(".*", any::<i64>(), 0..10)) {
        // let encoded = serialize(&_data);
        // let decoded: HashMap<String, i64> = deserialize(&encoded).unwrap();
        // prop_assert_eq!(decoded, _data);
    }

    // 2. ORACLE -- compare custom impl against stdlib sort
    #[test]
    #[ignore = "template — replace body with real my_sort vs stdlib comparison"]
    fn custom_sort_matches_stdlib(_xs in prop::collection::vec(any::<i32>(), 0..100)) {
        // let mut mine = _xs.clone();
        // my_sort(&mut mine);
        // let mut reference = _xs.clone();
        // reference.sort();
        // prop_assert_eq!(mine, reference);
    }

    // 3. MODEL/STATE-MACHINE -- stack LIFO invariant
    #[test]
    #[ignore = "template — replace body with real stack LIFO check"]
    fn stack_push_pop_is_lifo(_items in prop::collection::vec(any::<i32>(), 0..50)) {
        // let mut stack: Vec<i32> = Vec::new();
        // for &item in &_items {
        //     stack.push(item);
        //     prop_assert_eq!(stack.pop(), Some(item));
        // }
    }
}

Swift -- pragmatic mix (parameterized tests + custom generators)

Swift's property-testing story is weaker than other languages. The classic choice, SwiftCheck (typelift/SwiftCheck), has had no releases since 2019 and has open build failures on Swift 5.9+/6.0. Options:

Option	Status	Tradeoff
Parameterized tests (XCTest / swift-testing) + hand-rolled generators	Stable, works with toolchain	No shrinking; manual generator code
swift-gen (Point-Free)	Actively maintained	Data generation only — you write the test loop yourself
SwiftCheck	Unmaintained since 2019	Full property API but likely fails to compile on modern Swift

The template below uses option 1 (parameterized tests + custom generators) because it works without external dependencies. Upgrade to swift-gen if you need richer combinators.

// PROPERTY TEST TEMPLATE -- Copy this file as a starting point.
//
// When to use: parsers, serializers, validators, crypto, state machines,
// financial calculators. NOT for CRUD, UI glue, or pure plumbing.
//
// Pattern: XCTest with hand-rolled random generators. No external dependency.
// Run: swift test
//
// For richer combinators (without full property framework):
//   .package(url: "https://github.com/pointfreeco/swift-gen", from: "0.4.0")
// For stateful testing and advanced patterns, consider swift-testing's
// `@Test(arguments:)` parameterized tests (Swift 5.10+).

import XCTest

// @testable import MyApp

final class PropertyTests: XCTestCase {
    private let iterations = 100
    private var rng = SystemRandomNumberGenerator()

    // Hand-rolled generators — extend as needed
    private func randomInts(count: Int = Int.random(in: 0...50)) -> [Int] {
        (0..<count).map { _ in Int.random(in: -1000...1000) }
    }

    private func randomString(maxLen: Int = 32) -> String {
        let chars = "abcdefghijklmnopqrstuvwxyz "
        let len = Int.random(in: 0...maxLen)
        return String((0..<len).map { _ in chars.randomElement()! })
    }

    // 1. INVARIANT -- round-trip property
    func testSerializeRoundtripIsIdentity() throws {
        throw XCTSkip("Template — replace body with real serialize/deserialize round-trip")
        // for _ in 0..<iterations {
        //     let xs = randomInts()
        //     let encoded = serialize(xs)
        //     let decoded: [Int] = try deserialize(encoded)
        //     XCTAssertEqual(decoded, xs, "roundtrip failed for: \(xs)")
        // }
    }

    // 2. ORACLE -- compare custom impl against stdlib sort
    func testCustomSortMatchesStdlib() throws {
        throw XCTSkip("Template — replace body with real mySort vs sorted() comparison")
        // for _ in 0..<iterations {
        //     let xs = randomInts()
        //     let mine = mySort(xs)
        //     let reference = xs.sorted()
        //     XCTAssertEqual(mine, reference, "sort mismatch for: \(xs)")
        // }
    }

    // 3. MODEL/STATE-MACHINE -- stack LIFO invariant
    func testStackPushPopIsLIFO() throws {
        throw XCTSkip("Template — replace body with real stack LIFO invariant check")
        // for _ in 0..<iterations {
        //     let items = randomInts(count: Int.random(in: 0...20))
        //     var stack = Stack<Int>()
        //     for item in items {
        //         stack.push(item)
        //         XCTAssertEqual(stack.pop(), item, "LIFO violated after pushing \(items)")
        //     }
        // }
    }
}

Key points:

• Opt-in, never speculative — don't install framework deps unless the user confirms there are modules that benefit
• Placeholders follow the existing Phase A guidance — pass (Python), it.todo(...) (TypeScript/JS — marks as pending, not passing), t.Skip(...) (Go), XCTSkip(...) (Swift), commented-out assertions (Rust). Avoid always-passing placeholders like return true or expect(true).toBe(true) — they inflate pass counts
• No dangling unused imports — comment out imports alongside their usages. Templates must compile under strict settings (noUnusedLocals, -D warnings, etc.)
• Three patterns per template — invariant, oracle, state-machine — covers the high-value property categories
• Framework docs linked — developers will extend templates using framework-specific features (shrinking strategies, custom generators, stateful testing)
• Discovered by normal test runners — files matching the naming conventions (tests/test_properties*.py, __tests__/properties*.test.ts, properties_test.go, tests/properties.rs, PropertyTests.swift) are picked up by the default test commands. They run alongside example-based tests; no separate CI job needed
• NOT a PR gate — property tests are advisory signal, not required checks. Missing property tests should never block merge

A4. Post-Setup

After generating all files, verify the setup works end-to-end.

A4a. Run the Tests

Execute the test command for the detected language:

Language	Command
TypeScript/JS	npm test (or yarn test / pnpm test / bun test per detected package manager)
Python	pytest tests/
Go	go test ./...
Rust	cargo test
Swift	swift test (SPM) or xcodebuild test (Xcode)

A4b. Handle Results

Result	Action
All tests pass	Report success, show coverage baseline, proceed to 4c
Smoke test fails -- import side effects	Report: "Your app performs side effects on import (e.g., DB connections, env vars). Consider wrapping startup logic in a function. The smoke test verifies import-only."
Smoke test fails -- missing dependencies	Report: "Install missing dependencies first, then re-run Phase A."
Template test fails	Report as informational (not an error): "The template test references example endpoints/functions. Adapt it to your actual code."
Installation failed	Report the error (network, permissions, version conflict). Do not generate output files.

A4c. Report Summary

Show a summary of everything that happened:

## Phase A Complete: Test Infrastructure

**Language:** TypeScript (Express)
**Package manager:** npm

**Created:**
- vitest.config.ts (coverage: v8, reporter: lcov)
- package.json scripts: test, test:coverage
- __tests__/smoke.test.ts (1 test, passing)
- __tests__/_template.test.ts (3 tests, passing)

**Skipped:** (nothing -- full setup)

**Test results:** 4 tests passing
**Coverage baseline:** 12.3%

**Next steps:**
- Proceed to Phase B to wire CI/CD pipeline
- Phase C (CLAUDE.md) will auto-run after Phase B completes
- Use `busdriver:tdd` when ready to write tests for specific modules

---

Phase B: CI/CD Pipeline

Set up the full CI pipeline for new or existing repos: tests + coverage, action pinning, SBOM generation, build provenance attestations, security scanning backstop (Semgrep, Checkov, Zizmor), Dependabot, SSH commit signing, OpenSSF Scorecard, CodeScene behavioral analysis, and GitGuardian secrets detection.

When to Use

• Onboarding a new repo into the CI pipeline
• Adding or fixing Codecov, pinact, or GitGuardian for existing repos
• Deploying pipeline changes across multiple repos at once
• Fixing cross-platform CI failures (lightningcss, npm ci, vitest coverage, Swift iOS-only)
• Auditing CI pipeline completeness across the portfolio
• Adding SBOM generation or build provenance attestations
• Setting up SSH commit signing or troubleshooting signature issues
• Configuring Dependabot security alerts or version updates
• Deploying OpenSSF Scorecard or SECURITY.md
• Setting up CodeScene behavioral code analysis on PRs
• Adding security scanning CI backstop (Semgrep, Checkov, Zizmor) for defense-in-depth

Pipeline Components

Component	What It Does	Config Files
Codecov	Diff-coverage on PRs (80% target for new code)	codecov.yml + .github/workflows/tests.yml
Pinact	Auto-pin GitHub Actions to full SHA + precise version comments	.github/workflows/pinact.yml
GitGuardian	Secrets detection on push/PR (catches Gitleaks misses, different engine)	GitHub App (ggshield)
Syft SBOM	Generate Software Bill of Materials (dependency list)	compliance job in tests.yml
SBOM Attestation	Cryptographic SBOM provenance (GitHub Sigstore)	compliance job in tests.yml
Release Attestation	Attest source archives on GitHub Release (Sigstore)	attest job in release.yml
Trivy Vuln	Dependency vulnerability scanning (CRITICAL+HIGH)	compliance job in tests.yml
Trivy License	Dependency license compliance (CRITICAL only)	compliance job in tests.yml
LICENSE	Proprietary repo license (all rights reserved)	LICENSE file at repo root
Cosign	Keyless binary signing (forge release only)	.github/workflows/release.yml
Harden-Runner	Monitor network egress + detect code overwrite in Actions (Ubuntu only)	First step in every ubuntu job
Commitlint	Enforce Conventional Commits format (open-source repos only)	commitlint.config.js + commitlint job in tests.yml
semantic-release

Default Behavior: Audit Current Repo

When this skill is invoked without a specific task (e.g., user runs /ci-pipeline-setup or says "audit CI"), run a full checklist against the current repo. Check every component below using the exact commands shown. Present results as a table with pass/fail/N-A status.

Repo Settings (via API)

OWNER=$(gh repo view --json owner -q '.owner.login')
REPO=$(gh repo view --json name -q '.name')

# Repo settings
gh api "repos/$OWNER/$REPO" --jq '{
  allow_merge_commit, allow_squash_merge, allow_rebase_merge,
  allow_update_branch, delete_branch_on_merge, allow_auto_merge,
  visibility, default_branch
}'

# Actions permissions (also check selected_actions_url for allowlist if allowed_actions is "selected")
gh api "repos/$OWNER/$REPO/actions/permissions" --jq '{allowed_actions, sha_pinning_required}'

# Branch protection (use detected default branch, not hardcoded "main")
DEFAULT_BRANCH=$(gh repo view --json defaultBranchRef -q '.defaultBranchRef.name')
gh api "repos/$OWNER/$REPO/branches/$DEFAULT_BRANCH/protection/required_status_checks" --jq '{strict, contexts}' 2>&1

File Checks

Check	Command	Pass condition
Tests workflow	[ -f .github/workflows/tests.yml ]	File exists
Security backstop	[ -f .github/workflows/security.yml ]	File exists
Pinact workflow	[ -f .github/workflows/pinact.yml ]	File exists
Scorecard workflow	[ -f .github/workflows/scorecard.yml ]	File exists
Release workflow	[ -f .github/workflows/release.yml ]	File exists (N/A for non-release repos)
SHA pin script	[ -f .github/scripts/check-pinned-uses.sh ]	File exists
Dependabot	[ -f .github/dependabot.yml ]	File exists
Dependabot auto-merge	[ -f .github/workflows/dependabot-auto-merge.yml ]	File exists (N/A if no Dependabot config)
Scanners required	`gh api repos/$OWNER/$REPO/branches/$DEFAULT_BRANCH/protection/required_status_checks --jq '.contexts as $c	(["Actions security","Code security","Dependency CVEs","IaC misconfig"] - $c
Codecov config	See Codecov detection logic below	Three-way check
LICENSE	[ -f LICENSE ]	File exists
SECURITY.md	[ -f SECURITY.md ]	File exists
Release config	[ -f .releaserc.json ]	File exists (N/A for non-release repos)
Commitlint config	[ -f commitlint.config.js ]	File exists (N/A for non-release repos)
CodeScene config	[ -f .codescene/custom-quality-gates.json ]	File exists (N/A if CodeScene App not installed)
Property tests	Any of tests/test_properties*.py, __tests__/properties*.test.ts, properties_test.go, tests/properties.rs,

Codecov detection logic:

1. Has test script ("test" in package.json / Makefile test: target / go test / cargo test) AND coverage config (codecov.yml, .coveragerc, vitest coverage config) -> wire Codecov
2. Has source files (.ts/.tsx/.js/.jsx/.py/.go/.rs/.swift) but no test infrastructure -> Codecov | ❌ — set up test infrastructure first
3. No source files (markdown, JSON, shell only) -> Codecov | N/A

Workflow Hardening (check each workflow file)

For each .github/workflows/*.yml, verify:

Check	How to verify
timeout-minutes on every job	For each workflow, verify every job has timeout-minutes set
permissions declared	grep -L 'permissions' .github/workflows/*.yml — should return nothing
defaults.run.shell	Check defaults.run.shell: bash is declared (not just any shell: key in step-level overrides)
Concurrency group	Check push/PR workflows only (not cron-only workflows like scorecard)
Harden-Runner	For each ubuntu job (not macOS), verify harden-runner step exists. Check per-job, not per-file
SHA-pinned actions	bash .github/scripts/check-pinned-uses.sh — exit 0 = pass
No paths + paths-ignore on same trigger	Verify no workflow uses both paths and paths-ignore on the same trigger event (GitHub ignores paths-ignore when paths is present)
persist-credentials: false	Check all checkout steps except release/pinact (which need push access)

Content Checks (grep inside files)

Check	Command	Pass condition
Compliance job (SBOM+license+vuln)	grep -q 'sbom-action' .github/workflows/tests.yml	Found (N/A for no-dep repos)
Trivy in compliance	grep -q 'trivy-action|scanners.*vuln' .github/workflows/tests.yml	Found (N/A for no-dep repos)
Commitlint job	grep -q 'commitlint' .github/workflows/tests.yml	Found (N/A for non-release repos)
Semgrep in security.yml	grep -q 'semgrep' .github/workflows/security.yml	Found
Checkov in security.yml	grep -q 'checkov' .github/workflows/security.yml	Found
Zizmor in security.yml	grep -q 'zizmor' .github/workflows/security.yml	Found
Trivy vuln scan	grep -q 'trivy-action' .github/workflows/tests.yml OR grep -q 'trivy' .github/workflows/security.yml	Found in either (Trivy runs in compliance job in tests.yml; security.yml auto-skips if compliance exists)
Reports summary job	grep -q 'GITHUB_STEP_SUMMARY' .github/workflows/security.yml	Found
Artifact retention set	grep -q 'retention-days' .github/workflows/scorecard.yml	Found where upload-artifact is used

Present Results

Show a summary table:

| Component | Status | Notes |
|-----------|--------|-------|
| Repo: squash-only merge | ✅/❌ | |
| Repo: auto-merge | ✅/❌ | |
| Repo: branch protection | ✅/❌ | contexts: [...] |
| Repo: Actions selected | ✅/❌ | |
| Repo: SHA pinning required | ✅/❌ | |
| Tests workflow | ✅/❌ | |
| Security backstop | ✅/❌ | semgrep+checkov+zizmor+trivy+reports |
| Pinact | ✅/❌ | |
| Scorecard | ✅/❌ | |
| Release | ✅/❌/N-A | |
| Dependabot | ✅/❌ | ecosystems: [...] |
| Codecov | ✅/❌/N-A | |
| Compliance (SBOM+vuln+license) | ✅/❌/N-A | |
| Commitlint | ✅/❌/N-A | |
| LICENSE | ✅/❌ | |
| SECURITY.md | ✅/❌ | |
| SHA pin script | ✅/❌ | |
| Harden-Runner (all ubuntu jobs) | ✅/❌ | |
| Workflow hardening | ✅/❌ | timeouts, permissions, shell, concurrency |
| Commit signing | ✅/❌ | SSH or GPG |
| CodeScene | ✅/❌/N-A | quality gates config |

After showing results, suggest fixes for any ❌ items referencing the specific section in this skill.

---

B1. Process

B1a. Detect Stack

cd /path/to/repo  # <- Replace with the actual repository path
ls package.json pyproject.toml go.mod Cargo.toml Package.swift build.gradle.kts pom.xml 2>/dev/null

B1b. Configure Repo Settings (before deploying workflows)

Configure repo settings via API BEFORE deploying workflows. Without this, allowed_actions: "local_only" causes silent startup_failure on all workflows using external actions.

Important: If the org controls Actions permissions, repo-level API calls return 409 Conflict. Check org-level first: gh api orgs/ORG/actions/permissions/selected-actions. If org-level is set, modify it there instead of per-repo.

OWNER="owner"
REPO="repo"

# ── Repo settings (merge, auto-merge, branch cleanup) ──
gh api "repos/$OWNER/$REPO" -X PATCH \
  -F allow_merge_commit=false \
  -F allow_squash_merge=true \
  -F allow_rebase_merge=false \
  -F allow_update_branch=true \
  -F delete_branch_on_merge=true \
  -F allow_auto_merge=true \
  --silent

# ── Actions permissions ──
# Set to "selected" — allows github-owned + specific third-party
gh api "repos/$OWNER/$REPO/actions/permissions" -X PUT \
  -f allowed_actions=selected -F enabled=true -F sha_pinning_required=true

# Workflow permissions: try to enable "Allow GitHub Actions to create and
# approve PRs" (default is false). Required for `dependabot-auto-merge.yml`'s
# `hmarr/auto-approve-action` step to post the approving review that satisfies
# branch protection's required_pull_request_reviews. If this returns 409
# Conflict an enterprise admin has disabled GitHub Actions PR approval at the
# enterprise level — that's fine, the workflow falls back to annotate-only
# mode (the script handles this branch explicitly below; other failure modes
# like auth/network/typo errors abort the script instead of silently falling
# through).
#
# Value comparison in the workflow's `if:` is case-sensitive — the var must
# be the exact string `true` to enable approve+auto-merge.
PUT_STDERR=$(mktemp)
trap 'rm -f "$PUT_STDERR"' EXIT
if gh api "repos/$OWNER/$REPO/actions/permissions/workflow" -X PUT \
     --input - >/dev/null 2>"$PUT_STDERR" <<'EOF'
{
  "default_workflow_permissions": "read",
  "can_approve_pull_request_reviews": true
}
EOF
then
  # PUT succeeded → repo CAN have GitHub Actions approve PRs → opt in.
  gh variable set DEPENDABOT_AUTO_APPROVE --body "true" --repo "$OWNER/$REPO"
  echo "Dependabot auto-merge: OPTED IN (full hmarr + auto-merge workflow)"
elif grep -qE "409|Conflict|not allow|not permit|disabled" "$PUT_STDERR"; then
  # PUT explicitly rejected by org/enterprise policy → annotate-only mode.
  # ALSO unset any pre-existing DEPENDABOT_AUTO_APPROVE so re-running the
  # setup on a previously-opted-in repo correctly downgrades to annotate-only
  # if the policy changed (idempotent re-run). Tolerate "not found" (404)
  # but surface real errors (auth/network) so a failed delete cannot silently
  # leave stale opt-in state on a repo that's now enterprise-blocked.
  DEL_STDERR=$(mktemp)
  if gh variable delete DEPENDABOT_AUTO_APPROVE --repo "$OWNER/$REPO" 2>"$DEL_STDERR"; then
    : # deleted successfully
  elif grep -qE "404|Not Found|not found" "$DEL_STDERR"; then
    : # variable didn't exist → nothing to clear, idempotent no-op
  else
    echo "ERROR: failed to clear DEPENDABOT_AUTO_APPROVE for annotate-only mode:" >&2
    cat "$DEL_STDERR" >&2
    rm -f "$DEL_STDERR"
    exit 1
  fi
  rm -f "$DEL_STDERR"
  echo "Dependabot auto-merge: ANNOTATE-ONLY (enterprise blocks GitHub Actions PR approval; safe bumps merged manually)"
else
  # Any OTHER failure (auth, network, typo'd OWNER/REPO, unknown server error)
  # is a real error — surface it instead of silently treating as annotate-only.
  echo "ERROR: gh api PUT actions/permissions/workflow failed for unexpected reason:" >&2
  cat "$PUT_STDERR" >&2
  exit 1
fi

# Allowlist: github-owned always + specific third-party patterns
gh api "repos/$OWNER/$REPO/actions/permissions/selected-actions" -X PUT \
  --input - <<'EOF'
{
  "github_owned_allowed": true,
  "verified_creator_allowed": false,
  "patterns_allowed": [
    "step-security/harden-runner@*",
    "ossf/scorecard-action@*",
    "suzuki-shunsuke/pinact-action@*",
    "aquasecurity/trivy-action@*",
    "dependabot/fetch-metadata@*",
    "hmarr/auto-approve-action@*"
  ]
}
EOF

# ── Branch protection (required for safe auto-merge) ──
# Without this, auto-merge merges PRs immediately without waiting for CI.
# IMPORTANT: Run this AFTER deploying workflows and triggering at least one run.
# GitHub requires a check to have at least one recorded run before accepting it as required.

# Step 1: Discover actual check names from a recent workflow run.
# GitHub uses the `name:` field as the check name; falls back to the job key if no name is set.
ALL_CHECKS=$(gh api "repos/$OWNER/$REPO/commits/$(git rev-parse HEAD)/check-runs" \
  --jq '.check_runs[].name' | sort -u)
echo "Discovered checks:"
echo "$ALL_CHECKS"

# Step 2: Categorize checks.
# Default policy: test + lint + security scanners → required; external AI bots → advisory.
# Required (recommended baseline):
#   - Test/lint:   "test", "test (ubuntu-latest)", "commitlint", "version-drift"
#   - Security:    "Actions security" (zizmor), "Code security" (semgrep),
#                  "Dependency CVEs" (trivy), "IaC misconfig" (checkov)
# Advisory (recommended NOT required):
#   - External AI: "CodeRabbit", "Greptile Review", "cubic · AI code reviewer"
#                  (third-party availability — making them required causes
#                  PRs to hang when the service is slow/unavailable)
#   - "CodeScene": treated as advisory by convention (architectural signal,
#                  not a correctness gate)
#
# Why scanners are required by default (added 2026-05-07):
# Without scanners as required checks, a Dependabot bump of a vulnerable
# npm/pip dep can merge on `test` pass alone — Trivy can flag the CVE on the
# PR but it doesn't block merge unless required. Same applies to manual
# clicks of "Enable auto-merge" on a Dependabot PR with a failing scanner.
# Skipped jobs (helmet's job-level `if:` skip pattern in security.yml) still
# count as passing, so requiring these does not introduce false-positive
# blockage on PRs that don't touch security-relevant files.
#
# Present the discovered list to the user for confirmation before applying.
# User confirms which checks should be required.

# Step 3: Apply via API.
# Build the contexts JSON array from confirmed required checks using jq (handles escaping).
# Adjust the per-repo test names ("test (ubuntu-latest)", etc.) — the security
# scanner names are the same across all helmet-onboarded repos.
CONTEXTS=$(jq -nc '$ARGS.positional' --args \
  "test (ubuntu-latest)" "test (macos-latest)" "commitlint" \
  "Actions security" "Code security" "Dependency CVEs" "IaC misconfig")

jq -n --argjson ctx "$CONTEXTS" \
  '{required_status_checks:{strict:true,contexts:$ctx},enforce_admins:false,required_pull_request_reviews:null,restrictions:null}' \
  | gh api "repos/$OWNER/$REPO/branches/$DEFAULT_BRANCH/protection" -X PUT --input -

# ── Verify ──
gh api "repos/$OWNER/$REPO" --jq '{allow_merge_commit, allow_squash_merge, allow_rebase_merge, allow_update_branch, delete_branch_on_merge, allow_auto_merge}'
gh api "repos/$OWNER/$REPO/actions/permissions" --jq '{allowed_actions, sha_pinning_required}'
gh api "repos/$OWNER/$REPO/branches/$DEFAULT_BRANCH/protection/required_status_checks" --jq '{strict, contexts}'

Add patterns for any additional third-party actions the repo uses (e.g., anchore/sbom-action@*, aquasecurity/trivy-action@*, codecov/codecov-action@*).

Branch protection notes:

• strict: true requires the PR branch to be up-to-date with main before merging (works with allow_update_branch)
• Job name resolution: GitHub uses the name: field if present, otherwise the YAML job key. Example: a job key zizmor with name: Actions security appears as "Actions security" in status checks. Always verify with gh api "repos/$OWNER/$REPO/commits/<sha>/check-runs" --jq '.check_runs[].name' | sort -u
• Required check prerequisite: each check must have at least one recorded run on the repository for GitHub to accept it as a required check. Run workflows at least once before setting protection
• enforce_admins: false lets repo owner bypass (solo dev escape hatch). Set true for team repos
• required_pull_request_reviews: null skips review requirement — solo dev doesn't need self-approval. For repos that DO require reviews, dependabot-auto-merge.yml satisfies the gate by setting can_approve_pull_request_reviews: true on the repo's workflow permissions (configured above): the workflow's hmarr/auto-approve-action posts an approving review as github-actions[bot]. On enterprise-restricted repos where that setting is blocked at the enterprise level (PUT returns 409 Conflict), the workflow runs in annotate-only mode — the operator does not set vars.DEPENDABOT_AUTO_APPROVE, so the approve + auto-merge steps skip cleanly and only the manual-review comment fires for major / production-direct minor bumps. Safe bumps on annotate-only repos must be merged manually.
• bypass_pull_request_allowances: { apps: ["dependabot"] } on branch protection was previously documented as a fallback for enterprise-restricted repos, but does NOT work in practice: GitHub auto-merge evaluates branch protection from the perspective of the user that enabled auto-merge (github-actions[bot] from this workflow's gh pr merge --auto), not the PR author. The bypass list contains dependabot, not github-actions[bot], so bypass never applies. Empirically confirmed on Dive-And-Dev/perch#38 (2026-05-08): all CI green, bypass set, PR sat BLOCKED indefinitely. Do not rely on this mechanism.
• Without branch protection, allow_auto_merge merges immediately with no checks — always pair them
• Renaming a job's name: field silently breaks required check enforcement — the old name stays in branch protection but no longer matches any check, causing PRs to hang. Always update branch protection when renaming job display names

B2. Workflow Hardening (apply to ALL workflows)

Every workflow MUST include these patterns. Apply before deploying any component.

Mandatory in every workflow file:

# 1. Path filtering — skip CI for docs-only changes (push only)
# ⚠️ Do NOT add paths-ignore to pull_request when this workflow is a required
# status check — GitHub leaves the check "Pending" on filtered-out PRs, blocking merge.
on:
  push:
    branches: [main]
    # WARNING: If you later add `paths:` to this trigger, remove `paths-ignore` —
    # do not combine both. See audit checks for details.
    paths-ignore:
      - '**/*.md'
      - 'docs/**'
      - 'LICENSE'
  pull_request:
    # No paths-ignore here — required status checks must always run

# 2. Concurrency — cancel stale runs on same PR
concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true

# 3. Top-level deny-all — grant per-job only
permissions: {}

# 4. Explicit shell
defaults:
  run:
    shell: bash

Mandatory on every job:

jobs:
  example:
    runs-on: ubuntu-latest
    timeout-minutes: 10  # Always set — no runaway jobs

Security workflow uses paths (not paths-ignore) — only triggers when security-relevant files change:

on:
  pull_request:
    paths:
      - '.github/**'
      - '**/*.sh'
      - '**/*.js'
      - '**/*.py'
      - '**/*.yml'
      - 'package.json'
      - '**/package-lock.json'
      - '**/pnpm-lock.yaml'
      - '**/yarn.lock'
      - '**/go.sum'
      - '**/requirements*.txt'
      - '**/Dockerfile'
      - '**/*.tf'
      - '**/pyproject.toml'
      - '**/uv.lock'
      - '**/Cargo.lock'
      - '**/Package.resolved'
      # ⚠️ Do NOT add paths-ignore here — combining paths + paths-ignore
      # on the same trigger is undefined behavior in GitHub Actions.

SHA pin verification script (.github/scripts/check-pinned-uses.sh):

Extract the inline grep to a reusable script. Handles quoted uses: values, local action refs, and docker:// refs:

#!/usr/bin/env bash
set -euo pipefail
status=0
while IFS= read -r -d '' file; do
  while IFS= read -r raw; do
    line_no="${raw%%:*}"
    line="${raw#*:}"
    ref="$(printf '%s' "$line" \
      | sed -E "s/^[[:space:]]*uses:[[:space:]]*//; s/[[:space:]]+#.*$//; s/[[:space:]].*$//; s/^['\"]//; s/['\"]$//")"
    case "$ref" in
      ./*|docker://*) continue ;;  # Local refs + docker:// exempt
    esac
    if [[ ! "$ref" =~ ^[^@]+@[0-9a-f]{40}$ ]]; then
      echo "::error file=$file,line=$line_no::Unpinned or invalid action/workflow ref: $ref"
      status=1
    fi
  done < <(grep -nE '^[[:space:]]*uses:[[:space:]]*[^[:space:]]+@[^[:space:]]+' "$file" || true)
done < <(find .github/workflows -type f \( -name '*.yml' -o -name '*.yaml' \) -print0 2>/dev/null; \
         find .github/actions  -type f \( -name '*.yml' -o -name '*.yaml' \) -print0 2>/dev/null || true)
exit $status

Reports summary job — add to security.yml for PR summary. Requires the changes detection job from Section N. Security Scanning Backstop:

  reports:
    if: always()
    needs: [changes, trivy, semgrep, checkov, zizmor]
    runs-on: ubuntu-latest
    timeout-minutes: 2
    steps:
      - name: Write summary
        env:
          TRIVY: ${{ needs.trivy.result }}
          SEMGREP: ${{ needs.semgrep.result }}
          CHECKOV: ${{ needs.checkov.result }}
          ZIZMOR: ${{ needs.zizmor.result }}
        run: |
          {
            echo "## Security Backstop"
            echo "| Scanner | Result |"
            echo "|---------|--------|"
            echo "| trivy | $TRIVY |"
            echo "| semgrep | $SEMGREP |"
            echo "| checkov | $CHECKOV |"
            echo "| zizmor | $ZIZMOR |"
          } >> "$GITHUB_STEP_SUMMARY"
          if [[ "$TRIVY" == "failure" || "$SEMGREP" == "failure" || "$CHECKOV" == "failure" || "$ZIZMOR" == "failure" ]]; then
            echo "::error::One or more security scanners failed"
            exit 1
          fi

pull_request over pull_request_target — never use pull_request_target with untrusted code. All workflows use pull_request.

B3. Deploy Components

Deploy in this order — each is independent:

A. Tests + Coverage (Codecov)

Workflow — use tailored workflow per repo, NOT the universal template:

TypeScript/JavaScript (vitest)

steps:
  - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
    with:
      persist-credentials: false
  - uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
    with:
      node-version: 20
      cache: npm
  - run: npm install @vitest/coverage-v8
  - run: npx vitest run --coverage
  - uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
    with:
      token: ${{ secrets.CODECOV_TOKEN }}

Python (pytest)

steps:
  - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
    with:
      persist-credentials: false
  - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
    with:
      python-version: "3.12"
      cache: pip
  - run: pip install ".[dev]" pytest-cov
  - run: pytest --cov --cov-report=xml
  - uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
    with:
      token: ${{ secrets.CODECOV_TOKEN }}

Rust (cargo-llvm-cov)

steps:
  - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
    with:
      persist-credentials: false
  - uses: dtolnay/rust-toolchain@efa25f7f19611383d5b0ccf2d1c8914531636bf9 # stable
    with:
      toolchain: stable
      components: llvm-tools-preview
  - uses: Swatinem/rust-cache@82a92a6e8fbeee089604da2575dc567ae9ddeaaf # v2.7.5
  - uses: taiki-e/install-action@0c48e7d0d41c6c13ecc8a3a78cda3882cb2e9464 # v2.52.4
  - run: cargo llvm-cov --lcov --output-path lcov.info
  - uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
    with:
      token: ${{ secrets.CODECOV_TOKEN }}
      files: lcov.info

Key Rust optimizations:

• Swatinem/rust-cache caches ~/.cargo and target/ — turns 5+ min rebuilds into ~30s
• taiki-e/install-action@cargo-llvm-cov downloads pre-built binary (vs cargo install compiling from source, saving 2-3 min)
• components: llvm-tools-preview on toolchain is required by cargo-llvm-cov

Swift (iOS — xcodebuild)

runs-on: macos-latest  # ⚠️ 10x cost multiplier vs ubuntu-latest
steps:
  - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
    with:
      persist-credentials: false
  - run: |
      xcodebuild test \
        -scheme TestTargetName \
        -destination 'platform=iOS Simulator,name=iPhone 16' \
        -enableCodeCoverage YES
  - uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
    with:
      token: ${{ secrets.CODECOV_TOKEN }}

Go

steps:
  - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
    with:
      persist-credentials: false
  - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
    with:
      go-version-file: go.mod  # reads 'go' directive; fallback: go-version: "stable"
      cache: true
  - run: go test -coverprofile=coverage.out ./...
  - uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
    with:
      token: ${{ secrets.CODECOV_TOKEN }}

Codecov config (codecov.yml at repo root):

coverage:
  status:
    project: off          # No global coverage vanity metrics
    patch:
      default:
        target: 80%       # New code must have 80% coverage
        threshold: 5%
comment:
  layout: "condensed_header, diff, flags, components"
  behavior: default
  require_changes: true

Secret: After wiring Codecov, auto-set the repo secret from the CODECOV_TOKEN environment variable:

if [ -n "${CODECOV_TOKEN:-}" ]; then
  echo "$CODECOV_TOKEN" | gh secret set CODECOV_TOKEN --repo "$OWNER/$REPO"
  echo "✅ CODECOV_TOKEN secret set from environment"
else
  echo "⚠️  CODECOV_TOKEN env var not set — set it manually: gh secret set CODECOV_TOKEN --repo $OWNER/$REPO"
fi

B. Action Pinning (Pinact)

Workflow (.github/workflows/pinact.yml):

name: Auto-pin GitHub Actions
on:
  push:
    branches: [main]  # Change to match repo default branch
    paths:
      - '.github/workflows/**'

permissions: {}

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

defaults:
  run:
    shell: bash

jobs:
  pin:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: suzuki-shunsuke/pinact-action@cf51507d80d4d6522a07348e3d58790290eaf0b6 # v2.0.0
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}

Local pre-push: Run pinact run --fix .github/workflows/*.yml before pushing to avoid CI failures from imprecise version comments.

Key gotchas:

• branches: must match repo's actual default branch (main vs master)
• Do NOT use persist-credentials: false on checkout — pinact needs push access for auto-fix commits
• Version comments must be precise (# v4.2.2 not # v4) — pinact enforces this

C. Compliance (SBOM + License + Vulnerability — combined single job)

Combine SBOM generation, license compliance, and vulnerability scanning into a single compliance job. This saves 2 billable minutes per push (3 jobs → 1), since each GitHub Actions job is billed minimum 1 minute even if it runs in 15 seconds.

  compliance:
    if: github.event_name == 'push'
    needs: test  # or 'build' if the first job is named 'build'
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - name: Harden Runner
        uses: step-security/harden-runner@fa2e9d605c4eeb9fcad4c99c224cee0c6c7f3594 # v2.16.0
        with:
          egress-policy: audit
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
        with:
          persist-credentials: false
      - name: Generate SBOM
        uses: anchore/sbom-action@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
        with:
          format: spdx-json
          output-file: sbom.spdx.json
      - name: Trivy license scan
        uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1 # v0.35.0
        with:
          scan-type: fs
          scanners: license
          severity: CRITICAL
          exit-code: 1
      - name: Trivy vulnerability scan