Provides release management strategies including deployment patterns, version control, and rollback procedures. Use when planning releases, managing deployments, or when user mentions 'release', 'canary', 'blue-green', 'rollback', 'feature flag', 'release train', 'semantic versioning', 'changelog', 'migration'.
Provides release management strategies including deployment patterns, version control, and rollback procedures. Use when planning releases, managing deployments, or when user mentions 'release', 'canary', 'blue-green', 'rollback', 'feature flag', 'release train', 'semantic versioning', 'changelog', 'migration'.
type
skill
category
ops
status
stable
origin
tibsfox
modified
false
first_seen
"2026-02-07T00:00:00.000Z"
first_path
examples/release-management/SKILL.md
superseded_by
null
Release Management
Best practices for shipping software reliably through structured release processes, deployment strategies, and rollback automation.
Deployment Strategy Comparison
Choosing the right deployment strategy depends on risk tolerance, infrastructure budget, and rollback requirements.
Strategy
Zero Downtime
Rollback Speed
Infrastructure Cost
Complexity
Best For
Blue-Green
Yes
Instant (switch)
2x
Medium
Critical services, compliance
Canary
Yes
Fast (route away)
1.05-1.5x
High
High-traffic user-facing apps
Rolling
Yes
Slow (re-deploy)
1x
Low
Stateless microservices
Recreate
No (brief outage)
Slow (re-deploy)
1x
Low
Dev/staging, batch jobs
A/B Testing
Yes
Fast (route away)
1.2-1.5x
High
Feature experimentation
Shadow/Dark
Yes
N/A (no user impact)
1.5-2x
Very High
ML models, data pipelines
Decision Matrix
Is the service stateful?
YES --> Can you afford 2x infrastructure?
YES --> Blue-Green
NO --> Rolling (with drain + health checks)
NO --> Is it high-traffic (>1000 rps)?
YES --> Canary (gradual rollout)
NO --> Rolling (simple, cost-effective)
|
histogram_quantile(0.99,
sum(rate(http_request_duration_ms_bucket{
service="{{args.service-name}}",
canary="true"
}[2m])) by (le)
)
Feature Flag Implementation
LaunchDarkly Pattern with Gradual Rollout
// feature-flags.ts -- Structured feature flag managementimport * asLaunchDarklyfrom"@launchdarkly/node-server-sdk";
// --- Flag Configuration Types ---interfaceFlagConfig {
key: string;
description: string;
type: "boolean" | "multivariate" | "percentage";
owner: string; // Team responsiblecreatedAt: string; // For cleanup trackingmaxAge: string; // Expected lifetime: "temporary" | "permanent"cleanupTicket?: string; // JIRA ticket to remove flag
}
// Registry prevents flag sprawl -- all flags must be declaredconstFLAG_REGISTRY: Record<string, FlagConfig> = {
"new-checkout-flow": {
key: "new-checkout-flow",
description: "Redesigned checkout with one-page form",
type: "percentage",
owner: "checkout-team",
createdAt: "2025-11-01",
maxAge: "temporary",
cleanupTicket: "CHECKOUT-1234",
},
"payment-v2-api": {
key: "payment-v2-api",
description: "Route payments through v2 processor",
type: "boolean",
owner: "payments-team",
createdAt: "2025-10-15",
maxAge: "temporary",
cleanupTicket: "PAY-567",
},
};
// --- Client Initialization ---letldClient: LaunchDarkly.LDClient;
exportasyncfunctioninitFeatureFlags(): Promise<void> {
ldClient = LaunchDarkly.init(process.env.LAUNCHDARKLY_SDK_KEY!);
await ldClient.waitForInitialization({ timeout: 10 });
console.log("LaunchDarkly client initialized");
}
// --- Flag Evaluation with Fallback ---exportasyncfunctionisEnabled(flagKey: string,
context: LaunchDarkly.LDContext,
defaultValue = false): Promise<boolean> {
// Validate flag is registeredif (!FLAG_REGISTRY[flagKey]) {
console.warn(`Unknown flag: ${flagKey}. Returning default.`);
return defaultValue;
}
try {
const value = await ldClient.variation(flagKey, context, defaultValue);
returnBoolean(value);
} catch (err) {
// Flag evaluation failure should never break the appconsole.error(`Flag evaluation failed for ${flagKey}:`, err);
return defaultValue;
}
}
// --- Usage in Route Handler ---exportasyncfunctioncheckoutHandler(req: Request, res: Response) {
constuserContext: LaunchDarkly.LDContext = {
kind: "user",
key: req.user.id,
email: req.user.email,
custom: {
plan: req.user.plan, // Target by plan tierregion: req.user.region, // Target by geographycompany: req.user.orgId, // Target by organization
},
};
const useNewCheckout = awaitisEnabled("new-checkout-flow", userContext);
if (useNewCheckout) {
returnrenderNewCheckout(req, res);
}
returnrenderLegacyCheckout(req, res);
}
// --- Stale Flag Detection ---exportfunctiongetStaleFlags(maxAgeDays = 90): FlagConfig[] {
const now = Date.now();
returnObject.values(FLAG_REGISTRY).filter((flag) => {
if (flag.maxAge === "permanent") returnfalse;
const age = now - newDate(flag.createdAt).getTime();
return age > maxAgeDays * 24 * 60 * 60 * 1000;
});
}
Database Migration Strategy
Safe Migration Workflow
Database migrations during releases require special care because they cannot be rolled back as easily as code.
Migration Type
Risk Level
Rollback Strategy
Requires Downtime
Add column (nullable)
Low
Drop column
No
Add column (NOT NULL + default)
Medium
Drop column
No (with care)
Drop column
High
Cannot undo
No (expand-contract)
Rename column
High
Cannot undo easily
No (expand-contract)
Add index
Medium
Drop index
No (CONCURRENTLY)
Change column type
High
Revert type
Sometimes
Add table
Low
Drop table
No
Drop table
Critical
Restore from backup
No
Expand-Contract Migration Pattern
-- Phase 1: EXPAND (deploy with old code still running)-- Add new column alongside old oneALTER TABLE users ADDCOLUMN email_normalized VARCHAR(255);
-- Backfill data (run as background job, not in migration)UPDATE users SET email_normalized =LOWER(TRIM(email))
WHERE email_normalized ISNULL
LIMIT 10000; -- Batch to avoid locking-- Phase 2: MIGRATE (deploy new code that writes to both columns)-- Application writes to BOTH email and email_normalized-- Application reads from email_normalized with fallback to email-- Phase 3: CONTRACT (after all code uses new column)-- Only after verifying no code reads the old columnALTER TABLE users DROPCOLUMN email;
ALTER TABLE users RENAME COLUMN email_normalized TO email;
Migration Runner with Safety Checks
#!/usr/bin/env bash# migrate.sh -- Safe migration runner with pre-flight checksset -euo pipefail
DB_NAME="${DB_NAME:?DB_NAME required}"
MIGRATION_DIR="${MIGRATION_DIR:-./migrations}"
DRY_RUN="${DRY_RUN:-false}"# Pre-flight checkspreflight_check() {
echo"=== Pre-flight checks ==="# 1. Check for pending transactions that could block
BLOCKED=$(psql -d "$DB_NAME" -t -c \
"SELECT count(*) FROM pg_stat_activity
WHERE state = 'idle in transaction'
AND query_start < now() - interval '5 minutes';")
if [ "$BLOCKED" -gt 0 ]; thenecho"ERROR: $BLOCKED long-running idle transactions detected"echo"These may block DDL operations. Investigate before proceeding."exit 1
fi# 2. Check disk space (migrations can temporarily double table size)
DISK_FREE=$(df -BG /var/lib/postgresql | tail -1 | awk '{print $4}' | tr -d 'G')
if [ "$DISK_FREE" -lt 20 ]; thenecho"ERROR: Only ${DISK_FREE}GB free. Migrations may need more space."exit 1
fi# 3. Verify backup is recent (within last hour)
LAST_BACKUP=$(psql -d "$DB_NAME" -t -c \
"SELECT pg_last_xact_replay_timestamp();" 2>/dev/null || echo"N/A")
echo"Last backup/replica sync: $LAST_BACKUP"echo"=== Pre-flight passed ==="
}
# Run migrationsrun_migrations() {
for migration in"$MIGRATION_DIR"/*.sql; do
MIGRATION_NAME=$(basename"$migration")
# Check if already applied
APPLIED=$(psql -d "$DB_NAME" -t -c \
"SELECT count(*) FROM schema_migrations
WHERE name = '$MIGRATION_NAME';")
if [ "$APPLIED" -gt 0 ]; thenecho"SKIP: $MIGRATION_NAME (already applied)"continuefiecho"APPLYING: $MIGRATION_NAME"if [ "$DRY_RUN" = "true" ]; thenecho" DRY RUN -- would execute:"head -20 "$migration"continuefi# Apply with statement timeout to prevent long locks
psql -d "$DB_NAME" \
-v ON_ERROR_STOP=1 \
-c "SET statement_timeout = '30s';" \
-f "$migration"# Record migration
psql -d "$DB_NAME" -c \
"INSERT INTO schema_migrations (name, applied_at)
VALUES ('$MIGRATION_NAME', now());"echo" APPLIED: $MIGRATION_NAME"done
}
preflight_check
run_migrations
echo"=== Migrations complete ==="
Rollback Automation
Automated Rollback Script
#!/usr/bin/env bash# rollback.sh -- Automated release rollback with verificationset -euo pipefail
SERVICE="${1:?Usage: rollback.sh <service> [target-version]}"
TARGET_VERSION="${2:-}"# If empty, rolls back to previous
ENVIRONMENT="${ENVIRONMENT:-production}"
NAMESPACE="${NAMESPACE:-production}"# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'log() { echo -e "${GREEN}[ROLLBACK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARNING]${NC} $*"; }
err() { echo -e "${RED}[ERROR]${NC} $*" >&2; }
# Step 1: Determine rollback targetif [ -z "$TARGET_VERSION" ]; then# Get previous revision from Kubernetes
TARGET_VERSION=$(kubectl rollout history"deployment/$SERVICE" \
-n "$NAMESPACE" | tail -3 | head -1 | awk '{print $1}')
log"Auto-detected previous revision: $TARGET_VERSION"fi# Step 2: Create rollback record (audit trail)
ROLLBACK_ID="rb-$(date +%Y%m%d-%H%M%S)-${SERVICE}"log"Rollback ID: $ROLLBACK_ID"# Step 3: Notify team
curl -s -X POST "$SLACK_WEBHOOK_URL" \
-H 'Content-Type: application/json' \
-d "{
\"text\": \"ROLLBACK INITIATED: ${SERVICE} in ${ENVIRONMENT}\",
\"blocks\": [{
\"type\": \"section\",
\"text\": {
\"type\": \"mrkdwn\",
\"text\": \"*Rollback:* ${ROLLBACK_ID}\n*Service:* ${SERVICE}\n*Target:* revision ${TARGET_VERSION}\n*Initiated by:* $(whoami)\"
}
}]
}" 2>/dev/null || warn "Slack notification failed (non-blocking)"# Step 4: Execute rollbacklog"Rolling back $SERVICE to revision $TARGET_VERSION..."
kubectl rollout undo "deployment/$SERVICE" \
-n "$NAMESPACE" \
--to-revision="$TARGET_VERSION"# Step 5: Wait for rolloutlog"Waiting for rollback to complete..."if ! kubectl rollout status "deployment/$SERVICE" \
-n "$NAMESPACE" --timeout=300s; then
err "Rollback did not complete within 5 minutes"
err "Manual intervention required"exit 1
fi# Step 6: Verify healthlog"Verifying service health..."sleep 10 # Allow metrics to settle
HEALTH_URL="https://${SERVICE}.${ENVIRONMENT}.example.com/healthz"
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}""$HEALTH_URL" || echo"000")
if [ "$HTTP_STATUS" != "200" ]; then
err "Health check failed: HTTP $HTTP_STATUS"
err "Service may need manual investigation"exit 1
filog"Health check passed (HTTP $HTTP_STATUS)"log"Rollback $ROLLBACK_ID completed successfully"
Release Train Schedule
A release train ships on a fixed cadence regardless of what features are ready. Features that miss the train wait for the next one.
Cadence Options
Cadence
Suitable For
Trade-offs
Daily
SaaS, internal tools
Fast feedback, high automation needed
Weekly
B2B products
Balanced pace, manageable testing
Bi-weekly
Regulated industries
More testing time, slower delivery
Monthly
Enterprise, on-prem
Maximum stability, slow feedback
Weekly Release Train Example
Monday | Feature freeze for this week's release
Code complete -- all PRs merged to release branch
Automated regression suite runs
Tuesday | QA validation day
Manual exploratory testing on staging
Performance benchmarks compared to baseline
Wednesday | Release candidate tagged (e.g., v2.4.0-rc.1)
Deploy to pre-production / canary
Stakeholder sign-off window opens
Thursday | Production deploy (canary -> full rollout)
Monitoring period (4 hours minimum)
Incident response team on standby
Friday | Retrospective and metrics review
Hotfix window (if needed)
No new releases on Friday afternoon
