| name | data-retention |
| description | Design data retention policies covering deletion schedules, archival tiers, compliance requirements, and right-to-erasure workflows. Outputs retention schedules, automated deletion jobs, anonymization strategies, and audit trails. |
| argument-hint | ["data types","compliance requirements (GDPR/CCPA/HIPAA)","storage costs","query requirements for old data"] |
| allowed-tools | Read, Write, Bash |
Data Retention
Data retention balances legal compliance, storage cost, and query requirements. Too little retention violates business needs; too much creates compliance risk and storage cost. Every piece of data needs an explicit retention policy.
Process
- Inventory all data — what data exists, where it's stored, who owns it.
- Classify by sensitivity — PII, financial, operational, logs, analytics.
- Determine legal requirements — GDPR (3 years financial, erasure on request), HIPAA (6 years), CCPA.
- Define retention periods — by data type, not one-size-fits-all.
- Design archival tiers — hot (active DB) → warm (compressed archive) → cold (S3 Glacier).
- Automate deletion — manual processes are forgotten; schedule deletion jobs.
- Implement right-to-erasure — GDPR Article 17 requires deletion within 30 days of request.
- Audit the audit trail — deletion logs must themselves be retained.
Output Format
Retention Policy Definition
version: "1.0"
last_reviewed: "2024-01-15"
next_review: "2025-01-15"
data_categories:
user_accounts:
description: "User registration data, preferences, authentication"
contains_pii: true
retention_period: "active + 3 years after account deletion"
legal_basis: "GDPR Art. 6(1)(b) - contract; 7-year financial retention"
storage: [postgres, s3]
deletion_method: anonymize
right_to_erasure: true
notes: "Retain transaction records 7 years for tax; anonymize PII"
order_records:
description: "Purchase history, invoices, shipping details"
contains_pii: true
retention_period: "7 years from transaction date"
legal_basis: "Tax law requires 7-year financial record retention"
storage: [postgres, s3_archive]
deletion_method: anonymize_pii_keep_financial
right_to_erasure: false
notes: "Shipping address anonymized; order amounts retained for tax"
[, ]
[, ]
[, ]
Automated Deletion Jobs
import logging
from datetime import datetime, timezone, timedelta
import boto3
import psycopg2
from dataclasses import dataclass
from typing import Optional
logger = logging.getLogger(__name__)
@dataclass
class DeletionResult:
job_name: str
rows_affected: int
execution_time_ms: float
errors: list[str]
dry_run: bool
class RetentionJobRunner:
def __init__(self, db_url: str, dry_run: bool = True):
self.db_url = db_url
self.dry_run = dry_run
def run_all(self) -> list[DeletionResult]:
"""Run all scheduled retention jobs."""
jobs = [
self.delete_old_event_logs,
self.anonymize_deleted_users,
self.archive_old_orders,
self.purge_expired_sessions,
self.pseudonymize_old_access_logs,
]
results = []
for job in jobs:
try:
result = job()
results.append(result)
logger.info()
Exception e:
logger.error()
results.append(DeletionResult(
job_name=job.__name__,
rows_affected=,
execution_time_ms=,
errors=[(e)],
dry_run=.dry_run,
))
results
() -> DeletionResult:
cutoff = datetime.now(timezone.utc) - timedelta(days=)
start = datetime.now()
rows =
conn = psycopg2.connect(.db_url)
:
conn.cursor() cur:
.dry_run:
cur.execute(
,
(cutoff,)
)
rows = cur.fetchone()[]
logger.info()
:
:
cur.execute(, (cutoff,))
batch = cur.rowcount
conn.commit()
rows += batch
batch == :
._log_deletion(conn, , rows, cutoff)
:
conn.close()
elapsed = (datetime.now() - start).total_seconds() *
DeletionResult(, rows, elapsed, [], .dry_run)
() -> DeletionResult:
cutoff = datetime.now(timezone.utc) - timedelta(days=)
conn = psycopg2.connect(.db_url)
rows =
:
conn.cursor() cur:
.dry_run:
cur.execute(
,
(cutoff,)
)
rows = cur.fetchone()[]
:
cur.execute(, (cutoff,))
rows = cur.rowcount
conn.commit()
:
conn.close()
elapsed =
DeletionResult(, rows, elapsed, [], .dry_run)
():
.dry_run:
conn.cursor() cur:
cur.execute(, (table, rows, cutoff))
conn.commit()
():
s3 = boto3.client()
s3.put_bucket_lifecycle_configuration(
Bucket=bucket,
LifecycleConfiguration={
: [
{
: ,
: ,
: {: },
: [
{
: ,
:
},
{
: ,
:
},
],
: {: },
},
{
: ,
: ,
: {: },
: [
{
: ,
:
}
],
},
{
: ,
: ,
: {: },
: {: },
},
]
}
)
Right-to-Erasure Workflow
class ErasureRequestProcessor:
"""
Process GDPR Article 17 right-to-erasure requests.
Must complete within 30 days; best practice is 5-7 business days.
"""
def __init__(self, db, s3_client, email_service):
self.db = db
self.s3 = s3_client
self.email = email_service
async def process_erasure_request(self, user_id: str, request_id: str) -> dict:
results = {}
await self.db.execute(
"INSERT INTO erasure_requests (request_id, user_id, requested_at, status) VALUES ($1, $2, NOW(), 'processing')",
request_id, user_id
)
await self.db.execute("""
UPDATE users SET
email = 'erased_' || $1 || '@gdpr.invalid',
name = 'Erased User',
phone = NULL,
date_of_birth = NULL,
profile_photo_url = NULL,
is_erased = true,
erased_at = NOW()
WHERE id = $1
""", user_id)
results["user_account"] = "anonymized"
deleted_events = await self.db.fetchval(
"WITH d AS (DELETE FROM event_logs WHERE user_id = $1 RETURNING 1) SELECT COUNT(*) FROM d",
user_id
)
results["event_logs"] = f"deleted rows"
.db.execute(, user_id)
results[] =
._delete_s3_data(user_id)
results[] =
.db.execute(, user_id)
.db.execute(, user_id)
results[] =
.db.execute(
,
request_id, json.dumps(results)
)
.email.send_erasure_confirmation(request_id=request_id)
results
():
paginator = .s3.get_paginator()
page paginator.paginate(
Bucket=settings.DATA_LAKE_BUCKET,
Prefix=
):
obj page.get(, []):
.s3.delete_object(Bucket=settings.DATA_LAKE_BUCKET, Key=obj[])
Rules
- Every data element needs a retention period — "keep forever" is a policy decision, not an oversight.
- Financial data overrides erasure — GDPR right-to-erasure yields to legal financial record-keeping obligations.
- Anonymize instead of delete where referential integrity matters — preserve IDs, replace PII with anonymous values.
- Deletion jobs must be idempotent — running twice should produce the same result as running once.
- Audit the deletions — deletion audit logs must themselves be retained (typically 7 years).
- Test erasure workflows — regularly verify that right-to-erasure requests actually remove data from all systems.
- Staged deletion — soft-delete → grace period → anonymize → hard delete. Never skip stages.
- Cold storage is not deletion — archiving to Glacier is not erasure; still subject to right-to-erasure.
- Don't forget derived data — analytics aggregates computed from PII may themselves contain PII.
- Document legal basis for every retention period — when a regulator asks, you need the citation.
