Implementing AWS Macie for Data Classification
Overview
Amazon Macie is a fully managed data security and privacy service that uses machine learning and pattern matching to discover and protect sensitive data in Amazon S3. Macie automatically evaluates your S3 bucket inventory on a daily basis and identifies objects containing PII, financial information, credentials, and other sensitive data types. It provides two discovery approaches: automated sensitive data discovery for broad visibility and targeted discovery jobs for deep analysis.
When to Use
- When deploying or configuring implementing aws macie for data classification capabilities in your environment
- When establishing security controls aligned to compliance requirements
- When building or improving security architecture for this domain
- When conducting security assessments that require this implementation
Common Misconfigurations & Verification
- Automated discovery never enabled: enabling Macie (
enable-macie) does NOT start scanning. Run update-automated-discovery-configuration --status ENABLED and confirm with get-automated-discovery-configuration (must return status: ENABLED).
- Customer-managed CMK objects silently skipped: Macie cannot read objects encrypted with a customer-managed KMS key unless its service-linked role is granted
kms:Decrypt. Such objects appear under unclassifiableObjectCount, not as findings. Check: aws macie2 describe-buckets --query 'buckets[?unclassifiableObjectCount.total>0].[bucketName,unclassifiableObjectCount.total]'.
- Job scoping excludes real data: an
OBJECT_EXTENSION/OBJECT_KEY scope term that omits the actual prefixes/extensions means the job runs but classifies nothing. Verify the job actually processed objects via list-classification-jobs --query 'items[].statistics'.
- Findings lost after 90 days: sensitive-data findings auto-expire and are not retained unless you set an S3 export. Confirm
aws macie2 get-classification-export-configuration returns a destination bucket + KMS key.
- Security Hub integration assumed: findings only flow to Security Hub if both services are enabled in the same region; check
findingPublishingFrequency and that Security Hub shows the Macie product.
- Multi-account gaps: new org accounts are not auto-covered. After
enable-organization-admin-account, set auto-enable and confirm members with list-members --query 'members[?relationshipStatus!=Enabled]'.
aws macie2 get-macie-session --query 'status'
aws macie2 get-automated-discovery-configuration
aws macie2 get-classification-export-configuration
Prerequisites
- AWS account with S3 buckets containing data to classify
- IAM permissions for Macie service configuration
- AWS Organizations setup (for multi-account deployment)
- S3 buckets in supported regions
Enable Macie
Via AWS CLI
aws macie2 enable-macie
aws macie2 get-macie-session
aws macie2 update-automated-discovery-configuration \
--status ENABLED
Via Terraform
resource "aws_macie2_account" "main" {}
resource "aws_macie2_classification_export_configuration" "main" {
depends_on = [aws_macie2_account.main]
s3_destination {
bucket_name = aws_s3_bucket.macie_results.id
key_prefix = "macie-findings/"
kms_key_arn = aws_kms_key.macie.arn
}
}
Configure Discovery Jobs
Create a classification job for specific buckets
aws macie2 create-classification-job \
--job-type ONE_TIME \
--name "pii-scan-production-buckets" \
--s3-job-definition '{
"bucketDefinitions": [{
"accountId": "123456789012",
"buckets": [
"production-data-bucket",
"customer-records-bucket"
]
}]
}' \
--managed-data-identifier-selector ALL
Create a scheduled recurring job
aws macie2 create-classification-job \
--job-type SCHEDULED \
--name "weekly-sensitive-data-scan" \
--schedule-frequency-details '{
"weekly": {
"dayOfWeek": "MONDAY"
}
}' \
--s3-job-definition '{
"bucketDefinitions": [{
"accountId": "123456789012",
"buckets": ["all-data-bucket"]
}],
"scoping": {
"includes": {
"and": [{
"simpleScopeTerm": {
"comparator": "STARTS_WITH",
"key": "OBJECT_KEY",
"values": ["uploads/", "documents/"]
}
}]
}
}
}'
Custom Data Identifiers
Create a custom identifier for internal IDs
aws macie2 create-custom-data-identifier \
--name "internal-employee-id" \
--description "Matches internal employee ID format EMP-XXXXXX" \
--regex "EMP-[0-9]{6}" \
--severity-levels '[
{"occurrencesThreshold": 1, "severity": "LOW"},
{"occurrencesThreshold": 10, "severity": "MEDIUM"},
{"occurrencesThreshold": 50, "severity": "HIGH"}
]'
Create identifier for project codes
aws macie2 create-custom-data-identifier \
--name "project-code-identifier" \
--description "Matches project codes in format PRJ-XXXX-XX" \
--regex "PRJ-[A-Z]{4}-[0-9]{2}" \
--keywords '["project", "code", "initiative"]' \
--maximum-match-distance 50
Allow Lists
Create an allow list to suppress false positives
aws macie2 create-allow-list \
--name "test-data-exclusions" \
--description "Exclude known test data patterns" \
--criteria '{
"regex": "TEST-[0-9]{4}-[0-9]{4}-[0-9]{4}-[0-9]{4}"
}'
Managed Data Identifiers
Macie provides 300+ managed data identifiers covering:
| Category | Examples |
|---|
| PII | SSN, passport numbers, driver's license, date of birth, names, addresses |
| Financial | Credit card numbers, bank account numbers, SWIFT codes |
| Credentials | AWS secret keys, API keys, SSH private keys, OAuth tokens |
| Health | HIPAA identifiers, health insurance claim numbers |
| Legal | Tax identification numbers, national ID numbers |
Findings Management
List findings
aws macie2 list-findings \
--finding-criteria '{
"criterion": {
"severity.description": {
"eq": ["High"]
},
"category": {
"eq": ["CLASSIFICATION"]
}
}
}' \
--sort-criteria '{"attributeName": "updatedAt", "orderBy": "DESC"}' \
--max-results 25
Get finding details
aws macie2 get-findings \
--finding-ids '["finding-id-1", "finding-id-2"]'
Export findings to Security Hub
aws macie2 get-macie-session --query 'findingPublishingFrequency'
EventBridge Integration for Automated Response
{
"source": ["aws.macie"],
"detail-type": ["Macie Finding"],
"detail": {
"severity": {
"description": ["High", "Critical"]
}
}
}
Lambda function for automated remediation
import boto3
import json
s3 = boto3.client('s3')
sns = boto3.client('sns')
def lambda_handler(event, context):
finding = event['detail']
severity = finding['severity']['description']
bucket = finding['resourcesAffected']['s3Bucket']['name']
key = finding['resourcesAffected']['s3Object']['key']
sensitive_types = [d['type'] for d in finding.get('classificationDetails', {}).get('result', {}).get('sensitiveData', [])]
if severity in ['High', 'Critical']:
s3.put_object_tagging(
Bucket=bucket,
Key=key,
Tagging={
'TagSet': [
{'Key': 'macie-finding', 'Value': severity},
{'Key': 'sensitive-data', 'Value': ','.join(sensitive_types)},
{'Key': 'requires-review', 'Value': 'true'}
]
}
)
sns.publish(
TopicArn='arn:aws:sns:us-east-1:123456789012:security-alerts',
Subject=f'Macie {severity} Finding: {bucket}/',
Message=json.dumps({
: bucket,
: key,
: severity,
: sensitive_types,
: finding[]
}, indent=)
)
{: }
Multi-Account Deployment
Designate Macie administrator account
aws macie2 enable-organization-admin-account \
--admin-account-id 111111111111
Add member accounts
aws macie2 create-member \
--account '{"accountId": "222222222222", "email": "security@example.com"}'
Monitoring Macie Operations
Usage statistics
aws macie2 get-usage-statistics \
--filter-by '[{"comparator": "GT", "key": "accountId", "values": []}]' \
--sort-by '{"key": "accountId", "orderBy": "ASC"}'
Classification job status
aws macie2 list-classification-jobs \
--filter-criteria '{"includes": [{"comparator": "EQ", "key": "jobStatus", "values": ["RUNNING"]}]}'
References
