| name | ingest-cloudtrail-ocsf |
| description | Convert raw AWS CloudTrail events (JSON or NDJSON, single events or CloudTrail digest files) into OCSF 1.8 API Activity events (class 6003). Maps userIdentity to OCSF actor, sourceIPAddress to src_endpoint, eventName to api.operation, eventSource to api.service.name, and infers activity_id (Create / Read / Update / Delete) from the event verb. Sets status_id to Failure when CloudTrail records an errorCode. Use when the user mentions CloudTrail ingestion, AWS audit log normalization, OCSF pipeline for AWS, or feeding CloudTrail into a SIEM. Do NOT use for GCP audit logs (use ingest-gcp-audit-ocsf), Azure activity logs (use ingest-azure-activity-ocsf), or Kubernetes audit logs (use ingest-k8s-audit-ocsf). Do NOT use as a detection skill — this skill only normalises events, it does not flag anything. |
| purpose | Convert raw AWS CloudTrail events (JSON or NDJSON, single events or CloudTrail digest files) into OCSF 1.8 API Activity events (class 6003). Maps userIdentity to OCSF actor, sourceIPAddress to src_endpoint, eventName... |
| capability | ingest |
| persistence | none |
| telemetry | stderr_jsonl |
| privilege_escalation | none |
| license | Apache-2.0 |
| approval_model | none |
| execution_modes | jit, ci, mcp, persistent |
| side_effects | none |
| input_formats | raw |
| output_formats | ocsf, native |
| concurrency_safety | stateless |
ingest-cloudtrail-ocsf
Thin, single-purpose ingestion skill: raw CloudTrail JSON in → canonical event projection → OCSF 1.8 API Activity JSONL or native enriched JSONL out. No detection logic, no side effects, no AWS API calls. Reads files or stdin; writes JSONL or stdout.
Wire contract
Reads either of the two CloudTrail layouts that are emitted by the AWS service:
- Single event — one JSON object per line (NDJSON, e.g. EventBridge → Kinesis Firehose to S3)
- CloudTrail digest — top-level
{"Records": [...]} wrapping an array of events (the format aws s3 cp retrieves directly from the CloudTrail bucket)
The skill auto-detects which shape it's looking at and unwraps Records if present.
By default it writes OCSF 1.8 API Activity (class_uid: 6003, category_uid: 6). See ../OCSF_CONTRACT.md for the field-level pinning that every OCSF event matches.
When --output-format native is selected, it emits the same event in the repo's native enriched shape with stable event_uid, normalized provider/account/operation/status fields, and preserved actor/session/source context, but without the OCSF envelope fields.
Field mapping
The native output field list, the eventName → activity_id prefix table, the errorCode → status_id rules, and the explicit "not mapped (yet)" scope live in references/field-map.md. Keeping the detail there keeps this file under the progressive-disclosure target (#247) while detectors and reviewers still get the exact mapping one click away.
Usage
python src/ingest.py cloudtrail.json > cloudtrail.ocsf.jsonl
python src/ingest.py cloudtrail.json --output-format native > cloudtrail.native.jsonl
aws s3 cp s3://my-cloudtrail-bucket/AWSLogs/.../recent.json.gz - | gunzip | python src/ingest.py
Tests
tests/test_ingest.py runs the ingester against ../golden/cloudtrail_raw_sample.jsonl and asserts deep-equality against ../golden/cloudtrail_sample.ocsf.jsonl with volatile fields scrubbed. Plus unit tests for the activity_id mapping table, status_id detection, and Records-wrapper auto-detection.
