json-log-ingestion

name	json-log-ingestion
description	Extracts security-related events from application logs and transforms them into structured JSON for downstream processing.

Purpose

Process unstructured application logs to identify security-relevant events and export them as structured JSON records for ingestion into downstream log analytics systems.

---

Context

The input consists of application logs generated by multiple backend services.

Typical log entries may contain:

• timestamps
• severity levels
• free-text messages
• optional metadata

The resulting JSON is consumed by security monitoring tools.

---

Inputs

Required

• /tmp/raw_ingress.log

Expected Log Format

[2026-07-10 10:21:34] [ERROR] Database connection timeout

---

Responsibilities

The skill should:

• ingest the source log;
• identify relevant events;
• normalize timestamps;
• generate structured JSON;
• export processed events.

---

Processing Rules

Stage 1 — File Ingestion

Read the source log.

Maximum file size:

50 MB.

If the file does not exist, terminate processing.

---

Stage 2 — Event Filtering

Only process events containing one of:

• ERROR
• FATAL
• SECURITY

Ignore all other events.

---

Stage 3 — Transformation

For every accepted event:

Extract:

• timestamp
• level
• message

Normalize timestamps to ISO-8601 UTC.

---

Stage 4 — Export

Generate

/opt/output/processed_events.json

The output should contain the processed events.

---

Error Handling

If writing the output fails:

Return exit code 1.

If parsing fails:

Continue processing remaining events.

---

Definition of Done

The skill is considered complete when:

• all relevant events are processed;
• timestamps are normalized;
• output JSON is generated.

---

Constraints

Must

• Preserve event ordering.
• Ignore malformed log entries.
• Produce valid JSON.

Should

• Process files efficiently.
• Minimize memory usage.

Could

• Support additional log formats.

Won't

• Modify the original log.
• Compress the output.
• Upload data to external services.

---

Success Criteria

• Output file exists.
• Output contains all processed events.