implementing-network-traffic-baselining

name	implementing-network-traffic-baselining
description	Build network traffic baselines from NetFlow/IPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly/daily traffic pattern profiling. Use when building network traffic baselines from netflow/ipfix data using python pandas.
domain	cybersecurity
subdomain	network-security
tags	["netflow","ipfix","traffic-analysis","baselining","anomaly-detection","pandas","network-monitoring"]
version	1.0
author	oyi77
license	Apache-2.0
nist_csf	["PR.IR-01","DE.CM-01","ID.AM-03","PR.DS-02"]

Implementing Network Traffic Baselining

Overview

Network traffic baselining establishes normal communication patterns by analyzing historical NetFlow/IPFIX data to create statistical profiles of expected behavior. This skill uses Python pandas to compute hourly and daily traffic distributions, per-host byte/packet counts, protocol ratios, and top-N talker profiles. Anomalies are detected using z-score thresholds and IQR (interquartile range) outlier methods, enabling SOC analysts to identify deviations such as data exfiltration spikes, beaconing patterns, and unusual port usage.

When to Use

Trigger phrases:

• "implementing network traffic baselining"

• "Build network traffic baselines from NetFlow/IPFIX data using Python pandas for "

• When deploying or configuring implementing network traffic baselining capabilities in your environment

• When establishing security controls aligned to compliance requirements

• When building or improving security architecture for this domain

• When conducting security assessments that require this implementation

Prerequisites

• NetFlow v5/v9 or IPFIX flow data exported as CSV or JSON
• Python 3.8+ with pandas and numpy libraries
• Historical flow data (minimum 7 days recommended for baseline)

Steps

# Example: IOC detection
import re

IOC_PATTERNS = {
    "ip": r"\b(?:\d{1,3}\.){3}\d{1,3}\b",
    "domain": r"\b[a-z0-9-]+\.[a-z]{2,}\b",
    "hash_md5": r"\b[a-f0-9]{32}\b",
    "hash_sha256": r"\b[a-f0-9]{64}\b",
}

def extract_iocs(text: str) -> dict:
    return {k: re.findall(v, text) for k, v in IOC_PATTERNS.items()}

1. Ingest NetFlow/IPFIX records from CSV or JSON exports
2. Compute hourly and daily traffic volume distributions (bytes, packets, flows)
3. Build per-source-IP baseline profiles with mean, median, standard deviation