detecting-shadow-it-cloud-usage

name	detecting-shadow-it-cloud-usage
description	Detect unauthorized SaaS and cloud service usage (shadow IT) by analyzing proxy logs, DNS query logs, and netflow data using Python pandas for traffic pattern analysis and domain classification. Use when detecting unauthorized saas and cloud service usage (shadow it) by.
domain	cybersecurity
subdomain	cloud-security
tags	["shadow-IT","SaaS-discovery","proxy-logs","DNS-analysis","netflow","cloud-security","pandas"]
version	1.0
author	oyi77
license	Apache-2.0
nist_csf	["PR.IR-01","ID.AM-08","GV.SC-06","DE.CM-01"]

Detecting Shadow IT Cloud Usage

Overview

Shadow IT refers to unauthorized SaaS applications and cloud services used without IT approval. This skill analyzes proxy logs, DNS query logs, and firewall/netflow data to identify unauthorized cloud service usage, classify discovered domains against known SaaS categories, measure data transfer volumes, and flag high-risk services based on security posture and compliance requirements.

When to Use

Trigger phrases:

• "detecting shadow it cloud usage"

• "Detect unauthorized SaaS and cloud service usage (shadow IT) by analyzing proxy "

• When investigating security incidents that require detecting shadow it cloud usage

• When building detection rules or threat hunting queries for this domain

• When SOC analysts need structured procedures for this analysis type

• When validating security monitoring coverage for related attack techniques

Prerequisites

• Python 3.9+ with pandas, tldextract
• Proxy logs (Squid, Zscaler, or Palo Alto format) or DNS query logs
• SaaS application catalog/blocklist for classification
• Network firewall logs with FQDN resolution (optional)

Steps

# Example: IOC detection
import re

IOC_PATTERNS = {
    "ip": r"\b(?:\d{1,3}\.){3}\d{1,3}\b",
    "domain": r"\b[a-z0-9-]+\.[a-z]{2,}\b",
    "hash_md5": r"\b[a-f0-9]{32}\b",
    "hash_sha256": r"\b[a-f0-9]{64}\b",
}

def extract_iocs(text: str) -> dict:
    return {k: re.findall(v, text) for k, v in IOC_PATTERNS.items()}

1. Parse proxy access logs and extract destination domains with traffic volumes
2. Parse DNS query logs to identify resolved cloud service domains
3. Aggregate traffic by domain using pandas — total bytes, request counts, unique users