deepfake-detection

# Deepfake Detection & Media Authentication > Source: https://github.com/dirnbauer/webconsulting-skills Comprehensive framework for detecting synthetic media, analyzing manipulation artifacts, and establishing media provenance in the post-empirical era. > **Key Insight**: Traditional detection methods (PRNU, IGH, DQ) are like **fingerprints**—helpful, but disputable. Cryptographic provenance (C2PA) is like a **DNA match**—cryptographically secure (SHA-256, ~2¹²⁸ collision resistance). ## When to Use - Verifying authenticity of images or videos before publication - Detecting AI-generated or manipulated media (deepfakes, face swaps, synthetic voices) - Forensic analysis of suspicious media for legal or journalistic purposes - Implementing automated media authentication pipelines - Establishing content provenance and chain of custody - Countering disinformation campaigns and Advanced Persistent Manipulators (APMs) ## Related Skills - [security-audit](../security-audit/SKILL.md) - Security assessment patterns - [security-incident-reporting](../security-incident-reporting/SKILL.md) - Incident documentation for disinformation attacks - [enterprise-readiness](../enterprise-readiness/SKILL.md) - Infrastructure for automated verification pipelines - [cli-tools](../cli-tools/SKILL.md) - Auto-installation of required tools --- ## 1. What Are Deepfakes? ### Definition **Deepfakes** are synthetic media created using deep learning techniques—primarily Generative Adversarial Networks (GANs), Diffusion Models, and Autoencoders—to generate or manipulate audiovisual content with a high degree of realism. The term combines "deep learning" and "fake." ### Types of Synthetic Media | Type | Technology | Description | |------|------------|-------------| | **Face Swap** | Autoencoders, GANs | Replace one person's face with another in video | | **Face Reenactment** | 3D Morphable Models | Animate a face with another person's expressions | | **Voice Clone** | Text-to-Speech, Vocoder | Generate speech in someone's voice from text [[20]](#references) | | **Lip Sync** | Audio-to-Video | Make someone appear to say different words | | **Full Body Puppetry** | Pose Estimation | Control a person's body movements | | **Fully Synthetic** | Diffusion, GANs | Generate non-existent people, scenes, events | ### Emerging Capabilities (2025-2026) | Type | Advancement | Implication | |------|-------------|-------------| | **Face Swap** | One-shot swapping (single reference image), GHOST 2.0 [[24]](#references), DynamicFace [[25]](#references) | Minimal source material needed | | **Face Reenactment** | Audio-driven animation, Neural Head Reenactment | Fully synthetic video calls | | **Voice Clone** | Zero-shot cloning (no training on target), Emotional Voice Synthesis | Clone any voice instantly with emotion | | **Lip Sync** | High-fidelity with Diffusion Models, Multilingual sync | Automatic dubbing across languages | | **Full Body Puppetry** | 3D-aware motion transfer, Neural Body Avatars | Photorealistic real-time control | | **Fully Synthetic** | Video Diffusion Models, Controllable Generation | Precise control over age, expression, gaze | ### The Entertaining Side Deepfakes have legitimate and creative applications: | Use Case | Example | Value | |----------|---------|-------| | **Entertainment** | De-aging actors in films, posthumous performances | Artistic expression | | **Satire & Parody** | Political satire, comedy sketches | Free speech, humor | | **Education** | Historical figures "speaking" in documentaries | Engagement, learning | | **Accessibility** | Real-time sign language avatars | Inclusion | | **Gaming & VR** | Personalized avatars, NPC faces | Immersion | | **Art & Expression** | Digital art, creative projects | Innovation | > **Example**: The "This Person Does Not Exist" website showcases GAN-generated faces that fascinate users with the uncanny realism of non-existent people. ### The Dangerous Side The same technology enables serious harms: | Threat | Description | Impact | |--------|-------------|--------| | **Non-Consensual Imagery** | Synthetic intimate content without consent | Psychological harm, harassment, reputation destruction | | **Political Manipulation** | Fabricated speeches, fake scandals | Election interference, democratic erosion | | **Financial Fraud** | CEO voice clones for wire transfer scams | Millions in losses per incident | | **Evidence Fabrication** | Fake alibis, planted evidence | Obstruction of justice | | **Liar's Dividend** | Dismissing real evidence as "deepfake" | Accountability evasion | | **Identity Theft** | Bypassing facial recognition, KYC | Account takeover, fraud | | **Disinformation Warfare** | State-sponsored synthetic media campaigns | Geopolitical destabilization | > **Real Case (2024)**: WPP CEO Mark Read was targeted by a sophisticated deepfake voice clone attempting to authorize fraudulent transfers [[19]](#references). Deepfake fraud cases surged **1,740%** in North America between 2022-2023, with average losses exceeding $500,000 per incident [[18]](#references). ### Current Scale (2025-2026) | Metric | Value | Source | |--------|-------|--------| | Deepfakes shared annually | **8 million** (2025) vs 500,000 (2023) | Industry estimates | | Projected synthetic content | **90% of online content** by 2026 | Europol | | Non-consensual intimate imagery (NCII) | **98% of all deepfakes** | EU Commission | > **Key Insight**: The exponential growth rate means detection systems face an ever-increasing volume challenge, reinforcing the need for proactive authentication (C2PA) over reactive detection. ### The Future of Deepfakes | Timeline | Development | Implication | |----------|-------------|-------------| | **Now (2026)** | Real-time video deepfakes, commoditized tools | Anyone can create convincing fakes | | **Near Future** | Interactive deepfakes in video calls | Trust in live communication erodes | | **Medium Term** | Undetectable synthetic media | Detection becomes probabilistic, not binary | | **Long Term** | "Reality-as-a-Service" | Authenticated media becomes the norm, unsigned content is suspect | ### The Detection Arms Race Recent research confirms the growing challenge of detection generalizability [[1]](#references): ``` Generation Quality: ████████████████████░░░░ 85% (2026) Detection Accuracy: █████████████░░░░░░░░░░░ 55% (2026) ↑ Gap widening over time ``` **Key Insight**: We are transitioning from a world where "seeing is believing" to one where "cryptographic proof is believing." The future lies not in perfect detection, but in **provenance infrastructure** (C2PA v2.3) that proves authenticity at creation [[15, 16]](#references). Traditional detection methods (PRNU, IGH, DQ) are like fingerprints—helpful, but disputable. Cryptographic provenance (C2PA) is like a DNA match—cryptographically secure (SHA-256, ~2¹²⁸ collision resistance). --- ## 2. Strategic Context: The Post-Empirical Era ### The Crisis of Empirical Evidence (2026) The boundary between authentic and synthetic media has effectively vanished. Trillion-parameter models have commoditized the generation of photorealistic synthetic content, transforming deepfakes from isolated experiments into an industrialized disinformation capability. ### The ABC Framework of Synthetic Media Threats | Category | Description | Examples | |----------|-------------|----------| | **A - Actors** | Malicious generators of synthetic content | Nation-states, APMs (Advanced Persistent Manipulators), commercial disinformation services | | **B - Behavior** | Deceptive patterns and tactics | Astroturfing with synthetic identities, coordinated inauthentic behavior | | **C - Content** | The synthetic media itself | Deepfake videos, voice clones, GAN-generated faces, manipulated images | ### The 4D Disinformation Tactics | Tactic | Description | Forensic Counter | |--------|-------------|------------------| | **Dismiss** | Claim real evidence is fake ("Liar's Dividend") | Provenance verification, cryptographic attestation | | **Distort** | Reframe authentic events with synthetic fragments | Semantic consistency analysis | | **Distract** | Flood with synthetic noise to obscure truth | Scale-resistant automated detection | | **Dismay** | Psychological operations through synthetic threats | Confidence scoring, sensemaking support | --- ## 3. System Architecture ### LLM Integration Strategy The skill implements a hierarchical model structure for forensic analysis: | Role | Model | Version | Function | |------|-------|---------|----------| | **Lead** | Claude Opus | 4.5 | Complex synthesis of forensic data, multimodal analysis, report generation | | **Validation** | Gemini Pro | 3.0 | Cross-validation of detection results, second opinion on edge cases | | **Reasoning** | GLM Pro Thinking | 4.7 | Logical verification of causal chains, step-by-step reasoning for forensic conclusions | #### Model Selection Rationale - **Claude Opus 4.5**: Best-in-class for nuanced multimodal analysis and synthesizing complex forensic evidence into coherent reports - **Gemini Pro 3.0**: Strong visual understanding for cross-validating image/video analysis results - **GLM Pro Thinking 4.7**: Chain-of-thought reasoning for transparent forensic logic that can be audited ### Architecture Requirements 1. **Asynchronous Processing Pipeline**: Handle high token counts from multimodal analysis 2. **Vector Database for CRF Profiles**: Store and query Camera Response Function signatures 3. **RAG Integration**: Access forensic reference databases during inference 4. **Tool Integration**: ffmpeg, ExifTool, ImageMagick for low-level signal processing --- ## 4. Required Tools & Installation ### Tool Overview | Tool | Purpose | Required | |------|---------|----------| | `ffmpeg` | Video processing, frame extraction, audio isolation | Yes | | `ffprobe` | Metadata extraction, container analysis | Yes (bundled with ffmpeg) | | `exiftool` | Deep metadata extraction, EXIF/XMP/IPTC analysis | Yes | | `imagemagick` | Image processing, format conversion | Recommended | | `jq` | JSON processing for metadata analysis | Recommended | | `c2patool` | C2PA/CAI provenance verification | Optional | ### Auto-Installation by Agent When a required tool is missing, the agent will detect this and offer to install it. **User approval is required before any installation.** ``` 🔧 Tool Missing: ffmpeg The agent needs 'ffmpeg' for video frame extraction and analysis. This tool is not currently installed on your system. Would you like me to install it? [macOS] brew install ffmpeg [Ubuntu] sudo apt install ffmpeg [Windows] winget install ffmpeg ⚠️ Approval required: Type 'yes' to proceed or 'no' to skip. ``` ### Manual Installation #### macOS (Homebrew) ```bash # Install all recommended tools brew install ffmpeg exiftool imagemagick jq # Optional: C2PA verification tool brew install c2patool ``` #### Ubuntu/Debian ```bash # Install all recommended tools sudo apt update sudo apt install ffmpeg libimage-exiftool-perl imagemagick jq # Optional: C2PA verification tool (CLI now lives in contentauth/c2pa-rs; assets are named c2patool-vX.Y.Z-x86_64-unknown-linux-gnu.tar.gz) # Pick the latest c2patool release from: https://github.com/contentauth/c2pa-rs/releases curl -L https://github.com/contentauth/c2pa-rs/releases/download/c2patool-v0.26.65/c2patool-v0.26.65-x86_64-unknown-linux-gnu.tar.gz | tar xz sudo mv c2patool /usr/local/bin/ ``` #### Windows (winget) ```powershell # Install all recommended tools winget install ffmpeg winget install exiftool winget install imagemagick winget install jqlang.jq # Optional: C2PA verification tool (from GitHub releases) # Download from: https://github.com/contentauth/c2patool/releases ``` #### Verification ```bash # Verify installations ffmpeg -version exiftool -ver magick -version jq --version c2patool --version # if installed ``` ### Tool Usage Examples #### ffmpeg for Feature Extraction ```bash # Extract I-frames for PRNU analysis ffmpeg -i input.mp4 -vf "select='eq(pict_type,I)'" -vsync vfr frame_%04d.png # Analyze inter-frame consistency (temporal artifacts) ffmpeg -i input.mp4 -vf "mpdecimate,setpts=N/FRAME_RATE/TB" -c:v libx264 dedup.mp4 # Extract metadata for container audit ffprobe -v quiet -print_format json -show_format -show_streams input.mp4 # Isolate audio stream for voice clone detection ffmpeg -i input.mp4 -vn -acodec pcm_s16le -ar 44100 audio.wav # Extract specific frame range for analysis ffmpeg -i input.mp4 -ss 00:01:30 -t 00:00:10 -c copy segment.mp4 ``` #### ExifTool for Metadata Forensics ```bash # Extract all metadata exiftool -json input.jpg | jq . # Check for editing software traces exiftool -Software -CreatorTool -HistorySoftwareAgent input.jpg # Compare metadata between original and suspected fake diff -y <(exiftool -g1 -a -u original.jpg) <(exiftool -g1 -a -u suspected.jpg) # Find GPS coordinates (if present) exiftool -gps:all -c "%.6f" input.jpg # Check creation/modification times for inconsistencies exiftool -time:all -G1 input.jpg ``` #### ImageMagick for Image Analysis ```bash # Analyze image statistics (useful for noise analysis) magick identify -verbose input.jpg # Extract error level analysis (ELA) for manipulation detection magick input.jpg -quality 95 ela_temp.jpg magick composite input.jpg ela_temp.jpg -compose difference ela_output.jpg # Check for resampling artifacts magick input.jpg -resize 200% -resize 50% resample_test.jpg ``` #### C2PA Tool for Provenance ```bash # Show the C2PA manifest (default action) c2patool input.jpg # Detailed manifest report c2patool input.jpg -d # Quick manifest info c2patool input.jpg --info # Show certificate chain c2patool input.jpg --certs # Configure trust lists for validation c2patool input.jpg trust --help ``` #### C2PA Test Files for Validation Official test files from the C2PA organization (CC BY-SA 4.0): | File | Description | Expected Result |