media-dedup

name	media-dedup
description	Detect and reject duplicate media files by content hash
trigger	pipeline
depends_on	["media-classify"]
sandbox	true
inputs	[]
outputs	[{"name":"duplicates_found","type":"int","description":"Number of duplicate assets marked as rejected"}]
dependencies	{"python":["supabase"]}
tables	["asset_metadata (UPDATE status='rejected', add 'duplicate' tag)"]

media-dedup

Groups all assets by file_hash. When duplicates exist, keeps the one with the highest quality_score (or earliest created_at as tiebreaker) and marks the rest with status='rejected' and tag 'duplicate'.

Runs early in the pipeline to prevent wasted processing on duplicate files.

Pipeline Position

Step 3 of 8 — runs after media-classify, before media-process.