Photo Content Recognition & Curation Expert
Expert in photo content analysis and intelligent curation. Combines classical computer vision with modern deep learning for comprehensive photo analysis.
Decision Points
PHOTO LIBRARY SIZE?
├─ <1K photos → Single-pass processing
│ ├─ dHash for duplicates (fastest)
│ └─ Agglomerative clustering (known thresholds)
├─ 1K-10K photos → Optimized pipeline
│ ├─ pHash → DINOHash hybrid
│ └─ HDBSCAN if variance in cluster sizes >3x
└─ >10K photos → Batch processing
├─ GPU batching (batch_size=32)
└─ Incremental updates only
CLUSTERING ALGORITHM CHOICE?
├─ Face variance known? → Agglomerative (threshold=0.4, 0.6)
├─ Unknown distribution? → HDBSCAN (auto-threshold)
└─ Real-time additions? → Incremental clustering
DUPLICATE DETECTION ROBUSTNESS?
├─ Exact duplicates only? → dHash (Hamming ≤3)
├─ Brightness/contrast edits? → pHash (Hamming ≤5)
├─ Heavy crops/compression? → DINOHash (Hamming ≤10)
└─ Production system? → Hybrid (pHash→DINOHash)
BURST PHOTO SELECTION?
├─ Face quality available? → Weight faces 35%, sharpness 30%
├─ No faces detected? → Weight sharpness 50%, aesthetics 30%
└─ Action photos? → Weight sharpness 60%, exposure 25%
CONTENT FILTERING CONFIDENCE?
├─ Screenshot confidence >0.6? → Filter out
├─ NSFW confidence >0.8? → Filter out
├─ Face confidence <0.9? → Exclude from clustering
└─ Pet confidence <0.7? → Manual review
Failure Modes
1. Too Many Face Clusters (Symptom: 100+ clusters for 1000 photos)
Root Cause: Clustering threshold too strict, splitting same person into multiple clusters
Detection Rule: If clusters/unique_people ratio > 2.0, threshold too conservative
Fix:
- Increase distance threshold from 0.4 to 0.6
- Run second-pass merging with relaxed threshold
- Verify with manual spot-check of 5 random cluster pairs
2. False Duplicate Detection (Symptom: Different people flagged as duplicates)
Root Cause: Perceptual hash threshold too loose, faces in similar poses triggering matches
Detection Rule: If duplicate groups contain faces with cosine distance >0.7, hash threshold too aggressive
Fix:
- Tighten Hamming threshold from 10 to 5 bits
- Add face verification step: extract face embeddings from "duplicates"
- If face similarity <0.5, not duplicates
3. Burst Selection Missing Best Frame (Symptom: Blurry photos selected over sharp ones)
Root Cause: Face quality override masking sharpness issues when faces poorly detected
Detection Rule: If selected burst frame Laplacian variance <500 while other frames >1000
Fix:
- Cap face quality bonus at 20% total score
- Add face detection confidence check (>0.9 required for face bonus)
- Fallback to pure sharpness if no high-confidence faces
4. Screenshot False Negatives (Symptom: UI screenshots in final curated set)
Root Cause: Single-signal detection missing edge cases (dark mode, custom UI)
Detection Rule: If photos contain perfect rectangles + high text density but confidence <0.6
Fix:
- Lower combined threshold from 0.6 to 0.5 for multi-signal hits
- Add aspect ratio check for common device screens (16:9, 19.5:9, 4:3)
- OCR confidence boost for text >20% coverage
5. Memory Explosion on Large Libraries (Symptom: >16GB RAM usage on 10K photos)
Root Cause: Loading all images into memory simultaneously instead of streaming
Detection Rule: If memory usage >2GB per 1000 photos, batch size too large
Fix:
- Reduce batch size from 64 to 16 images
- Implement lazy loading with generators
- Cache only embeddings/hashes, not raw images
Worked Examples
Example 1: 500-Photo Family Library Curation
Scenario: Family vacation photos (500 images), many bursts and similar poses, want top 100 for sharing.
Step 1 - Library Analysis:
- Photo count: 500 → Use pHash→DINOHash hybrid
- Face variance unknown → HDBSCAN clustering
- Mix of portraits/landscapes → Multi-criteria burst selection
Step 2 - Duplicate Detection:
pHash_duplicates = find_duplicates_phash(photos, threshold=5)
verified_duplicates = []
for group in pHash_duplicates:
dino_hashes = [compute_dinohash(img) for img in group]
if hamming_distance(dino_hashes[0], dino_hashes[1]) <= 8:
verified_duplicates.append(group)
Step 3 - Face Clustering Decision:
- Extract face embeddings: 234 faces detected
- Check cluster size variance: max_size/min_size = 4.2 (>3x threshold)
- Choose HDBSCAN over agglomerative
Step 4 - Burst Selection Trade-offs:
- Detected 12 burst sequences (3-8 photos each)
- Burst #3: 5 photos of kids playing
- Frame 2: Highest sharpness (1850 Laplacian) but no faces detected
- Frame 4: Lower sharpness (1200) but 2 clear faces, both smiling
- Decision: Frame 4 selected (face quality 35% weight > sharpness 30%)
Final Output: 387 unique photos → 100 curated (top aesthetic + diversity)
Example 2: Burst Selection Edge Case - Blurry + Exposed Face
Scenario: Wedding ceremony burst of 15 shots, bride's face overexposed in sharpest frames.
Challenge: Best technical shot (frame 8) has bride's face blown out by direct sunlight.
Analysis Process:
frame_8_metrics = {
'sharpness': 2100,
'face_quality': 0.2,
'aesthetics': 0.75,
'exposure': 0.3
}
frame_11_metrics = {
'sharpness': 1650,
'face_quality': 0.9,
'aesthetics': 0.8,
'exposure': 0.85
}
frame_8_score = 2100*0.3 + 0.2*0.35 + 0.75*0.2 + 0.3*0.05 = 630 + 0.07 + 0.15 + 0.015 = 630.235
frame_11_score = 1650*0.3 + 0.9*0.35 + 0.8*0.2 + 0.85*0.05 = 495 + 0.315 + 0.16 + 0.0425 =
Expert Insight: Frame 11 selected despite lower sharpness because face quality penalty (-80%) outweighs sharpness advantage. Novice would pick frame 8 based on technical metrics alone.
Quality Gates
Photo Indexing Complete:
Duplicate Detection Complete:
Face Clustering Complete:
Burst Selection Complete:
Content Filtering Complete:
NOT-FOR Boundaries
This skill should NOT be used for:
-
GPS-based location clustering → Use event-detection-temporal-intelligence-expert instead
- Reason: Requires temporal event modeling and GPS clustering algorithms
-
Color palette extraction and harmony → Use color-theory-palette-harmony-expert instead
- Reason: Needs color theory knowledge and palette composition rules
-
Semantic image-text matching → Use clip-aware-embeddings instead
- Reason: Requires cross-modal understanding and text-image alignment
-
Video analysis or frame extraction → Use video processing specialist instead
- Reason: Different pipeline for temporal sequences and video codecs
-
Advanced photo editing → Use image editing specialist instead
- Reason: Focus is on analysis/curation, not pixel-level manipulation
-
3D scene reconstruction → Use computer vision specialist instead
- Reason: Requires stereo vision and depth estimation techniques
Delegation Rules:
- If task involves time/location →
event-detection-temporal-intelligence-expert
- If task involves colors/aesthetics →
color-theory-palette-harmony-expert
- If task involves text matching →
clip-aware-embeddings
- If task involves editing → Image editing specialist
