| name | cv-frame-prediction-averaging |
| description | Average per-frame sigmoid predictions across sampled video frames to produce a stable video-level classification probability |
Frame Prediction Averaging
Overview
Video classification models often process individual frames independently. Averaging the sigmoid outputs across all sampled frames reduces noise from individual frame variability (occlusion, motion blur, detection failures) and produces a more stable video-level prediction. This is simpler and often competitive with temporal models like LSTMs for binary classification tasks.
Quick Start
import torch
import numpy as np
def predict_video(model, frames, device="cuda"):
model.eval()
batch = torch.stack(frames).to(device)
with torch.no_grad():
logits = model(batch).squeeze()
probs = torch.sigmoid(logits)
return probs.mean().item()
video_prob = predict_video(model, face_tensors)
Workflow
- Sample N frames from the video (typically 15-20)
- Extract and preprocess face crops from each frame
- Run each frame through the classifier to get sigmoid probabilities
- Average all per-frame probabilities for the final video-level score
- Optionally clip to [0.05, 0.95] to avoid extreme predictions from single-frame errors
Key Decisions
- Averaging vs. max: mean is robust to outliers; max captures the single most suspicious frame (higher recall, lower precision)
- Weighted average: weight frames by face detection confidence to downweight low-quality detections
- Frames with no face: exclude from average rather than using a default value
- Clipping: clip final probability to [0.01, 0.99] to avoid log-loss penalties from extreme values
- vs. temporal models: averaging ignores frame order but is more robust with limited training data
References