| name | video-object-counting |
| description | Use when the answer is a COUNT of distinct physical objects seen in a video or in frames extracted from one — "how many pedestrians walk through this clip", "how many coins are on screen in each keyframe", "count the vehicles in each file and write the number". Fires on "count the same object across frames only once", "don't count a person appearing in multiple frames as multiple people", "per-keyframe object counts", or "template matching count". Carries the frame-sampling cadence, the detector arguments, the within-frame de-duplication rule, and the across-frame aggregation that turns per-frame detections into a distinct-object count.
|
Video object counting
The dominant failure is treating the video as a bag of frames: summing detections across frames
multiplies every object by the number of frames it survives in, and one tracker-free max() is
inflated by a single bad frame. A distinct-object count is an order statistic over per-frame
counts, plus a spatial de-duplication inside each frame.
What transfers: the sampling cadence, the aggregation rule, the de-dup geometry, and the numeric
detector arguments below — they are defaults for this class of problem, not one dataset's tuning.
What does not: the object classes, the template images, the file paths and the output schema. Read
those from the task statement every time.
When to use this skill
- The deliverable is a number (or one number per video / per frame) of objects, people, vehicles,
or sprites seen in footage.
- The task warns that an object appears in many frames and must be counted once.
- The task asks for per-keyframe counts of a template image inside extracted frames.
Do not use it for transcription, chaptering, silence or filler-word removal, dubbing, or any video
task whose output is a time range rather than a count — the aggregation rules here are wrong there.
Procedure
-
Choose the frame set from the wording, not from convenience.
| Task says | Frame set | Command |
|---|
| "keyframes", "I-frames", "important frames" | decoder I-frames, written to disk | ffmpeg -i in.mp4 -vf "select='eq(pict_type,I)'" -vsync vfr frame_%03d.png |
| "count the objects in the video" (no frame wording) | one frame per second | step = max(1, round(fps)) over range(0, total_frames, step) |
-vsync vfr is required; without it FFmpeg pads duplicates and the frame count changes. If the
task names an output pattern, use it exactly — a checker may re-open those files.
-
Apply any demanded image transform to the files on disk, not just in memory. If the task says
"convert the keyframes to grayscale", the saved PNGs must re-read as 2-D arrays
(cv2.imread(path, cv2.IMREAD_UNCHANGED).shape has length 2). Converting inside your script and
saving the original leaves a 3-channel file and fails a structural check.
-
Detect per frame with explicit arguments. Defaults are wrong more often than not.
| Argument | Value | Why |
|---|
classes | [0] for COCO person | otherwise every class is counted |
conf | 0.5 | the 0.25 default admits ghost boxes that break the aggregation |
imgsz | 1280 | surveillance frames have small subjects; 640 drops them |
max_det | 100 | raise the per-frame cap above the plausible crowd size |
verbose | False | per-frame logging swamps the run |
Prefer weights already present on disk over a network download, and point the config dir at a
writable path (YOLO_CONFIG_DIR=/tmp/...) before constructing the model.
-
De-duplicate within a frame when matching a template. cv2.matchTemplate with
TM_CCOEFF_NORMED at threshold 0.9 returns a cloud of near-identical hits per object. Walk the
hits in raster order and keep one only if its distance to every already-kept point exceeds a
small radius (3 px): a greedy non-maximum suppression. Using the raw np.where count multiplies
each object by ~10.
-
Aggregate across frames with a high percentile, never a sum. Collect one count per sampled
frame, sort, and take
idx = min(len(sorted_counts) - 1, int(len(sorted_counts) * 0.90))
distinct = int(sorted_counts[idx])
The reasoning that makes this the right estimator: when everyone is visible for a sustained part
of the clip, the per-frame count is a plateau at the true group size with occasional dips
(occlusion) and rare single-frame spikes (false positives). The 90th percentile reaches the
plateau but trims the spikes. sum() over-counts by an order of magnitude, max() inherits every
false positive, mean() is dragged down by occlusion, and a full tracker adds re-identification
failures the grader will punish.
-
Match the output schema exactly. Counts are integers, not floats. If a workbook is requested,
create exactly the sheets named — an extra default sheet fails a "exactly one sheet" check. Sort
data rows by filename, put the header in row 1, and emit no trailing blank rows.
-
Self-check before finishing. Print the per-frame count list. The reported number must lie
between the median and the maximum of that list; if it equals the sum or the frame count, the
aggregation step was skipped.
Worked example
Count distinct people per clip and write one workbook row per file.
import cv2
from openpyxl import Workbook
from ultralytics import YOLO
def count_people(model, path):
cap = cv2.VideoCapture(path)
fps = cap.get(cv2.CAP_PROP_FPS) or 1
total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT) or 0)
step = max(1, round(fps))
counts = []
for i in range(0, total, step):
cap.set(cv2.CAP_PROP_POS_FRAMES, i)
ok, frame = cap.read()
if not ok:
continue
r = model.predict(frame, classes=[0], conf=0.5, imgsz=1280,
max_det=100, verbose=False)[0]
counts.append(len(r.boxes))
cap.release()
if not counts:
return 0
s = sorted(counts)
return int(s[min(len(s) - 1, int(len(s) * 0.90))])
model = YOLO("yolov8n.pt")
wb = Workbook(); ws = wb.active; ws.title = "results"
ws.append(["filename", "number"])
for name in sorted(video_files):
ws.append([name, count_people(model, name)])
wb.save("count.xlsx")
Template-matching variant for per-frame sprite counts:
res = cv2.matchTemplate(img_gray, tmpl_gray, cv2.TM_CCOEFF_NORMED)
kept = []
for pt in zip(*np.where(res >= 0.9)[::-1]):
if all((pt[0]-q[0])**2 + (pt[1]-q[1])**2 > 3**2 for q in kept):
kept.append(pt)
count = len(kept)
References
