// Produce video analysis reports by discovering the deployed VSS agent, querying POST /generate for a timestamped captioned summary of the clip, then formatting the agent reply as the standard Video Analysis Report markdown.
| name | report |
| description | Produce video analysis reports by discovering the deployed VSS agent, querying POST /generate for a timestamped captioned summary of the clip, then formatting the agent reply as the standard Video Analysis Report markdown. |
| license | Apache-2.0 |
| metadata | {"version":"3.1.0","github-url":"https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization","tags":"nvidia blueprint operational"} |
Build timestamped video analysis reports by querying the VSS agent for a description of the video using POST …/generate. The agent runs video_understanding (and related tools) internally. Take the agent’s caption-style text with timestamps and paste it into the Video Analysis Report template below.
<sensor-id>"This skill requires the VSS base profile running on the host at $HOST_IP. Before any request:
Probe the VSS agent:
curl -sf --max-time 5 "http://${HOST_IP}:8000/docs" >/dev/null
If the probe fails, ask the user:
"The VSS
baseprofile isn't running on$HOST_IP. Shall I deploy it now using the/deployskill with-p base?"
/deploy skill. Return here once it succeeds.(If your caller has granted explicit pre-authorization to deploy
autonomously — e.g. the request says "pre-authorized to deploy
prerequisites", or you are running in a non-interactive evaluation
harness with that permission — skip the confirmation and invoke
/deploy directly.)
If the probe passes, proceed.
Run these steps in order:
Sensor / clip — Confirm which sensor id or video the user means. If unclear, ask before proceeding. If the sensor or video is not mentioned directly in the user request, the user may be referring to a video they mentioned previously.
VSS agent deployment — Resolve the agent HTTP base URL. Read VSS_AGENT_PORT, EXTERNAL_IP / HOST_IP, or compose / deployment docs for the machine where the stack runs. Typical pattern: http://<host>:<port> with port from env (often 8000 for the agent API).
Query the agent — POST ${VSS_AGENT_BASE_URL}/generate with JSON {"input_message": "<prompt>"}. Ask for a captioned summary with timestamps (chronological segments, seconds from clip start), e.g. describe scenes and events with time ranges. The sensor / file name must be included in the input message to the agent.
Report template — Copy the agent’s final text (timestamped caption/summary) into Analysis Results and fill Basic Information; return that markdown to the user. 0l
/generate)# Set from deployment (compose / .env / host where vss-agent listens)
export VSS_AGENT_BASE_URL="http://localhost:8000"
curl -s -X POST "${VSS_AGENT_BASE_URL}/generate" \
-H "Content-Type: application/json" \
-d '{"input_message": "Describe in detail what happens in the video for sensor <sensor-id>, with timestamps (start–end in seconds from clip start) for each segment or event."}' | jq .
Paste the agent’s timestamped summary under Analysis Results. Fill the table fields (timestamps, source, request).
# Video Analysis Report
## Basic Information
| Field | Value |
|-------|-------|
| **Report Identifier** | vss_report_<YYYYMMDD_HHMMSS> |
| **Date of Analysis** | <YYYY-MM-DD> |
| **Time of Analysis** | <HH:MM:SS> |
| **Reporting AI Agent** | <e.g. your label> |
| **Video Source** | <sensor_id or filename> |
| **Analysis Request** | <description of user's request to you> |
## Analysis Results
<agent output: timestamped caption / summary>
/generate patterns; this skill focuses on timestamped captions → report template.Deploy, debug, or tear down any VSS profile using a compose-centric workflow — config (dry-run) with env overrides, review resolved compose, then compose up. Use this skill when the user says "deploy vss", "deploy `profile`", "debug deploy", "verify deployment", or "why is my vss deploy broken".
Use this skill when working with the RTVI VLM or RT-VLM microservice API on VSS 3.1. Generate dense captions and alerts for stored video files and live RTSP streams via `/v1/generate_captions_alerts`; upload media via `/v1/files`; add and remove live streams with `/v1/streams/add` and `/v1/streams/delete/{stream_id}`; call OpenAI-compatible `/v1/chat/completions`; consume Kafka caption, incident, and error topics; or debug rtvi-vlm responses. For deployment, read `references/deploy-rt-vlm-service.md` first.
Summarize a video by calling the VLM NIM or the Long Video Summarization (LVS) microservice directly. For short videos (under 60s) call the VLM's OpenAI-compatible chat completions endpoint; for long videos (60s or longer) call the LVS microservice. Use when asked to summarize a video, describe what happens in a video, analyze a recording, call or debug LVS summarize/model/health/recommended-config/metrics endpoints, or configure and troubleshoot the LVS service that backs long-video summarization.
Query VIOS REST APIs: sensor list, recording timelines, video clip extraction, snapshot capture, add/delete sensors and streams
Manage and monitor VSS alerts after the alerts profile is deployed. The deployment's mode (CV vs VLM real-time) is fixed at deploy time and determines the workflow — start/stop real-time alerts via the VSS Agent on a VLM deployment, onboard CV alerts by adding RTSP streams to VIOS on a CV deployment, query incidents, customize verifier prompts. Use when asked to start/stop a real-time alert, check or list alerts, add a camera, use a sample video for alerts, customize alert prompts, or view verdicts.
Query video analytics data and metrics from Elastic search via the VA-MCP server (port 9901). This includes incidents, alerts, sensor data, and metrics. Use for any question about violations, alerts, incidents, object counts, speeds, occupancy, or anything that requires looking up recorded events. This is the primary way to answer a question that requires incidents, alerts and other metrics such as people counts and violations.