vss-generate-video-report

name	vss-generate-video-report
description	Use this skill when producing a VSS analysis report — Mode A per-clip VLM, Mode B incident-range via video-analytics, Mode C SOP compliance via the SOP tools. Not for standalone video summarization, real-time alerts or ad-hoc Q&A.
license	Apache-2.0
metadata	{"version":"3.3.0","author":"NVIDIA Video Search and Summarization team","github-url":"https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization","tags":"nvidia blueprint operational"}

Report

Generate a video analysis report by routing to one of three backends — never via POST /generate on the VSS agent.

Mode	Backend
A. Video clip	A1 /vss-manage-video-io-storage → clip URL → VLM chat/completions OR A2 local video file on disk or base64 video + explicit VLM endpoint
B. Incident range	/vss-query-analytics → incident list → narrative report
C. SOP compliance	VA-MCP get_sop_report (direct MCP call on ${VA_MCP_URL}) → SOP compliance report

If the request is ambiguous (e.g. "report on <sensor>" with no time range and no incident wording), default to Mode A. Ask only if the user mentions both a sensor and a time range. See Examples below for the request phrasings that route to each mode.

---

Instructions

0. Set SKILL_DIR to the "Base directory for this skill" path announced when this skill loads. All skill-relative reads (e.g. the default VLM prompt) resolve under $SKILL_DIR — never via cwd-relative paths.
1. Pick the mode — Mode A for a single recorded clip/sensor video, Mode B when the request names a time range or incidents/alerts, Mode C when the request asks for an SOP / compliance report (match against Examples).
2. Verify runtime prerequisites for that mode under Runtime prerequisites; hand off only when required services are missing (Mode A / B → /vss-deploy-profile; Mode C → /vss-build-vision-ai for the SOP tools).
3. Apply HITL mode under HITL prompt mode (legacy runtime flag) before Mode A Step 3. (Mode B and Mode C have no prompt-approval step.)
4. Run that mode's numbered steps — Mode A, Mode B, or Mode C below.
5. Rewrite every user-facing clip URL before embedding it in the report: prefer VSS_PUBLIC_URL origin rewrite on Kubernetes; fall back to $VSS_PUBLIC_HOST:$VSS_PUBLIC_PORT on Docker Compose (Browser-playable clip URL).
6. Return the rendered report markdown to the user.

Output contract for evaluators:

• Mode A top title MUST be exactly # Video Analysis Report.
• Mode A MUST include ## Basic Information followed by a pipe-table (Field | Value) with the exact required rows from the template: Report Identifier, Date of Analysis, Time of Analysis, Video Source, Clip Range, Clip URL, VLM, Analysis Request — every row filled with concrete values.
• Mode A MUST include ## Analysis Results containing the VLM caption/summary (with any <think>…</think> block stripped).
• Mode B top title MUST be exactly # Incident Range Report (never # Incident Report or sensor-named variants).
• Mode B MUST include ## Basic Information with the exact required rows from the template (Report Identifier, Range, Scope, Total Incidents, Confirmed / Rejected / Unverified).
• Mode B MUST use heading level # for the top title. Do not use ## Incident Report, ## Incident Range Report, or any alternate wording.
• Mode B empty-range output MUST be exactly one plain-text line (no markdown heading/table/list/extra lines) in this format: No incidents found for scope <scope> in range <start_time> to <end_time>.
• Mode C top title MUST be exactly # SOP Compliance Report, with the template's Basic Information / Compliance Summary / SOP Violations sections.

---

Examples

• "Generate a report for this video" / "report on <sensor-id>" → Mode A
• "Analyze warehouse_01.mp4" / "create an analysis report on the uploaded video" → Mode A
• "Report on incidents from 12:31Z to 12:32Z" → Mode B
• "Report on alerts today" / "what incidents happened on <sensor> last hour" → Mode B
• "Summarize alerts on <sensor> between <t1> and <t2>" → Mode B
• "Generate an SOP compliance report for <sensor> from <t1> to <t2>" / "compliance report on <sensor> last hour" / "SOP status report for <sensor>" → Mode C

---

Negative Triggers

Do not use this skill when the request is one of the following:

• Ad-hoc visual Q&A on a clip that do not ask explicitly for a report ("what color is the truck?", "what happens at 00:12?") → use /vss-ask-video.
• Archive/semantic similarity retrieval ("find forklifts", "search all videos for tailgating") → use /vss-search-archive.
• Read-only incident/metrics lookup without report rendering needs → use /vss-query-analytics.
• Deploy/teardown/profile changes ("deploy alerts", "switch profile", "bring up base") → use /vss-deploy-profile.
• Real-time alert/rule management requests → use /vss-manage-alerts.

Never route reports through VSS-agent POST /generate.

---

Runtime prerequisites

This skill is profile-agnostic for Mode A. A specific profile does not have to be pre-deployed as long as the chosen Mode A input path and VLM path are available. Mode C needs a VA-MCP that exposes the SOP tools (get_sop_*) over Elasticsearch mdx-vlm-captions-* — deployed by the SOP profile (compose via /vss-build-vision-ai; see skills/vss-build-vision-ai/references/services/sop.md § Patch specifics).

Endpoint resolution (Kubernetes vs Docker)

When operating against a deployed VSS stack (base, lvs, or alerts on Helm), resolve public endpoints once. Follow ../vss-build-vision-ai/references/deployment_resolution.md:

if [ -n "${VSS_PUBLIC_URL:-}" ]; then
  DEPLOYMENT_KIND="kubernetes"
  VSS_PUBLIC_URL="${VSS_PUBLIC_URL%/}"
  VSS_VIOS_URL="${VSS_PUBLIC_URL}/vst"
  VST_API_BASE="${VSS_VIOS_URL}/api/v1"
  # RT-VLM is at /rtvi-vlm on every profile; nothing is mounted at the origin /v1.
  : "${VLM_ENDPOINT:=${VSS_PUBLIC_URL}/rtvi-vlm/v1}"
  # Alerts / Mode B — force public VA-MCP; ignore leftover Docker :9901.
  VA_MCP_URL="${VSS_PUBLIC_URL}/va-mcp"
else
  DEPLOYMENT_KIND="docker"
  VSS_VIOS_URL="http://${HOST_IP}:30888/vst"
  VST_API_BASE="${VSS_VIOS_URL}/api/v1"
  VA_MCP_URL="http://${HOST_IP}:9901"
fi

On Kubernetes, do not use kubectl port-forward, Service DNS, NodePorts, or host-side container discovery for VIOS, the VLM, or VA-MCP. Mode A uses ${VST_API_BASE} and ${VLM_ENDPOINT} only; Mode B uses ${VA_MCP_URL}.

Mode-by-mode checklist (required)

Mode / Path	User must provide	Services that must be reachable	Storage/location requirement	Not required
Mode A / A1 (VIOS clip URL)	sensor and/or clip time range	VIOS + VLM endpoint	Clip is fetched from VIOS timeline/URL APIs	VA-MCP analytics
Mode A / A2 (local file or base64)	local VIDEO_FILE path or VIDEO_BASE64, plus explicit VLM endpoint/model	VLM endpoint only	For VIDEO_FILE, file must exist on the same machine/container filesystem where OpenClaw/agent executes and be readable by that process	VIOS, VA-MCP analytics
Mode B (incident range)	start_time / end_time (and optional sensor scope)	VA-MCP analytics (/vss-query-analytics + video_analytics__get_incidents)	Incident data must already exist in analytics backend for requested range/scope	VIOS, direct VLM path
Mode C (SOP compliance)	sensor and time range (relative phrases resolved against host clock)	VA-MCP with the SOP tools (get_sop_*) on ${VA_MCP_URL} + Elasticsearch mdx-vlm-captions-*	SOP detection docs must already be indexed for the requested sensor/range	VIOS, direct VLM path, report-time VLM

Hard gate behavior:

• If required services for the chosen row are not reachable, stop and report the missing dependency.
• Do not silently switch modes because a dependency is missing.
• Offer /vss-deploy-profile only after user confirmation.

Probe examples:

# Mode A path A1 — VIOS reachable
curl -sf --max-time 5 "${VST_API_BASE}/sensor/version" >/dev/null

# Mode A — VLM reachable (Kubernetes public /v1, or caller-supplied / Docker host port)
curl -sf --max-time 5 "${VLM_ENDPOINT:-http://${HOST_IP}:30082/v1}/models" >/dev/null

# Mode B — VA-MCP reachable via /health (K8s: ${VA_MCP_URL}/health; Docker: :9901/health)
curl -sf --max-time 5 "${VA_MCP_URL:-http://${HOST_IP}:9901}/health" >/dev/null

# Mode C — reachability is NOT sufficient; also REQUIRE the SOP tools on VA-MCP:
# tools/list on ${VA_MCP_URL}/mcp (two-step JSON-RPC, see Mode C Step 1) must include
# video_analytics__get_sop_report. If absent, the deployment lacks the SOP patch —
# hand off to /vss-build-vision-ai and do NOT proceed with Mode C.

If required local services are missing and the user wants local deployment, hand off to /vss-deploy-profile (typically -p base for Mode A path A1, -p alerts for Mode B), or to /vss-build-vision-ai to compose the SOP profile for the SOP tools (Mode C). Always confirm deploy with the user first.

---

VLM selection when unclear

If VLM/deployment choice is unclear and no default selection has been made, ask the user what VLM to use with these options:

1. Provide an endpoint — user supplies VLM_ENDPOINT and model id.
2. Use the public Ingress VLM — when VSS_PUBLIC_URL is set, probe ${VSS_PUBLIC_URL%/}/rtvi-vlm/v1/models (the RT-VLM mount, same on every profile). Do not use /vlm/v1 or the bare origin /v1.
3. Suggest options based on auto-discover — on Docker, probe the standard local VLM ports. For shared VLM-selection guidance, follow /vss-ask-video.
4. Deploy a local VLM — hand off to /vss-deploy-profile (with user confirmation) and then continue.

Auto-discover hints:

# Kubernetes / public Ingress (preferred when VSS_PUBLIC_URL is set)
if [ -n "${VSS_PUBLIC_URL:-}" ]; then
  curl -sf --max-time 5 "${VSS_PUBLIC_URL%/}/rtvi-vlm/v1/models" | jq -r '.data[].id'
fi

# Docker only — probe common local endpoints without inspecting any container.
if [ "${DEPLOYMENT_KIND:-docker}" != "kubernetes" ]; then
  curl -sf --max-time 5 "http://${HOST_IP}:30082/v1/models" | jq -r '.data[].id'   # local NIM / base default
  curl -sf --max-time 5 "http://${HOST_IP}:8018/v1/models" | jq -r '.data[].id'    # RT-VLM / alerts default
fi

---

HITL prompt mode (runtime-first, harness fallback)

Resolve HITL mode for Mode A only in this order:

1. Runtime config video_report_gen.hitl_enabled (legacy VSS source of truth)
2. Harness override HITL_ENABLED=true|false (fallback only when runtime config is unavailable)
3. If neither source is set, default to false

Behavior:

• resolved false: do not ask clarification; run Mode A with the current default prompt.
• resolved true: before Mode A Step 3, show the current prompt and ask the user to choose one of:
  • APPROVE — use the current prompt as-is.
  • EDIT: <instructions> — apply edits to the current prompt and show the revised prompt.
  • NEW: <full prompt> — replace with a brand-new prompt.

Guardrails (required):

• Do not treat yes, confirm, ok, or whitespace-only text as approval.
• Do not wait for an empty-string confirmation.
• Keep showing the same three choices (APPROVE | EDIT: ... | NEW: ...) after every EDIT or NEW response.
• Do not run report generation until the user explicitly responds with APPROVE.
• If the response is ambiguous, re-prompt with explicit APPROVE | EDIT: ... | NEW: ... options and continue the loop.
• If Step 3 resolves HITL via rule (3) (neither runtime nor fallback is set), include this note on the first report generation response in the session: HITL mode not set; defaulting to off. Set HITL_ENABLED=true to enable HITL.

---

Clip URLs: VLM input vs browser report link

VST may return clip URLs using an agent-internal host:port (Compose ${HOST_IP}:30888, or an in-cluster name). Keep that original URL as VIDEO_URL for local / in-cluster VLM frame pulls when the VLM can reach it. Do not rewrite the VLM input URL just to make it browser-playable.

Only create BROWSER_CLIP_URL for URLs shown in the rendered report.

Kubernetes — rewrite to the public Ingress origin and keep the clip under the public VIOS route. Ingress serves VIOS only under /vst, and VIOS /url responses return a bare /storage/temp_files/... path (and can carry a doubled http:// scheme — upstream Finding 8). Swapping only the authority would produce ${VSS_PUBLIC_URL}/storage/..., which Ingress hands to the UI catch-all instead of VIOS. Reduce to a path, then restore /vst — the same compat mapping Docker HAProxy applies:

: "${VSS_PUBLIC_URL:?Set VSS_PUBLIC_URL before rewriting clip URLs on Kubernetes}"
CLIP_PATH=$(printf '%s' "${RAW_URL}" | sed -E 's|^(https?://)+||; s|^[^/]*||')
case "${CLIP_PATH}" in
  /vst/*) BROWSER_CLIP_URL="${VSS_PUBLIC_URL%/}${CLIP_PATH}" ;; # already public VIOS
  /storage/*) BROWSER_CLIP_URL="${VSS_PUBLIC_URL%/}/vst${CLIP_PATH}" ;; # bare VIOS path
  *)
    echo "Cannot construct a public VIOS clip link from: ${RAW_URL}" >&2
    BROWSER_CLIP_URL=""
    ;;
esac

Verify the result before putting it in the report — it must begin with ${VSS_PUBLIC_URL}/vst/. Probe with GET, not HEAD: VST lazy-renders clips and returns 404 to HEAD until a GET materializes the file. If the URL fails either check, omit it from the report and call out why; do not block local VLM analysis:

case "${BROWSER_CLIP_URL}" in