vss-generate-video-report

# Report Generate a video analysis report by routing to one of three backends — **never via** `POST /generate` on the VSS agent. | Mode | Backend | Steps | |---|---|---| | **A. Video clip** | `A1` `/vss-manage-video-io-storage` → clip URL → **VLM chat/completions** OR `A2` local video file on disk or base64 video + a VLM endpoint (caller-supplied or Step 2-discovered) | [`references/report-types/video-analysis.md`](references/report-types/video-analysis.md) | | **B. Incident range** | `/vss-query-analytics` → incident list → narrative report | [`references/report-types/incident-range.md`](references/report-types/incident-range.md) | | **C. SOP compliance** | VA-MCP `get_sop_report` (direct MCP call on `${VA_MCP_URL}`) → SOP compliance report | [`references/report-types/sop-compliance.md`](references/report-types/sop-compliance.md) | If the request is ambiguous (e.g. "report on `<sensor>`" with no time range and no incident wording), default to **Mode A**. Ask only when the request names a sensor and a time range **and** carries neither incident / alert wording (→ Mode B) nor SOP / compliance wording (→ Mode C). Never run any mode's probe or gate until the mode is settled. See **Examples** below for the request phrasings that route to each mode. --- ## Instructions 0. **Set `SKILL_DIR`** to the "Base directory for this skill" path announced when this skill loads. All skill-relative reads (e.g. the default VLM prompt) resolve under `$SKILL_DIR` — never via cwd-relative paths. If no base directory was announced (this file was opened directly), `SKILL_DIR` is the directory containing this `SKILL.md`. Each fenced block is its own shell and nothing survives it, so the skill hands state over explicitly: the blocks that resolve shared values end by printing shell-quoted `NAME=value` lines (*Endpoint resolution* → `DEPLOYMENT_KIND`, `VSS_PUBLIC_URL`, `HOST_IP`, `VST_API_BASE`, `VA_MCP_URL`, `VLM_ENDPOINT`; Mode A Step 1 → `VIDEO_URL`, `CLIP_START`, `CLIP_END`, `CLIP_SECONDS`; Mode A Step 2 → `VLM_BACKEND`, `VLM_ENDPOINT`, `VLM_MODEL`; the clip-URL rewrite blocks (Kubernetes / Docker) take `RAW_URL` in and print `BROWSER_CLIP_URL`). Paste those lines as printed, plus `SKILL_DIR='<that path>'` (single-quoted — paths may contain spaces) and any gate result (`HITL_RESOLVED` / `HITL_PROMPT_FILE`, *HITL prompt mode*; `REASONING=true` only when the user asked for reasoning), at the top of the next block you run, with any caller-supplied value (e.g. `VLM_ENDPOINT` / `VLM_MODEL`) pasted **after** them so it wins; consuming blocks refuse to run (`${VAR:?}`) when a required value is missing. 1. **Pick the mode** — Mode A for a single recorded clip/sensor video (path `A1` VST clip URL or `A2` local file / base64 — the *Mode-by-mode checklist* rows), Mode B when the request is about incidents / alerts (usually with a time range), Mode C when the request asks for an SOP / compliance report (match against *Examples*). 2. **Verify runtime prerequisites** for that mode under *Runtime prerequisites*; hand off only when required services are missing (Mode A / B on Docker Compose → `/vss-build-vision-ai`; on Kubernetes report the missing public route to the deployment owner instead; Mode C → `/vss-build-vision-ai` for the SOP tools). 3. **Apply HITL mode** under *HITL prompt mode (runtime-first, harness fallback)* before Mode A Step 3 (`references/report-types/video-analysis.md`). (Mode B and Mode C have no prompt-approval step.) 4. **Run that mode's numbered steps** from its report-type file — the *Steps* column of the mode table above; open only the one you routed to, via `$SKILL_DIR/references/report-types/<file>`. This file holds routing, gates and the shared setup (endpoint resolution, VLM selection, HITL, clip-URL rewrite); the numbered steps live only in the report-type files. 5. **Rewrite every user-facing clip URL** before embedding it in the report: prefer `VSS_PUBLIC_URL` origin rewrite on Kubernetes; fall back to `$VSS_PUBLIC_HOST:$VSS_PUBLIC_PORT` on Docker Compose (*Clip URLs: VLM input vs browser report link*). 6. **Return the rendered report markdown** to the user. Output contract for evaluators: - Mode A top title MUST be exactly `# Video Analysis Report`. - Mode A MUST include `## Basic Information` followed by a pipe-table (`Field | Value`) with the exact required rows from the template: Report Identifier, Date of Analysis, Time of Analysis, Video Source, Clip Range, Clip URL, VLM, Analysis Request — every row filled with concrete values (a literal `N/A (<reason>)` as defined in Mode A Step 4 counts; a placeholder token does not). - Mode A MUST include `## Analysis Results` containing the VLM caption/summary (with any `<think>…</think>` block stripped). - Mode B top title MUST be exactly `# Incident Range Report` (never `# Incident Report` or sensor-named variants). - Mode B MUST include `## Basic Information` with the exact required rows from the template (Report Identifier, Range, Scope, Total Incidents, Confirmed / Rejected / Unverified). - Mode B MUST use heading level `#` for the top title. Do not use `## Incident Report`, `## Incident Range Report`, or any alternate wording. - Mode B empty-range output MUST be exactly one plain-text line (no markdown heading/table/list/extra lines) in this format: `No incidents found for scope <scope> in range <start_time> to <end_time>.` - Mode B zero results: do not invent or seed incidents, and do not fall back to Mode A or call the VLM. - Mode C top title MUST be exactly `# SOP Compliance Report`, with the template's Basic Information / Compliance Summary / SOP Violations sections. - Mode C empty-range output MUST be exactly one plain-text line, no heading, table, or template: `No SOP messages found for sensor <sensor_id> in range <start_time> to <end_time>.` `get_sop_report` signals an empty range as the tool result `{"error": "No VisionLLM messages found for the given filters."}` — that result, and only that, renders this line (see `references/report-types/sop-compliance.md` Step 3). A failed `tools/call`, a non-200 status, an empty body or one with no JSON-RPC response for the request (neither SSE `data:` events nor plain JSON), a JSON-RPC `error` envelope, `result.isError: true`, or any other error text is a failure: surface it per *Error Handling* and never render this line for it. --- ## Examples - "Generate a report for this video" / "report on `<sensor-id>`" → **Mode A** - "Analyze warehouse_01.mp4" / "create an analysis report on the uploaded video" → **Mode A** - "Report on incidents from 12:31Z to 12:32Z" → **Mode B** - "Report on alerts today" / "what incidents happened on `<sensor>` last hour" → **Mode B** - "Summarize alerts on `<sensor>` between `<t1>` and `<t2>`" → **Mode B** - "Generate an SOP compliance report for `<sensor>` from `<t1>` to `<t2>`" / "compliance report on `<sensor>` last hour" / "SOP status report for `<sensor>`" → **Mode C** --- ## Negative Triggers Do **not** use this skill when the request is one of the following: - Ad-hoc visual Q&A on a clip that do not ask explicitly for a report ("what color is the truck?", "what happens at 00:12?") → use `/vss-ask-video`. - Archive/semantic similarity retrieval ("find forklifts", "search all videos for tailgating") → use `/vss-search-archive`. - Read-only incident/metrics lookup without report rendering needs → use `/vss-query-analytics`. - Deploy/teardown/profile changes ("deploy alerts", "switch profile", "bring up base") → use `/vss-build-vision-ai`. - Real-time alert/rule management requests → use `/vss-manage-alerts`. Never route reports through VSS-agent `POST /generate`. --- ## Runtime prerequisites This skill is profile-agnostic for Mode A. A specific profile does **not** have to be pre-deployed as long as the chosen Mode A input path and VLM path are available. **Mode C** needs a **VA-MCP that exposes the SOP tools** (`get_sop_*`) over Elasticsearch `mdx-vlm-captions-*` — deployed by the SOP profile (compose via `/vss-build-vision-ai`; see that skill's `references/services/sop.md` § Patch specifics, inside its own directory wherever it is installed). ### Endpoint resolution (Kubernetes vs Docker) Run this block first, for every mode and on both Kubernetes (Helm **base**, **lvs**, **alerts**) and Docker Compose — including A2-only runs: every later block requires the hand-off it prints. Follow [`../vss-build-vision-ai/references/deployment_resolution.md`](../../vss-build-vision-ai/references/deployment_resolution.md): ```bash # VSS_ENDPOINT is the legacy alias for VSS_PUBLIC_URL (deployment_resolution.md); honour it here so # every mode resolves the same way (the Mode C blocks consume the VA_MCP_URL this block prints). [ -n "${VSS_PUBLIC_URL:-}" ] || VSS_PUBLIC_URL="${VSS_ENDPOINT:-}" VLM_ENDPOINT="${VLM_ENDPOINT%/}" # a caller-supplied endpoint keeps working with a trailing slash if [ -n "${VSS_PUBLIC_URL:-}" ]; then DEPLOYMENT_KIND="kubernetes" VSS_PUBLIC_URL="${VSS_PUBLIC_URL%/}" VSS_VIOS_URL="${VSS_PUBLIC_URL}/vst" VST_API_BASE="${VSS_VIOS_URL}/api/v1" # RT-VLM is at /rtvi-vlm on every profile; nothing is mounted at the origin /v1. : "${VLM_ENDPOINT:=${VSS_PUBLIC_URL}/rtvi-vlm/v1}" # Alerts / Mode B and Mode C — force public VA-MCP; ignore leftover Docker :9901. VA_MCP_URL="${VSS_PUBLIC_URL}/va-mcp" else DEPLOYMENT_KIND="docker" echo "No VSS_PUBLIC_URL / VSS_ENDPOINT exported — treating this deployment as Docker Compose (export VSS_PUBLIC_URL for Kubernetes)" >&2 [ -n "${HOST_IP:-}" ] || echo "HOST_IP not exported — assuming localhost; export HOST_IP=<Compose host> if the stack runs elsewhere (e.g. host.openshell.internal in the sandbox)" >&2 HOST_IP="${HOST_IP:-localhost}" # effective host — later blocks read HOST_IP directly, so hand it over resolved VSS_VIOS_URL="http://${HOST_IP}:30888/vst" VST_API_BASE="${VSS_VIOS_URL}/api/v1" VA_MCP_URL="http://${HOST_IP}:9901" fi # Hand-off — shell state does not survive this block: paste these lines, as printed, at the top of # every later block (probes, Mode A Step 2), BEFORE any caller-supplied VLM_ENDPOINT / VLM_MODEL line # (later lines win). VLM_ENDPOINT is printed only when known (Kubernetes route or caller-supplied); # on Docker, Mode A Step 2 discovers it. printf 'DEPLOYMENT_KIND=%q\nVSS_PUBLIC_URL=%q\nHOST_IP=%q\nVST_API_BASE=%q\nVA_MCP_URL=%q\n' \ "$DEPLOYMENT_KIND" "${VSS_PUBLIC_URL:-}" "${HOST_IP:-}" "$VST_API_BASE" "$VA_MCP_URL" [ -z "${VLM_ENDPOINT:-}" ] || printf 'VLM_ENDPOINT=%q\n' "$VLM_ENDPOINT" ``` On Kubernetes, do not use `kubectl port-forward`, Service DNS, NodePorts, or host-side container discovery for VIOS, the VLM, or VA-MCP. Mode A uses `${VST_API_BASE}` and `${VLM_ENDPOINT}` only; Mode B and Mode C use `${VA_MCP_URL}`. ### Mode-by-mode checklist (required) | Mode / Path | User must provide | Services that must be reachable | Storage/location requirement | Not required | |---|---|---|---|---| | **Mode A / A1 (VIOS clip URL)** | sensor and/or clip time range | VIOS + VLM endpoint | Clip is fetched from VIOS timeline/URL APIs | VA-MCP analytics | | **Mode A / A2 (local file or base64)** | local `VIDEO_FILE` path **or** `VIDEO_B64_FILE` (base64 written to a file, never pasted into a shell block), plus a VLM endpoint/model — caller-supplied, or discovered by Mode A Step 2 (Kubernetes public route / Docker ports) | VLM endpoint only | For `VIDEO_FILE`, file must exist on the same machine/container filesystem where OpenClaw/agent executes and be readable by that process | VIOS, VA-MCP analytics | | **Mode B (incident range)** | `start_time` / `end_time` (and optional sensor scope) | VA-MCP analytics (`/vss-query-analytics` + `video_analytics__get_incidents`) | Incident data must already exist in analytics backend for requested range/scope | VIOS, direct VLM path | | **Mode C (SOP compliance)** | sensor and time range (relative phrases resolved against host clock) | VA-MCP with the SOP tools (`get_sop_*`) on `${VA_MCP_URL}` + Elasticsearch `mdx-vlm-captions-*` | SOP detection docs must already be indexed for the requested sensor/range | VIOS, direct VLM path, report-time VLM | Hard gate behavior: - If required services for the chosen row are not reachable, stop and report the missing dependency. - Do not silently switch modes because a dependency is missing. - Offer `/vss-build-vision-ai` only after user confirmation. - Mode A: a clip **120 seconds or longer** never takes the direct VLM path — stop and prompt the user to deploy / use LVS (`/vss-build-vision-ai` + `/vss-summarize-video` on Docker Compose, confirm first; on Kubernetes report the missing `/lvs` route to the deployment owner) — unless LVS is already ready per the Mode A file's LVS check, in which case use it directly — then continue with the report template; details in `references/report-types/video-analysis.md` § Long-video rule. Probe examples: ```bash # Fresh shell: paste the Endpoint resolution hand-off lines above these probes and set the mode you # picked in Instructions step 1 (A1 | A2 | B | C). No Docker fallbacks here — a missing value must # fail loudly, not silently probe the wrong deployment. Only the probes for that mode run and decide # the exit code (non-zero = a required service is missing: stop and report it). : "${REPORT_MODE:?set REPORT_MODE=A1|A2|B|C at the top of this block (Instructions step 1; A1 = VST clip URL, A2 = local file / base64 — the Mode-by-mode checklist rows)}" case "${DEPLOYMENT_KIND:?paste the Endpoint resolution output at the top of this block}" in kubernetes|docker) ;; *) echo "ERROR: DEPLOYMENT_KIND must be kubernetes or docker, got '${DEPLOYMENT_KIND}'" >&2; exit 1 ;; esac VLM_ENDPOINT="${VLM_ENDPOINT%/}" # tolerate a trailing slash on a caller-supplied endpoint FAIL=0 case "${REPORT_MODE}" in A1|A2) if [ "${REPORT_MODE}" = "A1" ]; then # A2 (local file / base64) needs no VIOS curl -sf --max-time 5 "${VST_API_BASE:?paste the Endpoint resolution output at the top of this block}/sensor/version" >/dev/null \ && echo "VIOS ok: ${VST_API_BASE}" || { echo "VIOS unreachable: ${VST_API_BASE}" >&2; FAIL=1; } fi # VLM. Kubernetes: the public route from Endpoint resolution (a missing line is a paste error). # Docker: no endpoint yet — the standard host ports must answer /models (Mode A Step 2 then picks # endpoint + model); if neither does, the VLM is a missing dependency (hand off to /vss-build-vision-ai). if [ -n "${VLM_ENDPOINT:-}" ]; then curl -sf --max-time 5 "${VLM_ENDPOINT}/models" >/dev/null \