sop-monitoring-blueprints

Use this skill when building, deploying, evaluating, debugging, or measuring latency for the DeepStream SOP Inference Microservice — a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection (GEBD) plus VLM classification. Trigger even if the user does not name it: verify operator step sequence, detect missing or out-of-order SOP steps, score factory/work-cell video for procedure compliance, run VLM-based SOP checking on industrial cameras, or call /v1/chat/completions with a file, RTSP, or Basler camera. Also trigger for its internals: SOPVideoProcessor, DeepStream GEBD model (e.g. DDM) via Triton CAPI, nvds_custom_postprocess, Cosmos Reason 1/2 vLLM, SSE streaming, Kafka NvProto/JSON output, Basler/Pylon camera + emulation, Docker compose, chunk-level latency. Do NOT trigger for generic DeepStream pipelines, object detection/tracking, NIM imports, or video summarization.

Fine-tune Cosmos-Reason2 (CR2) VLM for SOP monitoring. Use when you need to launch and monitor a VLM training run with a given dataset ID.

Use when the user wants to run data augmentation on an annotated dataset, configure augmentation parameters, check augmentation status, or understand what each QA augmentation type does (BCQ, MCQ, GQA, DMCQ, DSQA, ENQA)

Fine-tune DDM-Net temporal boundary detector for SOP monitoring. Use when you need to launch and monitor a DDM-Net training run with a given dataset ID.

Use when running by-action VLM evaluation (per-action-clip inference + accuracy metrics) against the BP evaluation-ms HTTP API. Invoked as /sop-by-action-eval <inputs.yaml> [natural language parameter overrides]

Use when running the e2e evaluation pipeline (temporal segmentation + action recognition + accuracy) against the BP evaluation-ms HTTP API. Invoked as /sop-e2e-inference <inputs.yaml> [natural language parameter overrides]

Autonomous end-to-end orchestrator for SOP fine-tuning. Runs the full Import → Augment → DDM Train → VLM Train → Evaluate → RCA loop. Interprets RCA findings across DDM, VLM and augment axes, applies config fixes autonomously, and iterates until success criteria are met or max_pipeline_iterations reached. Call with a path to an inputs.yaml or with natural language.

Root cause analysis for SOP monitoring pipeline failures. Analyzes end-to-end evaluation logs, DDM temporal segmentation, VLM action recognition, training data, and fine-tuning configs to identify failure patterns and produce an evidence-driven RCA report with actionable improvement recommendations.

Orchestrate the end-to-end SOP pipeline, including preflight prerequisite checks, verifying models and downloading assets, generating the DeepStream SOP microservice with RTSP output, evaluating the microservice, and building, deploying, and testing the VSS SOP blueprint. Use when asked to run the full SOP pipeline, set up the SOP pipeline from scratch, execute preflight checks, verify models, download assets, generate the SOP microservice, evaluate the microservice, build the VSS blueprint, deploy the VSS blueprint, test the VSS blueprint, or manage the complete build-evaluate-deploy-test cycle.

Interact with the VIOS (Video IO & Storage) microservice in a running VSS profile — manage cameras/sensors, RTSP streams, recordings, snapshots, and storage. Use when asked to add a camera, add an RTSP stream, list sensors, show configured sensors/cameras/streams, what sources are available, check stream status, start/stop recording, get a snapshot, or manage video storage. Always query the VIOS API directly — do not navigate the UI to answer these questions.

Generate and query incident reports from VSS — look up incidents in Elasticsearch, analyze incident patterns, generate narrative reports. Use when asked about incidents, incident reports, PPE violations, safety events, or "what happened". Requires the alerts profile to be deployed.

Query video analytics incidents, alerts, sensor data, and metrics from Elasticsearch via the VA-MCP server (port 9901). Use for any question about what happened in video — PPE violations, alerts, incidents, object counts, speeds, occupancy, or anything that requires looking up recorded events. This is the primary way to answer "what happened", "show me alerts", "any violations", "how many people", etc.

Search video archives using natural language — find events, objects, actions, and people across recorded video using Cosmos Embed1 semantic search. Use when asked to search for something in video, find events, locate objects, or query video archives. Requires the search profile to be deployed.

Summarize long videos, generate shift reports, and analyze extended recordings. Use when asked to summarize a video, generate a shift summary, analyze a long recording, or create a daily activity report. Requires the LVS profile to be deployed.

Build a custom VSS SOP blueprint from the VSS 3.1 base, then deploy and test in a loop until fully operational. Use when asked to create the SOP blueprint structure, customize VSS compose for SOP, configure SOP services, set up the VSS agent for SOP, or scaffold the SOP app layer on top of met-blueprints 3.1.

Build the DS-SOP Docker image and deploy the VSS SOP blueprint end-to-end. Use when asked to deploy SOP, build DS SOP, install SOP, set up the SOP pipeline, verify SOP models, start the SOP blueprint, simulate RTSP for SOP, run the SOP API test, or tear down SOP.

Run post-deployment tests for the VSS SOP blueprint. Checks service health, ELK data pipeline, VIOS recording/livestream, and VSS agent (MCP, LLM, VLM, snapshot, video, report). Auto-debugs failures. Use when asked to test SOP, verify SOP deployment, check SOP services, validate SOP, run SOP health checks, or troubleshoot SOP after deploy.