Menu
[QA Method] Online bug monitoring from Application Insights: query both layers, dedup by fingerprint, triage new/spiking signatures, reproduce HIGH-confidence bugs live, report. Detect-and-report only — never files JIRA or auto-fixes.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Initialize / onboard this agentic-QA plugin onto a deployment. Installs deps, then asks the operator only what genuinely shapes the config — the environment NAME, the bug tracker (Jira / Azure Boards), the code host (GitHub / Azure Repos), and an auth preference per axis (PAT recommended, else browser/CLI login). Everything else — whether it is a native-platform or a CLIENT project, the client org, the contribution mode, the fork account — is DERIVED from the token + the filled env + a live module/repo scan. Writes project-profile.json + .env.<env> + .env.local + .mcp.json and verifies access. The whole point is to make /qa-fix route each bug to the RIGHT repo (client custom code vs native platform) and file to the RIGHT tracker. Use when standing the plugin up on a new machine or for a new customer.
Initialize / onboard this agentic-QA plugin onto a deployment. Installs deps, then asks the operator only what genuinely shapes the config — the environment NAME, the bug tracker (Jira / Azure Boards), the code host (GitHub / Azure Repos), and an auth preference per axis (PAT recommended, else browser/CLI login). Everything else — whether it is a native-platform or a CLIENT project, the client org, the contribution mode, the fork account — is DERIVED from the token + the filled env + a live module/repo scan. Writes project-profile.json + .env.<env> + .env.local + .mcp.json and verifies access. The whole point is to make /qa-fix route each bug to the RIGHT repo (client custom code vs native platform) and file to the RIGHT tracker. Use when standing the plugin up on a new machine or for a new customer.
[QA Methodology] Gather ALL fresh CI prerelease artifacts for a change (modules + platform + vc-frontend) and deploy them together to the test env (vc-deploy-dev@<TEST_ENV branch>) in ONE manifest update: resolve a tracker ticket's linked PRs across all repos (or an explicit --module/--platform/--theme/--pr set) → each PR's latest vc3prerelease build → minimal-diff repin of backend/packages.json (AzureBlob/BlobName + PlatformVersion) and theme/artifact.json → dry-run combined diff (default) or a gated deploy PR (direct same-repo when the account has write, else a fork PR) → --verify polls the env-branch pin + /api/platform/modules per target. Never merges (a human merges to deploy); writes route through gh's keyring token; prints the web-edit URL when it can't push. Unblocks /qa-test PR#N and /qa-verify-fix.
[QA Method] Triangulate each BL invariant against docs + live + source code, auto-apply confirmed changes to business-logic.md, and reconcile test-case coverage. Delegates the live axis to qa-testing-expert; runs the triangulation via ba-system-analyzer.
Bring up a local Virto Commerce stack (backend + storefront + DB + ES) via start-local, pinned to the ACTUAL deployed package manifest (vc-deploy-dev @ vcptcore-demo); optionally augment it with the module/PR versions a JIRA task needs. Use when asked to spin up / run / provision a local VC environment, reproduce a deployed env locally, or stand up an env to test a specific ticket.
[QA Method] Defect management lifecycle: JIRA Bug Workflow, triage, classification, report validation, verification protocol, defect metrics.
| name | qa-monitoring |
| description | [QA Method] Online bug monitoring from Application Insights: query both layers, dedup by fingerprint, triage new/spiking signatures, reproduce HIGH-confidence bugs live, report. Detect-and-report only — never files JIRA or auto-fixes. |
| argument-hint | [frontend|backend|both] [--since=MIN] [--dry-run] |
Methodology + reference for the /qa-monitoring command and its headless twin
ci/run-monitor.ts. The command file (commands/qa-monitoring.md) is the
terminal entry; this skill holds the data flow, the KQL probe library, the triage
taxonomy, and the dedup model. Detect-and-report only: the flow stops at a
report + Teams alert and a human decides whether to /qa-bug or /qa-fix.
Real production-like defects in the QA environment (storefront JS exceptions, failed xAPI/REST calls, backend 5xx, failed dependencies) sit in Application Insights unread until someone happens to query the portal. This flow polls App Insights on a cadence, keeps a memory of what it has already seen, and surfaces only genuinely new or surging problems — confirmed live before anyone is asked to act.
Like /qa-investigate, determine TEST_ENV and resolve the active App Insights
resources from the environment — never assume a fixed resource name:
APPINSIGHTS_APP_ID_BACKEND / APPINSIGHTS_API_KEY_BACKEND — platform/backend.APPINSIGHTS_APP_ID_STOREFRONT / APPINSIGHTS_API_KEY_STOREFRONT — storefront.APPINSIGHTS_RESOURCE_BACKEND / _STOREFRONT — display names, only for portal links.AZURE_SUBSCRIPTION_ID / AZURE_RESOURCE_GROUP — for portal links.
A layer with no App ID/key is skipped with a clear message; some envs have none.timespan param.reports/monitoring/.seen-fingerprints.json. Only NEW/SPIKING
proceed; cap at MONITOR_MAX_SIGNALS and log deferrals.REAL_BUG with a code-fixable REPRO_LAYER
is reproduced live by the layer's QA expert. Reproduced → draft bug report with a
## Fix Routing block; not reproduced → NEEDS_REVIEW (listed, not drafted).ci/monitoring/queries/)| Probe | Layer | Surfaces |
|---|---|---|
backend-exceptions.kql | backend | server-side exceptions by problemId |
backend-failed-requests.kql | backend | 5xx requests by route template |
backend-failed-dependencies.kql | backend | failed SQL/Redis/HTTP/payment deps |
frontend-exceptions.kql | frontend | browser JS exceptions by problemId |
frontend-browser-failures.kql | frontend | failed browser AJAX (path normalized) |
Each query summarizes by a stable signature with count_, first/last seen, and a
sample operation_Id for a portal deep-link. Signatures use route/path templates,
not concrete URLs, so they don't drift with test data (feedback_env_resilience).
ci/agents/monitor-triage-agent.md)REAL_BUG · KNOWN_ISSUE · NOISE · CONFIG_GATED · THIRD_PARTY · TRANSIENT.
Oracles: vc-bug-catalog.md (familiar failures), business-logic.md (BL-* violations
are always high severity), debugging-signals.md (benign-noise filter),
platform-patterns.md (expected desync/cache behavior). When ambiguous, prefer
NEEDS_REVIEW over REAL_BUG — a log line alone is not a defect.
ci/lib/fingerprint-store.ts)env|layer|probe|normalized-signature). The store is shared across
envs (vcst, vcptcore, …); the env (= TEST_ENV) keeps each env's signatures separate
inside the one file, so a vcst decision never masks the same signature in vcptcore.new|triaged|confirmed|noise|filed).count ≥ max(baseline×factor, baseline+minDelta) (factor/minDelta env-tunable).classify() reads the previous baseline; recordRun() updates it — order matters.actions/cache (see .github/workflows/monitor.yml).MONITOR_LAYERS (both|frontend|backend) · MONITOR_SINCE_MIN (35) · MONITOR_MAX_SIGNALS
(15) · MONITOR_SPIKE_FACTOR (3) · MONITOR_SPIKE_MIN_DELTA (20) · MAX_BUDGET_USD (15)
· MODEL · DRY_RUN (triage only — no repro/drafts).
commands/qa-monitoring.md · Twin: ci/run-monitor.ts## Fix Routing): commands/qa-bug.md Step 4 → /qa-fixreports/monitoring/ category: .claude/rules/reports.md/qa-bug → /qa-fix → /qa-verify-fix