risk-analysis

name	risk-analysis
description	Risk identification and test prioritization framework for Node.js + TypeScript projects (Vue 3 + Express + Socket.IO). Provides systematic approach to identifying potential issues and prioritizing testing efforts. Use when analyzing features for QA review.

Risk Analysis Framework

This skill provides a systematic approach to identifying risks in the harness-visualizer codebase and prioritizing testing efforts accordingly. Pair it with audit-rules for review criteria, and testing-patterns for how to write the tests it recommends.

Risk Categories

1. Security Risks (Project-Critical)

For this app the security model is "local user, possibly malicious project on disk" (see AGENTS.md). Missing any of these is a release blocker.

Common patterns:

• Filesystem reads/writes that bypass safeOpen
• Paths not resolved + realpath'd before allowlist check
• Symlink follow-through outside watch roots
• Missing extension allowlist or size cap on editable files
• Missing zod validation on REST or Socket.IO payloads
• Server binding to 0.0.0.0 instead of 127.0.0.1
• child_process invocation with user-controlled paths

Detection questions:

• Does the change touch backend/src/api/ or backend/src/security/?
• Does it read or write files outside test fixtures?
• Does it accept user input via REST body, query, params, or a Socket.IO event?
• Does it follow or compare symlinks?
• Does it shell out to anything?

Severity: Usually CRITICAL.

2. Data Integrity / Correctness Risks

Common patterns:

• Race conditions between concurrent writes (atomic-write pattern not used)
• Chokidar unwatch footgun: parent unwatch silently filters child events
• Scanner emitting stale HarnessEntry[] after pattern reload
• Pinia store updates that mutate arrays in place instead of replacing
• zod schema/type drift (schema validates one shape, type asserts another)
• id collision in derived sha256(scope::path::layer)[:8] truncation
• Pattern migration losing v1 keys without diagnostic

Detection questions:

• Does the change emit a harness:update payload? Does the order of fields match the schema?
• Does it modify watch roots? Does it dedup parent/child overlap first?
• Does it write a file the watcher is observing? Does it trigger redundant rescans?
• Does it add a derived id? Are collisions handled with a fallback?

Severity: HIGH for live-update / scanner code, MEDIUM for UI-only changes.

3. Real-time / Concurrency Risks

Common patterns:

• Race between harness:update debounce and configure-watch reconfiguration
• Socket.IO event order assumptions (clients may reconnect mid-flight)
• Vue watch(refToObject, cb) firing on every replace (use watch(() => ref.value?.id, cb))
• Stale Pinia state after reconnect (no connect handler re-requesting full state)
• Atomic write that races with chokidar pickup before fsync

Detection questions:

• Does the change rely on a Socket.IO message arriving in a specific order?
• Does it add a Vue watch against a ref whose value is replaced wholesale?
• Does it touch debounce or unwatch logic?
• Does it write a file the watcher is observing without going through safeOpen's atomic-write helper?

Severity: HIGH (these manifest as flaky tests + hard-to-reproduce UI bugs).

4. Performance Risks

Common patterns:

• Initial scan on large trees not using fast-glob
• Repeated scans triggered by app-managed file writes (missing **/.harness-visualizer/** in STANDARD_IGNORES)
• v-network-graph re-rendering on every entry mutation (consumes Record<id, T>, not arrays — needs a memoized adapter)
• JSON.parse on every websocket message without size cap
• Synchronous filesystem calls in hot paths

Detection questions:

• Does the change add file IO inside a chokidar event handler without debouncing?
• Does it produce a new array on every render in a Vue computed?
• Does it deserialize untrusted payloads without size limits?

Severity: MEDIUM (LOW unless tree size or message rate is high).

5. Integration Risks

Common patterns:

• chokidar version footguns (v5 unwatch behavior)
• v-network-graph upgrade breaking the Record adapter
• md-editor-v3 API surface changes between minor versions
• npm workspaces hoisting conflicts (revisit pnpm if a 4th workspace lands)
• Tailwind v4 CSS-first config drift from documented @import pattern

Detection questions:

• Does the change bump a peer dependency?
• Does it import from v-network-graph/lib/* (internals)?
• Does it touch the root vite.config.ts (single source of truth for Vite + Vitest)?

Severity: HIGH for dep bumps to chokidar / v-network-graph / md-editor-v3, MEDIUM otherwise.

6. UX Risks

Common patterns:

• Unclear error states in DirectoryPicker / MarkdownEditor (silent failure)
• Missing dark-mode coverage on a new component (useDark not wired through)
• Catalog scroll dead-zones in MarkdownEditor
• Diagnostic icons mismatched with DIAGNOSTIC_CODES

Detection questions:

• Does the change add a new diagnostic code? Is the icon mapping updated?
• Does it render text without dark-mode utility classes?
• Does it add a new interactive component without a Playwright (or Vitest+jsdom) smoke test?

Severity: LOW to MEDIUM.

Risk Scoring Matrix

Severity	Likelihood	Risk Score	Priority
CRITICAL	High	10	P0 - Block release
CRITICAL	Medium	9	P0 - Block release
CRITICAL	Low	8	P1 - Must fix before release
HIGH	High	8	P1 - Must fix before release
HIGH	Medium	6	P1 - Must fix before release
HIGH	Low	4	P2 - Should fix
MEDIUM	High	5	P2 - Should fix
MEDIUM	Medium	4	P2 - Should fix
MEDIUM	Low	2	P3 - Nice to have
LOW	Any	1-2	P3 - Nice to have

Risk Analysis Process

Step 1: Feature Understanding

1. Read the spec under specs/<phase-name>/ (especially requirements.md and design.md).
2. Cross-reference the relevant phase in ROADMAP.md.
3. Map the data flow: REST/Socket.IO entry → scanner/watcher/scoring → Pinia store → component render.
4. List external touchpoints: filesystem reads/writes, watch-root reconfiguration, websocket events.

Step 2: Code Review for Risk Indicators

Scan for these patterns:

// SECURITY RISKS
fs.readFile(userPath)                  // Bypasses safeOpen
fs.writeFile(somePath, data)           // Bypasses safeOpen + atomic write
path.join(root, req.body.path)         // Missing realpath + allowlist check
httpServer.listen(port)                // Default binds to 0.0.0.0 — must be 127.0.0.1
req.body / req.query / req.params      // Untyped — needs zod parse
child_process.exec(`cmd ${userInput}`) // Shell-out with user input

// DATA INTEGRITY / WATCHER RISKS
watcher.unwatch(parent); watcher.add(child) // Chokidar v5 recursive-ignore footgun
fs.writeFile(path, data)                    // Non-atomic; use tmp + rename via safeOpen helper
sha256(scope + path + layer).slice(0, 8)    // Truncated id — needs collision fallback

// REAL-TIME / VUE RISKS
watch(someRef, cb)                          // Fires on whole-ref replace; use watch(() => x.value?.id, cb)
const arr = computed(() => store.list)      // Returns new array reference each tick

// PERFORMANCE RISKS
for (const file of fs.readdirSync(root))    // Sync IO in hot path
const data = JSON.parse(socketMessage)       // No size cap on untrusted payload

Step 3: Risk Documentation

For each identified risk, document:

## RISK-001: [Short Description]

**Category**: Security / Data Integrity / Real-time / Performance / Integration / UX

**Severity**: CRITICAL / HIGH / MEDIUM / LOW

**Likelihood**: High / Medium / Low

**Risk Score**: [1-10]

**Description**:
[Detailed description of the risk]

**Location**:
- File: `backend/src/watcher/watcher.ts`
- Line: 45-67

**Potential Impact**:
[What could go wrong if this risk materializes — be concrete: "watcher silently drops events for the live-update demo, breaking the wow moment"]

**Recommended Tests**:
1. [Specific Vitest test that would catch this — file + describe/it names]
2. [Another test]

**Mitigation Strategy**:
[How the code should handle this risk — reference the relevant rule from audit-rules where applicable]

Step 4: Test Strategy Output

Generate a prioritized test strategy:

# Test Strategy: [Feature Name]

## Critical Path Tests (P0)
1. [Test 1] - Addresses RISK-001 — `backend/src/api/safe-open.test.ts`
2. [Test 2] - Addresses RISK-002 — `backend/src/watcher/watcher.test.ts`

## High Priority Tests (P1)
3. [Test 3] - Addresses RISK-003
4. [Test 4] - Addresses RISK-004

## Standard Tests (P2)
5. [Test 5] - Addresses RISK-005

## Optional Tests (P3)
6. [Test 6] - Nice to have coverage

Project-Specific Risk Patterns

Scanner / Watcher risks

Pattern	Risk	Test Strategy
unwatch(parent) before add(child)	Silent event filtering (chokidar v5)	Reconfigure roots, assert child events still arrive
Missing **/.harness-visualizer/** in ignores	Infinite rescans on app-managed writes	Write patterns.json, assert exactly 1 rescan fires
id derived from sha256(...)[:8] collision	Two distinct entries with same id	Construct a collision scenario; assert :N suffix fallback applied
Symlink resolves outside watch root	Path-traversal via crafted symlink	Failing-on-purpose test: symlink to /etc/passwd, expect 403

Socket.IO / real-time risks

Pattern	Risk	Test Strategy
Unknown event name from client	Server crash / silent ignore	Emit unknown event, assert unknown-event diagnostic
harness:update debounce coalesced too aggressively	Missed rapid edits	Burst-write 5 files within 250 ms, assert single coalesced update
Reconnect mid-pipeline	Stale frontend state	Disconnect/reconnect, assert client re-requests full state

Vue / Pinia risks

Pattern	Risk	Test Strategy
watch(ref, cb) on whole-object ref	Callback fires on every replace (perf + loops)	Trigger several updates, assert cb only fires on id change
Pinia store mutated in place	Reactivity loss / stale renders	Mount component, mutate store, assert DOM updates
v-network-graph consuming Node[] directly	Type mismatch (expects Record<id, T>)	Render with adapter, assert nodes appear; render without, fail

Security risks (failing-on-purpose tests are mandatory)

Pattern	Risk	Test Strategy
Path with .. traversal segment	Read outside watch root	POST /api/file?path=../../etc/passwd, expect 403
Symlink to /etc/passwd	Bypass via symlink follow	Create symlink, GET /api/file, expect 403
2 MB file write	Bypass size cap	POST 2 MB body, expect 413
.sh extension write	Extension allowlist bypass	POST path=foo.sh, expect 403
0.0.0.0 bind	Network exposure	Boot server, assert address() returns 127.0.0.1

Output Format

When analyzing a feature, produce a risk analysis document with:

1. Executive Summary: Brief overview of findings, headline counts by severity
2. Risk Register: Detailed list of all identified risks (use the RISK-NNN format above)
3. Test Strategy: Prioritized list of recommended tests, each linked to a risk by id
4. Coverage Recommendations: Which directories under backend/src/ or frontend/src/ need the most attention
5. Estimated Test Count: Breakdown by test type — unit (*.test.ts colocated), integration (backend/__test-helpers/withTmpRoot), component (@vue/test-utils + jsdom)