control-testing-procedures

name	control-testing-procedures
version	1.0.0
description	Write internal-audit-grade Test of Design (TOD) and Test of Operating Effectiveness (TOE) procedures for an organization's controls, covering audit objective, test steps, population and sampling, and evidence to obtain. Use when a user asks how to test a control, wants a test plan, test script, or audit program, asks what evidence proves a control is operating, or uploads a control list to be made auditable — including when they never say "TOD" or "test of design". Do NOT use for writing the controls themselves, mapping controls to frameworks, or drafting policies.
metadata	{"openclaw":{"category":"productivity","requires":{"bins":"[Truncated]"}}}

Control Testing Procedures

What this skill does

Turns a control statement into a defensible two-layer test program: a Test of Design confirming the control is built to meet its intent at a point in time, and a Test of Operating Effectiveness confirming it actually operated across the audit period. The two layers fail independently — a control can be well designed and never run, or run constantly while testing the wrong thing — so every control gets both. Done means the control owner could execute the test from the text alone, and an auditor would accept the resulting evidence.

Inputs

• Control statement (required) — control ID, name, and description. Supplied by the user, or pulled from Vanta.
• Framework mapping (optional) — SOC 2, ISO 27001/42001, HIPAA, PCI, NIST, or internal. Sets the assertion the test must support. Default: infer from the control language and label the inference.
• Audit period (optional) — TOE steps are meaningless without one. Default: use [audit period] as a placeholder rather than inventing dates.
• Implementation detail (optional) — who runs the control, how often, in which system, and whether it is monitored automatically. Default: infer a likely implementation and flag the assumption so the user can correct it.

Data access

None required — this skill works from a control statement the user pastes in. Where the vanta CLI is available and authenticated, use it to confirm which evidence already exists. See the vanta skill for auth and global flags.

Need	Command
The org's controls, with descriptions, domains, and owners	vanta controls list --page-size 100
Controls for one framework	vanta controls list --framework-matches-any <frameworkId> --page-size 100
One control	vanta controls get --id <controlId>
Framework IDs, when the user scoped by name	vanta frameworks list --page-size 100
Automated tests already covering a control	vanta controls list-tests --id <controlId> --page-size 100
Evidence documents already attached to a control	vanta controls list-documents --id <controlId> --page-size 100
Personnel, policy, computer, and vendor records as evidence sources	vanta people list, vanta policies list, vanta monitored-computers list, vanta vendors list

Pagination: pass --page-size 100 and follow nextCursor into --page-cursor until pageInfo.hasNextPage is false. The API allows 50 requests per minute — pace per-control lookups across a large library.

Read-only: this skill drafts test procedures and never modifies a control, its tests, or its documents.

Steps

1. Classify the control. Two axes, both stated in one line at the top of the output.

   • Orientation: operational (recurring activity leaving discrete instances — access reviews, onboarding, vendor reviews, training, incidents; has a real population, so TOE is population + sampling); capability (a built or configured property — encryption, a sandbox, a kill switch; no recurring population, so TOE leans on functional re-test); or hybrid (a configured capability that also emits countable runtime instances — test both layers).
   • Evidence source: automated/system-generated (prefer full-population testing; sampling is a fallback, not a default) or manual (sampling applies, and the test must confirm judgment was exercised, not just that a box was ticked).

2. Write the audit objective. One sentence, phrased as what the test must verify rather than what the control says, naming the assertion: existence, completeness, accuracy, authorization, timeliness, or restriction. Weak: "Verify that access reviews are performed." Strong: "Verify that access to in-scope production systems is reviewed at the defined frequency by an independent reviewer, that inappropriate access identified was revoked, and that the review covered the complete population of users."

3. Draft Test of Design steps. Two to four numbered, imperative steps answering whether the control as built addresses the risk. Cover: the governing artifact and whether it defines frequency, scope, ownership, and thresholds rather than gesturing at them; the enforcing mechanism and whether it is non-bypassable; scope across the full in-scope population; and the failure path, since a control with no defined consequence for an exception detects and nothing more. If the design is broken, report a design deficiency and say that effectiveness testing on it is not worth performing.

4. Draft Test of Operating Effectiveness steps. Select strategies from the Step 1 classification — most controls need two or three together:

   • Full-population testing — analyze every instance, report exception counts rather than a sample. Preferred wherever the data supports it.
   • Attribute sampling — define population, select, test against attributes.
   • Functional re-test — exercise the control including at least one negative case that should be blocked. A control never observed refusing something has not been tested.
   • Completeness reconciliation — reconcile against an independent source (HRIS, IdP, asset inventory, change records). Catches items that never entered the process; sampling can never substitute for it.

Output format

For 1–3 controls, deliver in the conversation using this structure per control:

## [Control ID] — [Control Name]

**Classification.** [Operational | Capability | Hybrid]; [automated | manual |
hybrid] evidence. Framework: [mapping]. Test = [strategy].

**Audit objective.** [One sentence.]

### Test of Design (point-in-time)
1. ...

### Test of Operating Effectiveness ([audit period])
1. ...

**Population and sampling.** [Definition, completeness source, sample size and
rationale, stratification.]

**Evidence to obtain.** [Artifact — source system; ...]

**Exception handling.** [What counts as a deviation and what to do on finding
one — omit where it would only restate the standard expansion rule.]

For 4 or more controls, deliver as a CSV written to the working directory (or a path the user names), one row per control, with these columns in order:

control_id, control_name, framework_mapping, classification, evidence_source, audit_objective, test_of_design, test_of_operating_effectiveness, population_and_sampling, evidence_to_obtain, exception_handling, assumptions

Number multi-step procedures inline within the cell ("1. … 2. …") so each row stays a single record. Report the file path; do not reproduce the CSV contents in the conversation. Alongside the file, summarize only what the user needs to act on, since nothing was confirmed before generating: close-call classifications and what changes if they are wrong, controls with no evidence source available today, and assumptions that would alter the test. Keep that summary under roughly 150 words and point to the affected control IDs rather than restating their rows — the file is the deliverable.

Guardrails

• Never include customer data, credentials, system identifiers, or sampled record contents in output.
• Never assert that a specific automated test or Vanta check covers a control unless you confirmed it — from the control's own test listing or from the user. Otherwise describe the evidence type and let them confirm the mapping.
• Label every inferred implementation detail as an assumption, in the assumptions column or inline, so a wrong inference is visible rather than silently baked into the test.
• Do not write effectiveness tests for a control whose design is deficient — report the design gap first.