Menu
Responsible for creating policies, generating and evaluating test cases, testing policies, and automatically improving them.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
| name | smith |
| description | Responsible for creating policies, generating and evaluating test cases, testing policies, and automatically improving them. |
This skill is responsible for creating policies, generating and evaluating test cases, testing, and automatically improving the policy. The target policy file is ./assets/policy.rego.
Before running any smith commands, activate the project virtual environment:
source .venv/bin/activate
If the user asks to create an OPA policy, you should strictly follow instructions in ./opa_policy/policy_creation/opa_policy_creation.md in the skill directory to create the policy without asking any additional questions.
After completion, remind the user: “The policy has been created. Next steps you can take: (1) generate test cases, (2) if you already have test cases, you can ask me to test the policy.”
If the user asks to generate test cases, you should strictly follow instructions in ./test_generation/test_generation.md in the skill directory.
After completion, remind the user: “Test cases have been generated. Note that some generated cases may have incorrect labels (allow vs disallow). Next steps: If you already have a policy, run policy testing.”
Adversarial (red-team) coverage depends on the ATTACK_TOOLS env var (ARES and/or Promptfoo; may be none). The CLI prints which tools were enabled vs skipped. If any attack tool was skipped, tell the user that adversarial coverage is reduced or absent, so “all tests passed” later reflects the legitimate cases only and not resistance to red-teaming.
If the user asks to test an existing policy, you should run smith --flag policy_testing.
After reporting results, remind the user of next steps based on the outcome:
./opa_policy/policy_cross_validation/policy_cross_validation.md to diagnose and fix structural/syntax issues before proceeding.If the user asks to cross-validate failed test cases, follow the instructions in ./test_generation/cross_validate.md in the skill directory.
After completion, remind the user: “Cross-validation is complete.
When the user asks to update or improve their policy, or you identify failed test cases, you should follow the workflow below strictly:
Run smith --flag policy_testing to identify FP/FN cases. Report the results clearly to the human.
After reporting, remind the user: “Next step: I will patch the policy to fix failed cases. Shall I proceed?”
If any false positives or false negatives are detected, read and follow ./opa_policy/policy_patch/policy_patch.md to fix the failed cases. Remember, policy_patch needs to fix all clusters before you move to the next step.
After patching, remind the user: “All failed cases have been patched. Next step: format the policy with Regal to fix style issues. Shall I proceed?”
After all clusters and failed cases are fixed, read and follow ./opa_policy/policy_regal/policy_regal.md to format your policy.
After formatting, remind the user: “Policy has been formatted. Next step: check for and remove duplicate rules. Shall I proceed?”
Read and follow ./opa_policy/policy_duplication/policy_duplication.md to reduce duplication in your policy.
After deduplication, remind the user: “Duplication check complete. The policy enhancement workflow is finished. You can re-run smith --flag policy_testing to confirm all tests still pass.”