eval-hooks

# Hooks evaluator Discover every Claude Code and Codex hook in scope, validate each one against the filesystem and the host's documented hook semantics, then run an interactive session to confirm or improve them. The goal is not just to score; it is to leave every hook working, correctly scoped, and safe to run. Sources of truth: the official Claude Code hooks reference (https://code.claude.com/docs/en/hooks) and the Codex hooks documentation (https://developers.openai.com/codex/hooks). When a rule below and the live documentation disagree, the documentation wins; report the disagreement instead of scoring against stale text. ## When to use - First time adding hooks (validate before committing) - A hook never fires, or fires on every tool call - The agent hangs noticeably before executing a tool - A policy hook is supposed to block but doesn't - After copying hooks from another project, machine, or host - Periodic hygiene: "are all these hooks still doing something useful?" ## Scope | Argument | Audit | |---|---| | none | Every Claude Code and Codex location listed below that exists | | `claude` or `codex` | That host only | | a file path | That file only (settings JSON, `hooks.json`, `config.toml`, skill or agent file) | | `audit-only`, or the user asks for no changes | Steps 1-4 and 6 only: list each proposed change instead of asking or editing, and omit the user-feedback lines from the report | Audit each host with its own rules. Never score a Codex hook against Claude Code semantics or the reverse. When a `ctxharness doctor --format json` report generated during this task is available, reconcile against it: every hook-layer finding it reports must appear in the audit, and every extra finding needs `file:line` evidence. --- ## Claude Code: Key concepts ### Event types (33) | Event | When it fires | Exit 2 effect | |---|---|---| | `SessionStart` | Session begins or resumes | Stderr to user only | | `Setup` | `--init-only`, or `--init` / `--maintenance` in `-p` mode | Ignored | | `UserPromptSubmit` | Prompt submitted, before processing | Blocks and erases the prompt | | `UserPromptExpansion` | A typed command expands into a prompt | Blocks the expansion | | `PreToolUse` | Before a tool call executes | Blocks the tool call | | `PermissionRequest` | A tool call needs a permission decision | **Not honored**: deny through the JSON `decision` object | | `PermissionDenied` | Auto mode denies a tool call | Ignored (only `hookSpecificOutput.retry` is read) | | `PostToolUse` | After a tool call succeeds | Stderr to Claude; the tool already ran | | `PostToolUseFailure` | After a tool call fails | Stderr to Claude | | `PostToolBatch` | After a full batch of parallel tool calls | Stops the agentic loop | | `Notification` | Claude Code sends a notification | Ignored | | `MessageDisplay` | While assistant text is displayed | Original text displayed | | `SubagentStart` | A subagent is spawned | Stderr to user only | | `SubagentStop` | A subagent finishes | Prevents the subagent from stopping | | `TaskCreated` | A task is being created | Rolls back the creation | | `TaskCompleted` | A task is being marked completed | Prevents completion | | `Stop` | Claude finishes responding | Prevents stopping, continues the turn | | `StopFailure` | The turn ends on an API error | Ignored (except `terminalSequence`) | | `TeammateIdle` | An agent team teammate is about to go idle | Keeps the teammate working | | `InstructionsLoaded` | A CLAUDE.md or `.claude/rules/*.md` file loads | Ignored | | `ConfigChange` | A configuration file changes mid-session | Blocks the change (except `policy_settings`) | | `CwdChanged` | The working directory changes | Stderr to user only | | `DirectoryAdded` | A directory is added via `/add-dir` or SDK `register_repo_root` | Stderr to debug log; already added | | `FileChanged` | A watched file changes on disk | Stderr to user only | | `WorktreeCreate` | A worktree is being created | Any non-zero exit fails creation | | `WorktreeRemove` | A worktree is being removed | Any non-zero exit fails removal if the directory remains | | `PreCompact` | Before compaction | Blocks compaction | | `PostCompact` | After compaction | Stderr to user only | | `PreModelSwitch` | Before a requested model switch | Blocks the switch | | `PostModelSwitch` | After the session model changes | Stderr to user only | | `Elicitation` | An MCP server requests user input | Denies the elicitation | | `ElicitationResult` | After the user answers an elicitation | Blocks the response (becomes decline) | | `SessionEnd` | Session terminates | Stderr to user only | ### Exit codes and JSON output - **Exit 2** blocks on the events whose row says so. No JSON can override it, not even `permissionDecision: "allow"`. - **JSON is read on every exit code**, not only 0. For events with a standard decision model, a valid JSON decision takes effect on exit 0 or any other non-2 code: `hookSpecificOutput.permissionDecision: "deny"` (PreToolUse, PreModelSwitch), `hookSpecificOutput.decision.behavior: "deny"` (PermissionRequest), top-level `decision: "block"` (UserPromptSubmit, UserPromptExpansion, PostToolUse, PostToolUseFailure, PostToolBatch, Stop, SubagentStop, ConfigChange, PreCompact, TaskCreated, PreModelSwitch). - **Any other non-zero code without valid JSON** is a non-blocking error: the action proceeds and the transcript shows the first stderr line. A missing or non-executable script exits with a code like 127 and lands in this bucket, so a mistyped path silently disables a policy gate. - **Exit 0 stdout** becomes context only on `UserPromptSubmit`, `UserPromptExpansion`, `SessionStart`, and `PostModelSwitch`; elsewhere it goes to the debug log. A policy hook is correctly built when it uses **either** exit 2 **or** a JSON decision supported by its event. Exit 1 alone is the defect. ### Timeout defaults (seconds) | Handler | Default | |---|---| | `command`, `http`, `mcp_tool` | 600 | | same, on `UserPromptSubmit`, `PreModelSwitch`, `PostModelSwitch` | 30 | | same, on `MessageDisplay` | 10 | | `prompt` | 30 | | `agent` | 60 | | `SessionEnd` | 1.5 total budget; a per-hook `timeout` raises it to the highest configured value, up to 60. Plugin hook timeouts do not raise it. `CLAUDE_CODE_SESSIONEND_HOOKS_TIMEOUT_MS` overrides it | A timed-out `command`, `http`, or `mcp_tool` hook on `PreToolUse` does **not** block the tool call; on `PreModelSwitch` it blocks the switch. `timeout` is not enforced on a running `async: true` hook; it is enforced with `asyncRewake`. ### Handler types and where they are allowed | Handler type | Allowed on | |---|---| | `command`, `mcp_tool` | Every event | | `http` | Every event except `SessionStart` and `Setup` | | `prompt` | `PermissionDenied`, `PostToolBatch`, `PostToolUse`, `PostToolUseFailure`, `PreToolUse`, `Stop`, `SubagentStop`, `TaskCompleted`, `TaskCreated`, `TeammateIdle`, `UserPromptExpansion`, `UserPromptSubmit`, `PermissionRequest` | | `agent` (experimental) | Same as `prompt`, except `PermissionRequest`, where an agent hook is skipped | On `PermissionDenied`, prompt and agent hooks run but their output is discarded. ### Handler fields | Field | Applies to | Notes | |---|---|---| | `type` | all | `command`, `http`, `mcp_tool`, `prompt`, `agent` | | `if` | all | One permission rule, such as `Bash(git *)`. Evaluated only on `PreToolUse`, `PostToolUse`, `PostToolUseFailure`, `PermissionRequest`, `PermissionDenied`; on any other event the hook never runs. Best-effort filter, not an enforcement mechanism | | `timeout` | all | Seconds | | `statusMessage` | all | Spinner text | | `once` | all | Honored only in skill frontmatter; ignored in settings files and agent frontmatter | | `command` | command | Shell string, or the executable when `args` is set | | `args` | command | Exec form: no shell, each element one argument. Preferred when referencing `${CLAUDE_PROJECT_DIR}` or `${CLAUDE_PLUGIN_ROOT}` | | `shell` | command | `bash` or `powershell`; ignored when `args` is set | | `async` | command | Runs in the background; cannot block or decide | | `asyncRewake` | command | Background, and wakes Claude when the process exits 2 | | `url`, `headers`, `allowedEnvVars` | http | Blocks only through a 2xx JSON body; status codes alone never block | | `server`, `tool`, `input` | mcp_tool | Uses an already connected server | | `prompt`, `model` | prompt, agent | `$ARGUMENTS` receives the hook input | ### Matchers | Matcher value | Evaluated as | |---|---| | `"*"`, `""`, or omitted | Match all | | Only letters, digits, `_`, `-`, spaces, `,`, `\|` | Exact string, or a list separated by `\|` or `,` (hyphens need v2.1.195+) | | Any other character | Unanchored JavaScript regex (`Edit.*` also matches `NotebookEdit`; anchor with `^...$`) | `FileChanged` and `StopFailure` use a narrower exact set (letters, digits, `_`, `|`). Each event filters a different field: | Event | Matcher filters | |---|---| | `PreToolUse`, `PostToolUse`, `PostToolUseFailure`, `PermissionRequest`, `PermissionDenied` | tool name (`Bash`, `Edit\|Write`, `mcp__server__.*`) | | `SessionStart` | `startup`, `resume`, `clear`, `compact`, `fork` | | `Setup` | `init`, `maintenance` | | `SessionEnd` | `clear`, `resume`, `logout`, `prompt_input_exit`, `other` | | `Notification` | notification type | | `SubagentStart`, `SubagentStop` | agent type | | `PreCompact`, `PostCompact` | `manual`, `auto` | | `PreModelSwitch`, `PostModelSwitch` | canonical target model name | | `ConfigChange` | configuration source | | `DirectoryAdded` | `slash_command`, `register_repo_root` | | `FileChanged` | literal filenames to watch | | `StopFailure` | error type | | `InstructionsLoaded` | load reason | | `UserPromptExpansion` | command name | | `Elicitation`, `ElicitationResult` | MCP server name | **No matcher support** (a matcher is silently ignored): `UserPromptSubmit`, `PostToolBatch`, `Stop`, `TeammateIdle`, `TaskCreated`, `TaskCompleted`, `WorktreeCreate`, `WorktreeRemove`, `CwdChanged`, `MessageDisplay`. ### Execution model - All matching hooks run **in parallel**. - The same handler defined in more than one settings file runs **once**. A plugin's or a skill's copy of the same handler stays separate. - `PermissionRequest` hooks also run in sessions that cannot show a prompt, such as background subagents in `-p` mode; if no hook decides, the call is denied. - Several `PreToolUse` decisions resolve as `deny > defer > ask > allow`. The documentation does not define which `updatedInput` wins when several hooks rewrite the same call. - `Stop` and `SubagentStop` continuations, through `decision: "block"` or `additionalContext`, are bounded by the `stop_hook_active` input and an 8-consecutive-continuation cap. - Command hooks run without a controlling terminal: interactive commands (`read`, `fzf`, `gum`, any TUI) cannot get input. Use `terminalSequence` in JSON output for notifications. --- ## Claude Code: Locations scanned | Location | Scope | Notes | |---|---|---| | Managed policy settings | Organization | Cannot be disabled by user, project, or local `disableAllHooks` | | `~/.claude/settings.json` | User | Not read by cloud sessions | | `.claude/settings.json` | Project, committed | | | `.claude/settings.local.json` | Project, personal | | | Plugin `hooks/hooks.json` | While the plugin is enabled | | | Skill frontmatter `hooks:` | Rest of the session once the skill is invoked | `once: true` honored here only | | Subagent frontmatter `hooks:` | While that subagent runs | `Stop` becomes `SubagentStop`. Project subagent hooks run only after workspace trust is accepted, including in `-p` | `~/.claude/settings.local.json` is **not** a documented settings location. If it exists, report it as undocumented and do not score its hooks as active. Also record, from the effective settings: - `disableAllHooks: true` disables every non-managed hook; only a managed-level value disables managed hooks. - `allowManagedHooksOnly` blocks user, project, local, and plugin hooks (plugins force-enabled in managed `enabledPlugins` are exempt). - `allowedHttpHookUrls` and `httpHookAllowedEnvVars` restrict HTTP hooks from every source. - Interactive sessions hold back every settings-file hook until workspace trust is accepted; `-p` and SDK sessions treat the folder as trusted. --- ## Codex: Key concepts ### Locations | Location | Notes | |---|---| | `~/.codex/hooks.json` | User | | `~/.codex/config.toml` inline `[hooks]` | User | | `<repo>/.codex/hooks.json`, `<repo>/.codex/config.toml` | Loaded only when the project `.codex/` layer is trusted | | Plugin `hooks/hooks.json`, or a `hooks` entry in `.codex-plugin/plugin.json` | Manifest paths must stay inside the plugin root | | Managed `requirements.toml` `[hooks]` | Trusted by policy; `allow_managed_hooks_only = true` skips user, project, session, and plugin hooks | Every source loads; higher layers do not replace lower ones. A layer holding both `hooks.json` and inline `[hooks]` is merged with a startup warning: flag it and recommend one representation per layer. `[features] hooks = false` turns hooks off (`codex_hooks` is a deprecated alias). ### Trust review Every non-managed hook, plugin hooks included, must be reviewed and trusted in `/hooks` before it runs. Trust is recorded against the hook's current hash, so an edited hook is skipped until trusted again. When auditing, report that a changed hook needs review; do not assume it runs. The `/hooks` browser is the authoritative view of trust. Current Codex builds also persist it under `[hooks.state]` in `config.toml`, which is undocumented: report what you observe there without printing hash values, and never treat `[hooks.state]` as an event. `--dangerously-bypass-hook-trust` bypasses the check for one invocation only: flag any automation that relies on it. Check how `codex` is launched before reading trust state as runtime behavior: `type codex` and `which -a codex`. A terminal integration can put a wrapper first on `PATH` that adds `--dangerously-bypass-hook-trust` and `-c hooks...` overrides to every invocation; Codex then lists the flag among its startup warnings. Observed with cmux, whose wrapper skips the injection when `CMUX_CODEX_HOOKS_DISABLED=1`. Under such a wrapper every hook runs without review: report that trust findings do not describe what runs. ### Events (12) and matchers | Event | Matcher filters | |---|---| | `PreToolUse`, `PostToolUse`, `PermissionRequest` | tool name (`Bash`, `apply_patch`, MCP names; `Edit` or `Write` also match `apply_patch`) | | `SessionStart` | `startup`, `resume`, `clear`, `compact` | | `SessionEnd` | currently only `other` | | `PreCompact`, `PostCompact` | `manual`, `auto` | | `SubagentStart`, `SubagentStop` | subagent type | | `UserPromptSubmit`, `Stop`, `Interrupt` | not supported (matcher ignored) | Codex matchers are regex strings. `Interrupt` and `SessionEnd` do not run for subagents. ### Handlers and output - Only `command` and `mcp_tool` handlers run. `prompt` and `agent` handlers are parsed and skipped: flag them as dead configuration.