| name | human-in-the-loop-bypass |
| description | Review AI agent systems for approval step circumvention, human-in-the-loop bypass patterns, deferred confirmation exploitation, and social engineering techniques that cause human reviewers to approve malicious agent actions. |
| last_reviewed | "2026-04-30T00:00:00.000Z" |
Human-in-the-Loop Bypass
First Principle
A human approval step that can be bypassed, confused, or deferred is not a control — it is theater.
Human-in-the-loop (HITL) controls are placed at the highest-risk points in AI pipelines — irreversible actions, high-value transactions, sensitive data access. They exist precisely because automated checks are insufficient. An attacker who can bypass, confuse, or socially engineer the approval step removes the last line of defense. HITL bypass is not just an AI security problem; it combines agentic attack techniques with classical social engineering.
Attack Mental Model
- Approval context manipulation — the agent presents a misleading summary to the human reviewer. The approval UI shows "send email to team" while the underlying action is "send email to external attacker domain." The human approves based on the summary, not the underlying action.
- Deferred execution injection — the agent obtains approval for action A, then queues action B as a "follow-up" that executes automatically without triggering a second approval. The attacker's payload is in action B.
- Approval fatigue exploitation — a high-volume workflow generates many low-risk approval requests. Reviewers habituate to approving quickly. The attacker inserts a high-risk request that is approved because the reviewer is in approval-mode.
- Out-of-band approval circumvention — a system is designed so that the agent can take the action directly if the human does not respond within a timeout. The attacker delays the notification and lets the timeout lapse.
Control Lens
| Principle | What It Means Here |
|---|
| Validate | The approval UI presents the exact parameters of the action to be executed — not a model-generated summary. What the human sees is what the system will execute. |
| Scope | Approvals are scoped to a single, specific, immutable action. They cannot be extended to cover follow-up actions or action chains not present at approval time. |
| Isolate | Timeout-based auto-approval does not exist. A non-response results in cancellation, not execution. |
| Enforce | All approved actions are logged with the reviewer identity, the exact parameters shown at approval time, and the parameters actually executed. Discrepancies are treated as incidents. |
HIL.1 Approval Context Integrity
The core vulnerability: Human reviewers approve a description. If the description is generated by the agent — which may be operating on attacker-influenced content — the description can be crafted to appear benign while the underlying parameters are malicious.
Check
- Does the approval UI display the raw, structured action parameters (destination, payload, scope) — or a model-generated prose summary?
- Is there a cryptographic link between the parameters shown at approval time and the parameters executed after approval? Can the agent modify action parameters between display and execution?
- Are action parameters locked at the point of approval request creation and verified unchanged at execution time?
Action
- Lock action parameters at approval creation time and hash them:
import hashlib, json
from dataclasses import dataclass
@dataclass
class ApprovalRequest:
action_type: str
parameters: dict
parameter_hash: str
requested_by: str
requested_at: str
def create_approval_request(action_type: str, parameters: dict) -> ApprovalRequest:
canonical = json.dumps(parameters, sort_keys=True)
param_hash = hashlib.sha256(canonical.encode()).hexdigest()
return ApprovalRequest(
action_type=action_type,
parameters=parameters,
parameter_hash=param_hash,
requested_by=current_agent_id(),
requested_at=utcnow_iso(),
)
def execute_after_approval(approval_id: str, approval_decision: ApprovalDecision):
request = load_approval_request(approval_id)
if not approval_decision.approved:
return
canonical = json.dumps(request.parameters, sort_keys=True)
if hashlib.sha256(canonical.encode()).hexdigest() != request.parameter_hash:
log_security_incident("approval_parameter_tampered", approval_id)
raise IntegrityError("Action parameters changed after approval was shown")
execute_action(request.action_type, request.parameters)
- Show structured parameters in the approval UI, not prose summaries. If the action is "send email", show
To: external@example.com | Subject: ... | Body: (first 200 chars) — not "send a summary email to the team."
- Mark attacker-influenced content in the approval context. If action parameters include content derived from user input, external documents, or retrieved web content, flag those fields visually as untrusted-origin.
Failure Modes
- An agent presents "Archive old project files" for approval. The underlying
archive tool takes a destination parameter that the agent set to attacker-ftp://.... The reviewer approves based on the description.
- An agent receives approval for "send report." It sends the report and also queues a follow-up action to "cc external contacts" that executes automatically because it is classified as a sub-task of the approved action.
HIL.2 Approval Scope Containment and Anti-Fatigue
The core vulnerability: Approval scope creep and reviewer fatigue both undermine HITL. Broad approvals allow action chains; fatigue causes reviewers to approve without scrutiny.
Check
- Does an approval for action A authorize any follow-up actions? Can the agent chain actions by presenting subsequent steps as implied by the original approval?
- Is the approval queue volume monitored for sudden spikes that may indicate approval fatigue manipulation?
- Is timeout-based auto-approval implemented anywhere in the system? If a reviewer does not respond, what happens?
Action
- Prohibit approval chaining. Each approval authorizes one specific action instance — not a class of actions, not a workflow. A new high-risk action in the same workflow requires a new approval request, even if it seems logically related.
- Eliminate auto-approval timeouts. Non-response must result in cancellation with a notification to the reviewer explaining what was cancelled and why. Document this policy and enforce it in code, not only in policy.
def check_approval_timeout(approval_id: str) -> None:
request = load_approval_request(approval_id)
if utcnow() - request.requested_at > timedelta(hours=4):
cancel_approval_request(
approval_id,
reason="timeout_no_response",
notify_reviewer=True,
)
- Monitor and alert on approval queue volume anomalies. A sudden 10× spike in approval requests is a signal of either a bug or an attacker flooding the queue to cause fatigue. Alert and pause new requests for review.
Minimum Deliverable Per Review
Quick Win
Eliminate timeout-based auto-approval. Find every approval request handler and verify non-response results in cancellation. This single change removes the most common HITL bypass pattern — the attacker who just waits.
References
