design-tool-use-sandboxing

name

design-tool-use-sandboxing

description

WHEN/WHERE/WHO: [Scheduling: Agents or architects granting LLMs access to terminal commands, file systems, or external APIs.] HOW: [Structural: Use this SKILL to enforce boundary protections, user-approval loops, and containerized execution for dangerous tools.] WHY: [Scheduling: Autonomous agents carry user-level privileges. Unconstrained tool execution leads to catastrophic code deletion or security breaches.]

Design Tool Use Sandboxing

Who

Architects, security engineers, or coding agents integrating tool-use (function calling) capabilities into autonomous systems.

What

Implement secure execution environments and "human-in-the-loop" approval gates for any tools that mutate state, execute arbitrary code, or interact with external networks.

When

• Triggered by requests like: "give the agent a bash tool", "allow the agent to run scripts", or "add secure tool calling".
• Must be used anytime an agent is given write, delete, or execute permissions.

Where

Applies to the tool definition and execution layers of the agent orchestration framework.

Why

When an LLM hallucinates a command like rm -rf /tmp/cache but makes a syntax error resulting in rm -rf / tmp/cache, the host machine is destroyed. Agents must be treated as untrusted users. Proper sandboxing and human-approval gates are mandatory for safety.

Inputs

• Requested Tools: The list of functions the agent needs to call.
• Execution Environment: The target OS or container where tools will run.

Output (Logical Evidence)

• Secure Tool Wrappers: Code that wraps raw functions with validation and approval logic.
• Sandbox Architecture: A Dockerfile or DevContainer configuration for isolated execution.

Optimization Readiness

• Failure Signals: Dangerous tools bypass approval, schemas accept malformed arguments, host execution remains unsandboxed, or the workflow normalizes risky autonomy without clear human gates.
• Evidence To Collect: Wrapper schemas, approval prompts, rejected tool calls, sandbox configs, and examples of blocked versus permitted mutative actions.
• Safe Mutation Boundaries: Refine tool categorization, validation rules, approval UX, and sandbox examples without weakening the core safety boundary between read-only and mutative execution.
• Acceptance Criteria: Accept revisions only if dangerous actions are gated or isolated, malformed inputs are rejected cleanly, and users can inspect exactly what an agent wants to execute.
• Rejected Revision Handling: Record unsafe bypass proposals, weak validation patterns, and ambiguous approval wording so they are not reintroduced.
• Transfer Check: Verify the workflow still works across host execution, containerized execution, and mixed tool environments.
• Stop Rule: If the execution environment cannot provide either safe sandboxing or explicit approval gates, stop and warn instead of enabling dangerous tools.

Constraints (Logical Boundaries)

• Mandatory Approval for Mutative Actions: Any tool that alters the file system, modifies databases, or executes shell commands MUST pause and wait for explicit human approval, unless running in a fully disposable, isolated container.
• Strict Parameter Validation: Tool inputs generated by the LLM must be validated against a strict schema (e.g., Pydantic) before execution.

One More Thing

If the user asks to disable human-in-the-loop approvals for shell commands on their host machine, strongly warn them of the risks and ensure they explicitly accept responsibility before proceeding.

---

How (Structural Workflow)

1. Categorize the Tools

• Safe (Read-Only): Tools like read_file, get_weather, search_web. These can execute autonomously.
• Dangerous (Mutative/Exec): Tools like run_bash, write_file, git_commit, drop_table. These require sandboxing or approval.

2. Implement Input Validation

• Wrap all tools with strict type-checking (e.g., Pydantic BaseModel).
• If an LLM passes a malformed argument, the tool wrapper must catch the exception and return a clear error message back to the LLM so it can self-correct, rather than crashing the orchestrator.

3. Implement Human-in-the-Loop (HITL) Gates

• For dangerous tools running on the host machine, inject an interrupt in the execution graph.
• The system must display the exact command/arguments to the user: Agent wants to execute: 'rm -rf ./build'. Approve? (Y/n).
• If the user rejects, return a message to the agent: "Tool execution rejected by human. Find an alternative approach."

4. Implement Containerized Execution (Optional but Recommended)

• If full autonomy is required for dangerous tools, execute them inside a Docker container or ephemeral virtual machine.
• Mount only the specific workspace directories required; never mount the root filesystem or sensitive directories like ~/.ssh.

5. Add Timeout Limits

• Wrap all tool executions in a timeout block (e.g., timeout=30s) to prevent an agent from running an infinite while true loop script that hangs the orchestrator.