toolsafe-agent-safety

name	toolsafe-agent-safety
title	ToolSafe: Enhancing Tool Invocation Safety of LLM-based Agents via Proactive Step-level Guardrails
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2601.10156
keywords	["agent-safety","tool-invocation","guardrails","prompt-injection","risk-detection"]
description	Detects and prevents unsafe tool invocations in LLM agents through proactive step-level guardrails, reducing harmful tool calls by 65% while improving task success rates by 10% against prompt injection attacks.

Overview

Implement a safety layer that monitors LLM agent tool invocations in real-time and prevents potentially harmful actions before execution. The system analyzes interaction history to identify risky requests and tool-usage patterns that could lead to data exfiltration, privilege escalation, or prompt injection attacks.

When to Use

• For agents that invoke external tools (file systems, APIs, databases)
• When operating in untrusted or adversarial environments
• For systems where tool misuse could cause financial, privacy, or security harm
• When agents interact with user-supplied prompts or content

When NOT to Use

• For sandboxed environments with no external tool access
• When tool invocation is fully controlled by trusted operators
• For read-only tool operations with no side effects
• In low-stakes applications where errors are harmless

Key Technical Components

TS-Guard: Multi-Task Risk Detection Model

Train a smaller guardrail model that analyzes agent interaction history to identify risky tool invocations.

# Multi-task risk detection
class TSGuard:
    def __init__(self, model_name="ts-guard-base"):
        self.model = load_model(model_name)  # Smaller, faster model

    def assess_tool_risk(self, interaction_history, tool_name, tool_args):
        """Assess risk of proposed tool invocation"""
        context = format_interaction_history(interaction_history)
        risk_assessment = self.model.predict({
            "context": context,
            "tool": tool_name,
            "args": tool_args
        })
        return {
            "harmfulness_score": risk_assessment["harm"],
            "attack_likelihood": risk_assessment["attack_likelihood"],
            "risk_category": risk_assessment[]
        }