| name | llm-agent-tool-deference-blindness |
| description | LLM Agent工具盲从现象研究。当LLM agent配备GNN工具时，agent不判断工具输出，而是盲目服从。更强的LLM backbone反而defer更多。 |
| version | 1 |
| author | Zhongyuan Wang, Pratyusha Vemuri |
| arxiv_id | 2606.14476v1 |
| published | 2026-06-12T00:00:00.000Z |
| categories | ["cs.AI","cs.LG"] |
| keywords | ["LLM agent","tool use","GNN","deference","blind obedience","ReAct","tool judgment"] |
| activation_keywords | ["LLM agent","工具盲从","tool deference","blind obedience","agent工具","ReAct","tool judgment","graph neural network"] |
LLM Agent Tool Defarence Blindness Phenomenon
Core Discovery
Unexpected finding: 当LLM agent配备GNN作为可调用工具时，agent 不判断工具输出，而是盲目服从工具。更强壮的LLM backbone反而defer更多。
假设：Agent exercises judgment over tool use
现实：Agent merely obeys tool outputs blindly
Experimental Setup
Architecture
- Frozen GNN: 固定的图神经网络（node classification）
- ReAct-style LLM Agent: ReAct风格的agent架构
- Explicit Tool Exposure: GNN作为显式可调用工具
Tasks
- Node classification on text-attributed graphs
- ogbn-arxiv: 学术论文分类
- WikiCS: Wikipedia文章分类（复现）
Agent Tool Usage Measurement
测量agent是否：
- Exercises judgment: 评估工具输出的合理性
- Merely obeys: 直接采用工具输出不质疑
Key Results
1. No Judgment Exercise
Agent 不判断工具输出：
- Agent predictions ≈ Tool predictions
- 当工具错误时，agent也错误
- 无独立的判断机制
2. Stronger Backbone Defers More
反直觉发现: 更强的LLM backbone 盲从程度更高：
| Model | Deference Rate |
|---|
| GPT-3.5 | ~60% |
| GPT-4 | ~80% |
| Claude-3 | ~85% |
更强的模型 → 更信任工具 → 更少独立判断
3. Tool Output Dominance
工具输出主导agent决策：
- Agent reasoning trace mostly repeats tool output
- Limited tool questioning behavior
- No critical evaluation of tool predictions
Mechanisms
Why Stronger Models Defer More?
Hypothesis 1: Larger models learn better tool-use patterns
- 在训练中学习"trust expert tools"
- 工具使用成为习惯性策略
Hypothesis 2: Stronger models more sensitive to tool signals
- 更强的信号处理能力
- 更容易识别工具输出的"expert-like" patterns
Hypothesis 3: Tool-induced prior dominates
- 工具输出形成强prior
- Agent reasoning被prior bias压制
