菜单
Train LLMs to disambiguate tool calls in enterprise settings where multiple similar APIs exist and parameters are incomplete. Generates synthetic multi-turn dialogues with realistic ambiguity to improve tool selection accuracy by 27+ percentage points.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/ADu2021/skillXiv --skill enterprise-tool-calling-finetuning命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
基于 SOC 职业分类
正在显示 SKILL.md
| name | enterprise-tool-calling-finetuning |
| title | Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2507.03336 |
| keywords | ["Tool Calling","Enterprise APIs","Disambiguation","Fine-Tuning","Interactive Agents"] |
| description | Train LLMs to disambiguate tool calls in enterprise settings where multiple similar APIs exist and parameters are incomplete. Generates synthetic multi-turn dialogues with realistic ambiguity to improve tool selection accuracy by 27+ percentage points. |
In enterprise environments, a single business query often maps to multiple near-duplicate APIs, and most real calls arrive missing required parameters. Current LLMs trained on clean, fully-specified tool-calling benchmarks fail dramatically when deployed against production APIs. DiaFORGE solves this mismatch by generating realistic, ambiguous multi-turn dialogues where the agent must disambiguate between competing tools and ask clarifying questions—turning a brittle single-turn problem into a robust interactive one.
The gap between benchmark performance and production reality is massive: static benchmarks use fully-specified queries with one obvious tool, while real APIs have overlapping functionality, required parameters, and user confusion. DiaFORGE bridges this gap through synthetic dialogue generation that mimics real deployment dynamics.
The framework operates on the insight that realistic tool-calling is fundamentally interactive. Rather than expecting agents to infer all details from a single query, train them to:
This interactive approach reduces both false positives (wrong tool calls) and false negatives (refusing to act).
Generate synthetic dialogues using the UTC-Gen multi-agent system. The engine seeds conversations with a ground-truth tool and includes semantic "distractor" tools:
from diaforge.generator import UTCGen
gen = UTCGen(api_catalog="enterprise_apis.json")
dialogue = gen.generate_dialogue(
ground_truth_tool=,
distractors=[
,
,
],
information_stages=[
{: , : []},
{: , : [, ]},
{: , : [, ]}
]
)
(dialogue)
Fine-tune an open-source model (like Llama) on the generated dialogues. Use supervised learning with loss masking to focus on assistant responses:
from diaforge.training import ToolCallingTrainer
trainer = ToolCallingTrainer(model="meta-llama/Llama-2-7b-chat")
# Load generated dialogues
dialogues = load_synthetic_data("generated_dialogues.jsonl")
# Fine-tune focusing only on assistant turns
trainer.train(
dialogues=dialogues,
loss_mask="assistant_only", # Only optimize assistant responses
epochs=3,
batch_size=32,
learning_rate=2e-5
)
Evaluate models using dynamic evaluation where they interact with a user simulator:
from diaforge.evaluation import DynamicEvaluator
evaluator = DynamicEvaluator(
user_simulator="interactive", # Simulated user responds to agent questions
api_catalog="enterprise_apis.json"
)
# Full interactive rollout: agent picks tool, gets feedback
results = evaluator.evaluate(
model=trained_model,
test_scenarios=test_dialogues,
metrics={
"tool_accuracy": "Did agent pick the right tool?",
"false_positives": "Any incorrect tool calls?",
"abstention_rate": "When does agent refuse to act?",
"success_rate": "Did tool call complete the user's goal?"
}
)
print(f"Tool accuracy: {results['tool_accuracy']:.1%}")
print(f"Success rate: {results['success_rate']:.1%}")
Use this approach when:
Avoid this for:
| Scenario | Distractors | Difficulty |
|---|---|---|
| Simple disambiguation | 2-3 similar tools | Baseline |
| Semantic overlap | 4-5 tools with overlapping capabilities | Medium |
| Parameter interdependence | Tools where required params depend on previous choices | Hard |
| Multi-step resolution | Requires 3+ clarifying questions to fully specify | Expert |
Generate ~5,000 dialogues mixing all difficulty levels for robust coverage.
The paper reports DiaFORGE-trained models achieve 27 percentage points improvement over GPT-4o and 49 points over Claude-3.5-Sonnet on dynamic evaluation.
"Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky" - arXiv:2507.03336