toucan-tool-agentic-data-synthesis

name	toucan-tool-agentic-data-synthesis
title	TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environments
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.01179
keywords	["synthetic-data","agent-training","tool-use","MCP","data-generation"]
description	Synthesize large-scale training data for LLM agents by generating diverse tool-use trajectories from real Model Context Protocol (MCP) environments. A 1.5M-example dataset created via multi-stage pipeline: query generation, quality filtering, trajectory creation with real tool execution, validation, and diversification.

TOUCAN: Synthesizing Tool-Agentic Training Data at Scale

The scarcity of high-quality, permissively licensed tool-use training data limits agent research. Existing datasets are small, domain-specific, or closed-source. TOUCAN addresses this by automatically generating 1.5 million trajectories from nearly 500 real-world Model Context Protocol (MCP) environments, creating a diverse, realistic foundation for agent training.

The challenge is that synthetic data generation for agents requires more than prompting LLMs to invent examples. Agents interact with real tools where failures matter. TOUCAN uses actual tool execution to ensure generated trajectories are grounded in reality, not fantasies.

Core Concept

TOUCAN's pipeline generates tool-use trajectories through six stages:

1. Query Generation: Five different models produce diverse tool-use queries (what users ask agents to do)
2. Quality Filtering: Model-based filtering removes low-quality, ambiguous, or infeasible queries
3. Trajectory Creation: Three teacher models generate agent trajectories using two agentic frameworks with real tool execution
4. Validation: Rule-based and model-based validation ensures outputs are correct and follow expected formats
5. Diversification: Three extension mechanisms expand dataset variety (multi-turn conversations, paraphrasing, task variants)
6. Curation: Manual sampling and review to maintain quality across 500 MCPs

Architecture Overview

• Query generator: Prompt ensemble creating diverse tool-use scenarios
• Filter module: Removes infeasible or low-quality queries via model classification
• Tool executor: Real MCP runtime executing trajectories (not simulation)
• Teacher ensemble: Multiple models generating alternative solution paths
• Validator: Step-level and trajectory-level correctness checks
• Extender: Multi-turn dialog simulator, paraphraser, task variant generator

Implementation Steps

Start by generating diverse queries that exercise different tools in an MCP environment:

from toucan import QueryGenerator, MCP_Registry

# Initialize query generation with model ensemble
generators = [
    "gpt-4o",
    "claude-3-sonnet",
    ,
    ,
    ,
]

 ():
    
    queries = []

    
     i, generator_name  (generators):
        prompt = 

        num_per_generator = num_queries // (generators)
        generator_queries = [
            llm_call(generator_name, prompt)
             _  (num_per_generator)
        ]
        queries.extend(generator_queries)

     queries