d-core-task-decomposition

name	d-core-task-decomposition
title	D-CORE: Incentivizing Task Decomposition for Complex Tool Use
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.02160
keywords	["Task Decomposition","Reasoning Models","Tool Use","Self-Distillation","GRPO"]
description	Mitigate lazy reasoning in Large Reasoning Models via self-distillation teaching task decomposition, followed by Diversity-Aware GRPO with entropy-based advantage functions, enabling effective decomposition without external teachers while balancing structured reasoning with diversity.

D-CORE: Incentivizing Task Decomposition via Self-Distillation and DA-GRPO

Large Reasoning Models often exhibit "lazy reasoning"—generating extensive but ineffective processes instead of decomposing complex tasks into executable subtasks. D-CORE addresses this through a two-stage approach: self-distillation that bootstraps decomposition learning without external teachers, followed by Diversity-Aware GRPO (DA-GRPO) that encourages high-entropy reasoning while maintaining structured decomposition.

Core Concept

The key insight is that lazy reasoning and over-diversification are dual problems. Standard self-distillation produces overly-structured trajectories with low diversity, while GRPO trained on sparse rewards encourages entropy but loses decomposition structure. D-CORE solves this by combining bootstrapped decomposition learning with entropy-based advantage functions that reward both correctness and reasoning diversity.

Architecture Overview

• Self-Distillation Stage: Prompt LRM to decompose queries, generate reasoning, apply SFT without external data
• Entropy-Aware GRPO (DA-GRPO): Substitute entropy for advantage when standard advantage approaches zero (sparse reward case)
• Bootstrapping Process: Iteratively improve decomposition quality by training on synthesized data
• Diversity-Correctness Balance: Entropy encourages exploration; correctness reward provides signal
• Generalization: Demonstrated across multiple benchmarks with 5-30% improvements

Implementation

Step 1: Self-Distillation via Bootstrapped Decomposition

Generate and synthesize training data for task decomposition.

def self_distillation_stage(model, queries, reasoning_model, num_iterations=3):
    """
    Bootstrap task decomposition without external teacher.
    1. Prompt model to decompose query
    2. Generate reasoning for subtasks
    3. Compose into trajectory
    4. Apply SFT on synthesized data
    """
    synthesized_trajectories = []
    
    for iteration in range(num_iterations):
        for query in queries:
            # Step 1: Prompt decomposition
            decomposition_prompt = f"""
            Break down this complex task into executable subtasks:
            
            
            Format: 
            Subtask 1: [description]
            Subtask 2: [description]
            ...
            """
            decomposition = model.generate(decomposition_prompt, max_tokens=)
            subtasks = parse_subtasks(decomposition)
            
            
            trajectory = {: query, : decomposition, : []}
            current_context = 
            
             subtask  subtasks:
                reasoning_prompt = 
                
                reasoning = reasoning_model.generate(reasoning_prompt, max_tokens=)
                tool_calls = extract_tool_calls(reasoning)
                
                
                tool_results = []
                 tool_call  tool_calls:
                    result = execute_tool(tool_call)
                    tool_results.append(result)
                
                trajectory[].append({
                    : subtask,
                    : reasoning,
                    : tool_calls,
                    : tool_results
                })
                
                
                current_context += 
            
            
            final_prompt = 
            final_answer = model.generate(final_prompt, max_tokens=)
            trajectory[] = final_answer
            
            synthesized_trajectories.append(trajectory)
        
        
        sft_model = model
        optimizer = torch.optim.Adam(sft_model.parameters(), lr=)
        
         trajectory  synthesized_trajectories:
            
            trajectory_text = 
            trajectory_text += 
             step  trajectory[]:
                trajectory_text += 
                trajectory_text += 
            trajectory_text += 
            
            
            logits = sft_model(trajectory_text, return_logits=)
            loss = compute_language_model_loss(logits, trajectory_text)
            loss.backward()
            optimizer.step()
            optimizer.zero_grad()
    
     model