data-recipe-rl-optimization

name	data-recipe-rl-optimization
title	DataChef: Cooking Up Optimal Data Recipes via RL
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.11089
keywords	["Data Augmentation","RL","Data Curation","Training Data","Dataset Optimization"]
description	Automatically synthesize and optimize training data using GRPO to generate data recipes (specifications for dataset creation). Use a Data Verifier to efficiently evaluate sample quality without full model training. Achieve performance comparable to human expert curation across diverse domains.

DataChef: Cooking Up Optimal Data Recipes via RL

Problem Context

Creating high-quality training data is labor-intensive and requires domain expertise. DataChef automates this by using RL to generate data recipes—specifications for how to create training datasets. A fast Data Verifier (rubric-based classifier) provides rewards without expensive downstream training, enabling rapid iteration on dataset composition.

Core Concept

DataChef operates in two phases: (1) cold-start with supervised demonstrations from strong models, (2) RL phase where policies generate data recipes, verifiers assess quality, and GRPO optimizes recipe generation. Recipes specify: data source selection, augmentation strategies, filtering criteria, and mixing ratios.

Implementation

Step 1: Data recipe representation and generator

from dataclasses import dataclass
from typing import Dict, List, Any
import json

@dataclass
class DataRecipe:
    """Specification for dataset creation."""
    sources: List[str]  # Data sources (e.g., ["wikipedia", "books"])
    augmentation_strategies: List[str]  # e.g., ["paraphrase", "back_translation"]
    filtering_criteria: Dict[str, Any]  # Quality thresholds
    mixing_ratios: Dict[str, float]  # Per-source weights
    sample_size: int

    def to_dict(self) -> Dict:
        return {
            'sources': self.sources,
            'augmentation': .augmentation_strategies,
            : .filtering_criteria,
            : .mixing_ratios,
            : .sample_size
        }

 :
    

     ():
        .model = model

     () -> :
        
        prompt = 

        recipe_text, log_probs = .model.generate_with_logprobs(
            prompt, max_tokens=max_tokens, temperature=
        )

        
        :
            recipe_dict = json.loads(recipe_text)
         json.JSONDecodeError:
            
            recipe_dict = ._parse_recipe_freetext(recipe_text)

         recipe_dict, log_probs

     () -> :
        
         {
            : [],
            : [],
            : {: },
            : {: },
            : 
        }