synthetic-data-generator

name	synthetic-data-generator
description	Synthetic data generation covering LLM-based data creation, statistical data synthesis, privacy-preserving techniques (differential privacy, k-anonymity), tabular data generation, text augmentation, image augmentation, validation frameworks, and quality assurance for generated datasets. Use when the user asks about synthetic data generator, synthetic data generator best practices, or needs guidance on synthetic data generator implementation. Do NOT use when the user needs a different specialized skill or is asking about an unrelated technology domain.
license	Apache-2.0
metadata	{"author":"foundry-skills","version":"1.0.0","tags":"ai-ml data-science guide","category":"ai-machine-learning","subcategory":"applied-ai","depends":"","disclaimer":"none","difficulty":"intermediate"}

Synthetic Data Generator

Overview

Synthetic data generation creates artificial datasets that preserve the statistical properties of real data without exposing sensitive information. This skill covers LLM-based text generation, statistical tabular synthesis, privacy-preserving methods, augmentation strategies, and validation to ensure synthetic data is useful and safe.

When to Use Synthetic Data

Use Case	Why Synthetic	Risk Level
ML training augmentation	Insufficient labeled data	Low
Software testing	Need realistic test fixtures	Low
Data sharing across teams	Privacy regulations (GDPR, HIPAA)	Medium
Bias correction	Underrepresented groups	Medium
Rare event simulation	Insufficient edge case examples	Low
Public datasets / demos	Cannot expose real customer data	High

Decision Tree

Privacy / compliance:
  Sharing externally? -> Full synthetic with differential privacy
  Internal only? -> Pseudonymization may suffice

Training data:
  Enough real data? -> Augmentation for diversity / class balance
  Insufficient? -> Full synthetic generation to bootstrap

Testing / development:
  Need realistic distributions? -> Statistical synthesis from real profile
  Just need structure? -> Rule-based Faker data

Research / demos:
  Must look realistic? -> LLM-generated + statistical synthesis
  Just need volume? -> Simple random generation

Tabular Data Generation

Statistical Synthesis with SDV

from sdv.metadata import Metadata
from sdv.single_table import GaussianCopulaSynthesizer, CTGANSynthesizer
import pandas as pd

class TabularSynthesizer:
    """Generate synthetic tabular data preserving statistical properties."""

    def __init__(self, method: str = "gaussian_copula"):
        self.method = method
        self.synthesizer = None

    def fit(self, real_data: pd.DataFrame) -> None:
        metadata = Metadata.detect_from_dataframe(real_data)
        if self.method == "gaussian_copula":
            self.synthesizer = GaussianCopulaSynthesizer(metadata)
        elif self.method == "ctgan":
            self.synthesizer = CTGANSynthesizer(
                metadata, epochs=300, batch_size=500,
                generator_dim=(256, 256), discriminator_dim=(256, 256),
            )
        self.synthesizer.fit(real_data)

    def generate(self, n_rows: int) -> pd.DataFrame:
        return self.synthesizer.sample(num_rows=n_rows)

Method Comparison

Method	Speed	Quality	Privacy	Best For
Gaussian Copula	Fast	Good	Medium	Continuous data, prototyping
CTGAN	Slow	Very Good	Medium	Mixed types, complex distributions
TVAE	Medium	Good	Medium	Continuous-heavy datasets
Faker (rule-based)	Very Fast	Low-Medium	High	Testing, structure only

Rule-Based with Faker

from faker import Faker
import random
import pandas as pd

fake = Faker()
Faker.seed(42)

def generate_customer_data(n: int = 1000) -> pd.DataFrame:
    records = []
    for _ in range(n):
        records.append({
            "customer_id": fake.uuid4(),
            "name": fake.name(),
            "email": fake.email(),
            "city": fake.city(),
            "signup_date": fake.date_between(start_date="-3y", end_date="today"),
            "plan": random.choices(
                ["free", "basic", "pro", "enterprise"], weights=[0.5, 0.3, 0.15, 0.05],
            )[0],
            "monthly_spend": round(random.lognormvariate(3.5, 1.2), 2),
            "is_active": random.random() > 0.15,
        })
    return pd.DataFrame(records)

LLM-Based Text Generation

from openai import OpenAI
import json

class LLMDataGenerator:
    """Generate synthetic text data using LLMs."""

    def __init__(self, model: str = "gpt-4o-mini"):
        self.client = OpenAI()
        self.model = model

    def generate_batch(
        self, schema: dict, n_samples: int,
        context: str = "", examples: list[dict] = None, batch_size: int = 10,
    ) -> list[dict]:
        all_samples = []
        for i in range(0, n_samples, batch_size):
            current_batch = min(batch_size, n_samples - i)
            prompt = (
                f"Generate exactly {current_batch} synthetic data samples.\n\n"
                f"Schema:\n```json\n{json.dumps(schema, indent=2)}\n```\n\n"
            )
            if context:
                prompt += f"Context: {context}\n\n"
            if examples:
                prompt += f"Examples:\n{json.dumps(examples[:3], indent=2)}\n\n"
            prompt += "Return as JSON: {\"samples\": [...]}\n"

            response = .client.chat.completions.create(
                model=.model,
                messages=[
                    {: , : },
                    {: , : prompt},
                ],
                response_format={: },
                temperature=,
            )
            batch = json.loads(response.choices[].message.content)
            all_samples.extend(batch.get(, []))
         all_samples[:n_samples]

Text Augmentation

Technique	Quality	Speed	Best For
Synonym replacement	Medium	Fast	Simple vocabulary variation
Random deletion	Low-Medium	Fast	Robustness training
Back-translation	High	Slow	Natural paraphrasing
LLM paraphrase	Very High	Slow	High-quality augmentation
Entity swapping	Medium	Fast	Named entity variation

class TextAugmenter:
    def synonym_replace(self, text: str, n: int = 2) -> str:
        """Replace n random words with synonyms (uses WordNet)."""
        words = text.split()
        import random
        indices = random.sample(range(len(words)), min(n, len(words)))
        for idx in indices:
            syn = self._get_synonym(words[idx])
            if syn:
                words[idx] = syn
        return " ".join(words)

    def random_deletion(self, text: str, p: float = 0.1) -> str:
        """Randomly delete words with probability p."""
        import random
        words = text.split()
        remaining = [w for w in words if random.random() > p]
        return " ".join(remaining) if remaining else words[0]

    def llm_paraphrase(self, text: str, style: str = "neutral") -> str:
        """Use an LLM to paraphrase while preserving meaning."""
        client = OpenAI()
        response = client.chat.completions.create(
            model=,
            messages=[{: , : }],
            temperature=,
        )
         response.choices[].message.content.strip()

Image Augmentation

import albumentations as A
from albumentations.pytorch import ToTensorV2

def get_training_augmentation(task: str = "classification") -> A.Compose:
    if task == "classification":
        return A.Compose([
            A.RandomResizedCrop(224, 224, scale=(0.8, 1.0)),
            A.HorizontalFlip(p=0.5),
            A.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2, hue=0.1, p=0.8),
            A.GaussianBlur(blur_limit=(3, 7), p=0.3),
            A.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
            ToTensorV2(),
        ])
    elif task == "detection":
        return A.Compose([
            A.RandomResizedCrop(640, 640, scale=(0.5, 1.0)),
            A.HorizontalFlip(p=0.5),
            A.RandomBrightnessContrast(p=0.5),
            A.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
            ToTensorV2(),
        ], bbox_params=A.BboxParams(format="pascal_voc", min_visibility=, label_fields=[]))

Privacy-Preserving Generation

Differential Privacy

import numpy as np

class DPMechanism:
    @staticmethod
    def laplace_mechanism(true_value: float, sensitivity: float, epsilon: float) -> float:
        """Add Laplace noise for epsilon-differential privacy."""
        return true_value + np.random.laplace(0, sensitivity / epsilon)

    @staticmethod
    def dp_histogram(data: np.ndarray, bins: int, epsilon: float) -> tuple:
        """Create differentially private histogram."""
        counts, edges = np.histogram(data, bins=bins)
        noisy = np.array([max(0, c + np.random.laplace(0, 2 / epsilon)) for c in counts])
        return noisy, edges

Epsilon guidelines:
  < 1.0  -> Strong privacy (significant noise)
  1-5    -> Moderate privacy (typical)
  5-10   -> Weak privacy (minimal noise)
  > 10   -> Negligible privacy

K-Anonymity

import pandas as pd

def check_k_anonymity(data: pd.DataFrame, quasi_ids: list[str], k: int = 5) -> dict:
    group_sizes = data.groupby(quasi_ids).size()
    min_group = group_sizes.min()
    violating = (group_sizes < k).sum()
    return {
        "satisfies": min_group >= k,
        "k_actual": int(min_group),
        "violating_groups": int(violating),
    }

Validation Framework

from scipy import stats
import pandas as pd
import numpy as np

class SyntheticDataValidator:
    def __init__(self, real: pd.DataFrame, synthetic: pd.DataFrame):
        self.real = real
        self.synthetic = synthetic

    def validate_distributions(self) -> dict:
        results = {}
        for col in self.real.columns:
            if col not in self.synthetic.columns:
                continue
            if self.real[col].dtype in ["float64", "int64"]:
                stat, p = stats.ks_2samp(self.real[col].dropna(), self.synthetic[col].dropna())
                results[col] = {"test": "ks", "stat": round(stat, 4), "p": round(p, 4), "ok": p > 0.05}
        return results

    def validate_correlations(self) -> dict:
        cols = [c for c  .real.select_dtypes(include=[np.number]).columns  c  .synthetic.columns]
        real_corr = .real[cols].corr()
        synth_corr = .synthetic[cols].corr()
        diff = np.linalg.norm(real_corr.values - synth_corr.values, )
        norm_diff = diff / (np.linalg.norm(real_corr.values, ) * )
         {: (diff, ), : (norm_diff, ),
                :   norm_diff <     norm_diff <   }

     () -> :
        
         sklearn.ensemble  RandomForestClassifier
         sklearn.metrics  f1_score
        feats = [c  c  .real.select_dtypes(include=[np.number]).columns  c != target]
        model = RandomForestClassifier(n_estimators=, random_state=)
        model.fit(.synthetic[feats].fillna(), .synthetic[target])
        pred = model.predict(.real[feats].fillna())
        tstr = f1_score(.real[target], pred, average=)
         {: (tstr, )}

Cost and Performance

Method	Speed	Setup Cost	Quality	Privacy
Faker	100K+ rows/sec	Low	Low	High
Gaussian Copula	10K rows/sec	Low	Good	Medium
CTGAN	100 rows/sec	Medium (GPU)	Very Good	Medium
LLM generation	10-50 rows/min	Low (API cost)	Excellent	High

Checklist

• Define purpose: training augmentation, privacy, testing, or sharing
• Profile the real dataset (distributions, correlations, constraints)
• Select generation method matching quality and compute budget
• Implement pipeline with reproducible seeds
• Validate statistical fidelity (distribution and correlation tests)
• Run utility tests (train-on-synthetic, test-on-real)
• Verify privacy guarantees (k-anonymity, differential privacy)
• Check for data leakage (no real records copied verbatim)
• Document methodology, parameters, and validation results
• Establish monitoring if synthetic data is regenerated periodically

Output Format

# Synthetic Data Generator Analysis

## Context Assessment
[Situation summary and constraints]

## Recommended Approach
[Primary recommendation with rationale]

## Implementation Steps
1. [Step with specific details]
2. [Step with specific details]
3. [Step with specific details]

## Trade-offs and Considerations
- [Key trade-off 1]
- [Key trade-off 2]

## Next Steps
- [Immediate action item]
- [Follow-up action item]

Example

Input: "Help me implement synthetic data generator for a medium-scale production application"

Output: A structured analysis covering current state assessment, recommended synthetic data generator approach with specific patterns, implementation roadmap with milestones, and risk mitigation strategies tailored to the application scale and constraints.

Edge Cases

• Legacy system integration: When synthetic data generator must coexist with legacy approaches, provide a gradual migration path rather than a complete rewrite
• Scale mismatch: When the solution complexity exceeds the project scale, recommend a simpler approach and note when to revisit
• Team skill gaps: When the team lacks experience with the recommended approach, include learning resources and simpler alternatives
• Conflicting requirements: When constraints conflict (e.g., performance vs. maintainability), explicitly state the trade-off and recommend based on stated priorities