jmagly/aiwg-training

dataset-docs

Generate Datasheet, Model Card, and Data Statement from a dataset manifest

dataset-reproduce

Deterministically rebuild a dataset from its manifest and verify fixity equivalence

dataset-version

Create a versioned training dataset with manifest, fixity, provenance, and archive snapshot

flow-dataset-build

End-to-end training dataset pipeline — acquire sources through publication

decontamination-check

Detect training-eval overlap against benchmark sets before dataset publication

example-synthesizer

Generate SFT training examples from raw sources using Self-Instruct / Evol-Instruct / SQuAD / STaR patterns

format-adapter-alpaca

Convert canonical training examples to Alpaca format for training frameworks

format-adapter-chatml

Convert canonical training examples to ChatML format for training frameworks

format-adapter-jsonl

Convert canonical training examples to JSONL format for training frameworks

format-adapter-parquet

Convert canonical training examples to Parquet format for training frameworks

format-adapter-sharegpt

Convert canonical training examples to ShareGPT format for training frameworks

preference-generator

Generate preference pairs (chosen/rejected) for DPO/KTO/ORPO/SimPO training

synthetic-data-generator

Large-scale synthetic training data generation with Model Collapse recursion guard

acquire-training-source

Acquire a training data source with license validation and delegate ingest to the semantic memory kernel

example-quality-assess

GRADE quality assessment adapted for individual training examples