// Publish local dataset artifacts to a Hugging Face dataset repo. Use when uploading a JSONL dataset, pushing a filtered dataset variant, syncing a matching .metadata.json sidecar, or renaming a dataset file in the target repo. This skill is about USING the checked-in dataset publish script via CLI — never ad hoc Python.
Complete reference for the fine-tuning pipeline (SFT, KTO, GRPO), cloud HF Jobs workflows, autonomous experiment search, checkpoint evaluation, and LoRA surgery. Covers training CLI flags, YAML configuration, model presets, dataset requirements, LoRA settings, training monitoring, hyperparameter search, and post-training optimization. Use when training models, configuring training runs, choosing hyperparameters, running cloud experiments, inspecting HF jobs, or troubleshooting training issues. This skill is about USING the training system via CLI and YAML — never modifying source code.
Complete reference for the SynthChat synthetic dataset generation system. Covers CLI commands (generate, improve, validate), scenario YAML authoring, rubric YAML authoring, settings configuration, evaluation, and full workflow. Use when generating datasets, writing rubrics/scenarios, configuring models/workers, improving dataset quality, or running evaluations. This skill is about USING the system via CLI and YAML — never modifying source code.
End-to-end case studies showing how to implement the full training pipeline for different skill types. Covers three complete worked examples — tool-calling training, essay-style training, and agentic search (RAG agent) training — demonstrating dataset design, synthetic generation, validation, fine-tuning, evaluation, and iteration. Use when onboarding to the project, understanding how all components fit together, explaining the pipeline to others, or planning a new training capability. This skill is about UNDERSTANDING the system holistically — reference the other skills for specific CLI commands.
Complete reference for model upload and deployment. Covers HuggingFace upload, save strategies (LoRA, merged 16-bit, merged 4-bit), GGUF conversion, model merging, model cards, and the full upload workflow. Use when uploading models, creating GGUF files, merging LoRA adapters, or deploying to HuggingFace. This skill is about USING the upload/deployment tools via CLI — never modifying source code.
Complete reference for the config-first model evaluation system. Covers the Evaluator CLI, assertion-driven YAML scenarios, response views, backend configuration, presets, scoring, LLM-as-judge, model comparison, and HuggingFace integration. Use when evaluating models, writing test prompts, comparing training runs, or interpreting eval results. This skill is about USING the evaluation system via CLI and YAML.
Create structured research notes from experiment runs and analysis artifacts. Use when creating a note at run launch, updating it as training/evaluation/loss stages finish, summarizing a finished run, comparing experiment outcomes, extracting hypotheses from eval/loss artifacts, or proposing next-run actions grounded in `.tracking/experiments/<id>/analysis/` outputs. This skill is about turning repo-native experiment evidence into stable, machine-readable markdown.
| name | dataset-publishing |
| description | Publish local dataset artifacts to a Hugging Face dataset repo. Use when uploading a JSONL dataset, pushing a filtered dataset variant, syncing a matching .metadata.json sidecar, or renaming a dataset file in the target repo. This skill is about USING the checked-in dataset publish script via CLI — never ad hoc Python. |
| allowed-tools | Read, Bash, Write, Grep, Glob |
Publish a local dataset JSONL to a Hugging Face dataset repo with the skill-owned script:
python3 scripts/publish_dataset_to_hf.py
The script accepts:
dataset_pathrepo_idIt also auto-uploads a matching metadata sidecar if present:
dataset.jsonldataset.metadata.json| Task | Command |
|---|---|
| Dry-run a dataset upload | python3 scripts/publish_dataset_to_hf.py DATASET.jsonl namespace/repo --dry-run |
| Upload dataset + sidecar | python3 scripts/publish_dataset_to_hf.py DATASET.jsonl namespace/repo |
| Upload under a new repo filename | python3 scripts/publish_dataset_to_hf.py DATASET.jsonl namespace/repo --path-in-repo new_name.jsonl |
| Upload with explicit metadata file | python3 scripts/publish_dataset_to_hf.py DATASET.jsonl namespace/repo --metadata-path DATASET.metadata.json |
| Skip metadata sidecar | python3 scripts/publish_dataset_to_hf.py DATASET.jsonl namespace/repo --no-metadata |
HF_TOKEN from the environment or repo .envpath_in_repo = basename(dataset_path)*.metadata.json sidecars for dotted filenames correctly--dry-run first.Upload a filtered SFT dataset:
python3 scripts/publish_dataset_to_hf.py \
Datasets/synthchat/my_filtered_dataset.jsonl \
professorsynapse/claudesidian-synthetic-dataset \
--dry-run
python3 scripts/publish_dataset_to_hf.py \
Datasets/synthchat/my_filtered_dataset.jsonl \
professorsynapse/claudesidian-synthetic-dataset
Rename on upload:
python3 scripts/publish_dataset_to_hf.py \
Datasets/synthchat/my_filtered_dataset.jsonl \
professorsynapse/claudesidian-synthetic-dataset \
--path-in-repo nonthinking_tools_sft_filtered_03.22.26.jsonl
Upload without a sidecar:
python3 scripts/publish_dataset_to_hf.py \
Datasets/synthchat/my_filtered_dataset.jsonl \
professorsynapse/claudesidian-synthetic-dataset \
--no-metadata
--dry-run before the real upload when testing a new dataset variant..metadata.json sidecar.