azure-ai-fine-tuning

name	azure-ai-fine-tuning
description	Use when the user wants to fine-tune a model on Azure AI Foundry — including dataset preparation, training, evaluation, and deployment.
argument-hint	["describe the model you want to fine-tune and the task it should perform"]

Goal

Help the user fine-tune a model on Azure AI Foundry. This covers the full lifecycle:

1. Dataset creation — generate or prepare training data
2. Dataset evaluation — assess data quality before training
3. Base model evaluation — benchmark the un-tuned model
4. Training type selection — choose SFT, DPO, or RFT
5. Dataset format conversion — convert data to the right format
6. Training job submission — launch and monitor training runs
7. Training curve analysis — detect overfitting, pick checkpoints
8. Iterative experimentation — plan successive runs from results
9. Model deployment — deploy fine-tuned models with correct format/SKU
10. Model evaluation — score outputs with an LLM judge

$ARGUMENTS

Workflow

Read the workflow file that matches the user's current stage:

• "Just do it for me" / autonomous fine-tuning (experimental) → workflows/auto-finetune.md
• First time / just want to get started → workflows/quickstart.md
• Starting from scratch → workflows/full-pipeline.md
• Need a dataset → workflows/dataset-creation.md
• Have a deployed agent — distill its traces into a smaller model → workflows/traces-to-dataset.md
• No traces yet — generate Q&A from a doc or tool-use from an OpenAPI spec → workflows/synthetic-datagen.md
• Training and iterating → workflows/iterative-training.md
• Results are bad → workflows/diagnose-poor-results.md
• Reviewing results & planning next run → workflows/experiment-review.md

If the user wants full control, start with workflows/full-pipeline.md. If they want automation, try workflows/auto-finetune.md (experimental).

References

Read the relevant reference file before performing any step:

File	When to read
references/training-types.md	Choosing between SFT, DPO, and RFT
references/hyperparameters.md	Setting learning rate, batch size, epochs
references/dataset-formats.md	Preparing or converting training data
references/data-generation-api.md	Foundry Data Generation API (preview) — sources, recipes, scenarios for traces→dataset and synthetic datagen
references/deployment-formats.md	Deploying a fine-tuned model
references/evaluation-methodology.md	Designing an eval rubric
references/training-curve-analysis.md	Reading training logs and curves
references/foundry-cli.md	Using the azd ai finetuning CLI for submit/deploy
references/vision-fine-tuning.md	Fine-tuning with image data (gpt-4o, gpt-4.1)
references/cost-management.md	Training costs, hosting tiers, budget planning
references/distillation.md	Teacher-student model distillation workflow
references/agentic-rft.md	Tool calling + endpoint graders for agentic RFT
references/grader-design.md	Designing effective RFT graders (type selection, partial credit, threshold calibration)
references/reward-hacking-prevention.md	Preventing reward hacking in RFT (grader alignment, monitoring, iteration)
references/platform-bugs.md	Known platform bugs and workarounds
references/large-file-uploads.md	Uploading large training files (>100MB) via chunked Uploads API

Scripts

Reusable Python scripts in scripts/. Each is self-contained with inline documentation.

Script	Purpose
auto_finetune.py	Autonomous orchestrator (experimental, SFT only) — runs the full loop: analyze → generate → prepare → baseline → train → evaluate → review → iterate. Good for exploration; use individual scripts for production workflows or RFT.
auto_rft.py	Autonomous RFT orchestrator (experimental) — full loop for reinforcement fine-tuning: validate → prepare → calibrate → baseline → submit → monitor → evaluate → iterate. Use this for RFT; use auto_finetune.py for SFT.
submit_training.py	Submit SFT, DPO, or RFT jobs (SDK + REST fallback)
monitor_training.py	Poll a running job until completion, streaming events in real time
generate_dataset.py	Generate fine-tuning or eval data via the Foundry Data Generation API (preview) — traces → SFT/eval, doc → Q&A, OpenAPI spec → tool-use. SDK + REST modes. Also includes --tools-from/--tools-to-openapi-out converter for OpenAI tool-spec → OpenAPI 3.0.
calibrate_grader.py	Run base model through your RFT grader to find optimal pass_threshold
generate_distillation_data.py	Generate training data from a teacher model for distillation (legacy custom-script approach)
check_training.py	Pull training curves, detect overfitting, list checkpoints
deploy_model.py	Deploy fine-tuned models via ARM REST API
cleanup.py	List and delete old deployments, files, and pending jobs to reclaim quota
evaluate_model.py	Run held-out eval with 2-dimension LLM judge
convert_dataset.py	Convert between SFT, DPO, and RFT JSONL formats
score_dataset.py	LLM-judge quality scoring on training data
common.py	Shared auth helper — get_clients() tries /v1/, Foundry SDK, AzureOpenAI in order

Always validate data before submitting jobs — run validate_sft.py / validate_dpo.py / validate_rft.py first, then data_stats.py for the overview.

Sample data: examples/sample-data/ contains sft_sample.jsonl, dpo_sample.jsonl, and rft_sample.jsonl — use these as format references.

CLI alternative: For quick single-job workflows, the azd ai finetuning CLI can replace submit_training.py and deploy_model.py. See references/foundry-cli.md.

Quick Reference

Task	Command
Validate SFT data	python scripts/validate/validate_sft.py data.jsonl
Triage "User data has failed data safety check" errors	python scripts/content_safety_check.py --jsonl train.jsonl --endpoint https://<resource>.cognitiveservices.azure.com --api-key $env:AZURE_CONTENT_SAFETY_KEY --drop-out clean.jsonl
Transform Foundry traces output into Azure FT-ready JSONL (for traces-to-SFT distillation)	python scripts/transform_traces_jsonl.py --jsonl raw_traces_dg.jsonl --system-prompt-file system.md --tools-file tools.json --out sft.jsonl
Full autopilot for traces distillation (auto-runs the transform)	python scripts/auto_finetune.py auto --description "..." --task-name <name> --model <student> --teacher <strong> --datagen-agent-name <agent> --datagen-agent-version <v> --datagen-hours 720 --traces-system-prompt-file system.md --traces-tools-file tools.json
Generate dataset from agent traces	python scripts/generate_dataset.py --source traces --agent-name <name> --agent-version <v> --recipe traces --scenario sft --max-samples 200 --train-split 0.8 --hours 24 --download
Generate Q&A from a doc	python scripts/generate_dataset.py --source prompt-file --prompt-file policy.md --recipe qna --scenario sft --teacher gpt-4.1-mini --max-samples 100 --train-split 0.9 --download
Generate Q&A from a large doc (chunked workaround for SimpleQnA per-source saturation)	python scripts/chunk_and_generate.py --source-text big-doc.txt --chunks 10 --teacher gpt-4.1 --recipe qna --scenario sft --max-samples-per-chunk 100 --concurrency 2 --out merged.jsonl --cleanup-uploads
Filter generated data on quality (LLM judge)	python scripts/quality_filter.py --jsonl generated.jsonl --judge gpt-4.1-mini --threshold 4 --drop-out filtered.jsonl
Deep-diagnose why an autopilot iteration didn't ship	python scripts/diagnose_iteration.py --work-dir ./auto_ft_run --judge gpt-4.1
Convert OpenAI tools to OpenAPI 3.0 (for tool-use)	python scripts/generate_dataset.py --tools-from openai_tools.json --tools-to-openapi-out openapi.json
Generate tool-use SFT (upload openapi.json first)	python scripts/generate_dataset.py --source file --file-id <openapi-file-id> --recipe tool-use --scenario sft --teacher gpt-4.1-mini --max-samples 50 --train-split 0.8 --download

Common Errors

Error	Cause	Fix
"API version not supported"	Older openai SDK on /v1/ endpoint	Upgrade to openai>=1.0
"does not support fine-tuning with Standard TrainingType"	OSS model needs globalStandard	Use --use-rest flag or set trainingType: "globalStandard"
Job stuck in post-training eval	Under-provisioned tool endpoint (RFT)	Scale to S2+, enable Always On
"DeploymentNotReady" / 500 on deploy	Too many deployments or ARM race condition	Clean old eval deployments, retry after 5 min
Content safety block at deployment	PII-dense training data	Remove problematic document types
"BadRequestForDependentService"	Deployment still warming up	Wait 5+ minutes after deployment creation
Queue stuck ("jobs ahead")	Standard tier capacity exhausted	Cancel and resubmit on developerTier or globalStandard

Rules

• Always evaluate the base model before fine-tuning — you need a baseline to measure improvement.
• Verify your deployments exist before starting — model names must match actual deployment names. A model like gpt-4.1 must be deployed in your resource before you can use it as a teacher or student.
• For data generation, ask the user which model deployment to use. Data Designer and distillation scripts both require an LLM endpoint — don't assume a model is available. The user may have gpt-4.1-mini, gpt-5.4, or any other model deployed.
• Start with 2 epochs for SFT, 1–2 epochs for DPO (explicitly set — Azure defaults to 3), and task-dependent for RFT. Learning rate multiplier 1.0 unless you have reason to change.
• Never skip dataset quality review — garbage in, garbage out.
• Use the same held-out validation set across ALL experiments for comparable results.
• When overfitting is detected, deploy an earlier checkpoint before retraining with fewer epochs.
• Clean up deployments after evaluation to avoid quota exhaustion.
• RFT grader field names MUST match training data: Before submitting an RFT job, verify that the grader's item.get('field_name') calls match the actual field names in your JSONL data. A mismatch (e.g., grader reads reference_answer but data uses answer) silently returns 0.0 for every sample — the grader never raises an error, it just gets empty strings. Always print and diff the grader source vs. the first line of your training JSONL.
• RFT tasks must be hard enough for the base model to fail: If the base model already scores 100% on your grader from rollout 1, RFT has no signal to learn from — the reward gradient is zero. Before submitting RFT, run the base model on your validation set and grade it with your grader. If pass rate is already > 90%, either (a) make the grader stricter, (b) use harder tasks, or (c) skip RFT — the model is already good enough.

Resource & Subscription Management

Resources may span multiple subscriptions. Always verify both the subscription AND resource before submitting jobs or querying status. The az CLI only searches the active subscription.

Map your resources before starting:

Resource	Subscription	RG	Endpoint	Use
<your-primary-resource>	<subscription-name>	<resource-group>	https://<your-primary-resource>.cognitiveservices.azure.com/	Primary FT resource
<your-secondary-resource>	<subscription-name>	<resource-group>	https://<your-secondary-resource>.cognitiveservices.azure.com/	Secondary / overflow

Tip: Run az cognitiveservices account list --query "[].{name:name, rg:resourceGroup, endpoint:properties.endpoint}" -o table to discover all your resources across subscriptions.

Before querying or submitting jobs:

1. Ask the user which resource/project the job is on, or check the Foundry UI URL
2. Run az account set --subscription "<sub name>" to switch to the correct subscription
3. Verify with az account show --query name -o tsv

Common mistake: Forgetting to switch subscriptions before querying. If a job returns 404, try the other subscription before assuming it's lost.

Platform Gotchas

• Verify you're submitting to the correct resource AND subscription: Azure AI Foundry projects connect to a specific AIServices/OpenAI resource, which lives on a specific subscription. Jobs submitted to a different resource won't appear in the portal or telemetry. Symptoms: jobs show via API but not in the Foundry UI; "phantom" failures that the team can't reproduce; 404s when querying a valid job ID. Always (1) switch to the correct subscription first, (2) use the project endpoint (https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1/) or verify the OAI endpoint matches the resource connected to your Foundry project. A common mistake is submitting to <resource-A> instead of <resource-B> — all "platform 500" failures were actually jobs on the wrong resource.

• Transient HTTP 500 failures: Azure AI Foundry FT jobs can fail with "A system error was encountered, please try again later" (HTTP 500). Single retries often succeed. Retry once or twice, then wait and check. If failures persist, verify you're hitting the correct resource endpoint before filing a support ticket.

• All OSS FT jobs require trainingType: globalStandard: The Python SDK fails with "does not support fine-tuning with Standard TrainingType" for all OSS models (Ministral-3B, Qwen-32B, Llama-3.3-70B-Instruct, gpt-oss-20b). Use the REST API with "trainingType": "globalStandard" in the JSON payload. See scripts/submit_training.py for the fallback.

• OSS model FT uses Global deployment tier: Ministral-3B, Qwen-32B, Llama-3.3-70B-Instruct, and gpt-oss-20b support fine-tuning via Global (not Standard regional). Any regional resource can use Global. The model catalog API incorrectly reports capabilities.fine_tune = false for these models — ignore the flag. Developer tier is only available for OpenAI models, not OSS. Model ID format: Ministral-3B (not gpt-*); FT output format: Ministral-3B.ft-{jobid}-suffix. Note: gpt-oss-20b is the model name but the versioned ID on the platform is gpt-oss-20b-11.

• Deployment format matters: A wrong model.format gives an unhelpful HTTP 500. See references/deployment-formats.md for the exact mapping. For OSS models, use --model-format "OpenAI-OSS" with --sku-name "GlobalStandard". Deploy via CLI: az cognitiveservices account deployment create --name <resource> --resource-group <rg> --deployment-name <name> --model-name <model> --model-version "1" --model-format "OpenAI-OSS" --sku-capacity 100 --sku-name "GlobalStandard".

What to Expect by Training Type

These patterns are based on extensive end-to-end testing across SFT, DPO, and RFT.

SFT Distillation (Most Reliable)

• Teacher→student distillation (e.g., mini→nano) typically achieves 58–100% teacher gap closure with 200–300 examples and 2 epochs
• Fine-tuned small models can sometimes surpass the teacher on tasks with clear input→output patterns (summarization, entity extraction, code generation)
• Pattern tasks distill best: Code generation, PII redaction, and structured extraction show the highest gap closure
• Tasks requiring open-ended reasoning or alignment show weaker distillation results

DPO (Use With Caution)

• DPO fails when the base model is already strong. If the base model scores >4.5/5 on your task, DPO will likely degrade quality rather than improve it
• DPO is prone to degeneration (repetitive/garbage output), especially on sensitive topics — even at epoch 1
• When DPO helps: The base model has a clear quality gap on your task, and you have well-differentiated preference pairs

RFT (For Verifiable Tasks)

• RFT works best for tasks with verifiable answers: math, code with test suites, structured output with exact-match grading
• Typical improvement: +10 percentage points on exact-match accuracy for math/reasoning tasks
• The grader matters more than hyperparameters — invest in grader quality before tuning LR/epochs
• See references/reward-hacking-prevention.md for the #1 RFT failure mode

Common Pitfalls

Pitfall	What happens	Fix
Skipping baseline evaluation	You can't measure improvement	Always evaluate the base model first
Too few examples (<100)	Model learns format but not domain knowledge	Use 200–500 examples minimum
DPO on strong base model	Quality degrades	Use SFT instead, or skip fine-tuning
Misaligned RFT grader	Reward hacking — model games the grader	Use same grading logic for training and eval
Small OSS models on large label sets	Model invents synonym labels (capacity limit)	Use ≥20B parameter models for 50+ classes

Testing this skill

The skill ships with two test suites under tests/:

cd Skills

# Fast checks (default — skips anything that hits the live service)
python -m pytest tests/ -v

# End-to-end suite against a real Foundry project
$env:FOUNDRY_PROJECT_ENDPOINT = "https://<resource>.services.ai.azure.com/api/projects/<project>"
$env:FOUNDRY_TEACHER_MODEL    = "gpt-4.1"             # any deployed chat model
$env:FOUNDRY_AGENT_NAME       = "<your-agent>"        # for traces/agent tests
$env:FOUNDRY_AGENT_VERSION    = "1"
$env:E2E_JOB_TIMEOUT          = "900"                  # 15 min per non-tool-use job
$env:E2E_TOOL_USE_TIMEOUT     = "2700"                 # 45 min cap on tool-use jobs
python -m pytest tests/test_data_generation_e2e.py -m live -v

-m live selects the 9 live tests; the default -m "not live" runs only the 8 CLI/argparse tests plus the 48 existing skill-consistency tests. Authentication uses DefaultAzureCredential — run az login first.