| name | add-llm-dataset |
| description | Use when adding or documenting a local supervised fine-tuning dataset for Figaro LLM PEFT runs. Explains how to place JSONL files under datasets/llm, supported prompt_completion and messages formats, frontend sft.dataset_path discovery, and verification. |
Add Local LLM Dataset
Use this workflow when the user wants Figaro to recognize a local SFT dataset in the frontend.
Target Directory
Place LLM supervised fine-tuning datasets under:
datasets/llm/
The backend discovers .jsonl files recursively. The frontend option value will be the project-relative file path:
./datasets/llm/<dataset>.jsonl
Nested paths are supported:
./datasets/llm/alpaca/train.jsonl
Use a separate validation file when enabling LLM evaluation:
./datasets/llm/validation.jsonl
Supported Formats
Use JSON Lines: one JSON object per line.
For sft.format = prompt_completion, each row should contain:
{"prompt":"Question","completion":"Answer"}
For sft.format = messages, each row should contain a chat-style message list:
{"messages":[{"role":"user","content":"Question"},{"role":"assistant","content":"Answer"}]}
Keep every line valid JSON. Do not wrap the file in a top-level array.
Discovery Rules
Figaro scans datasets/llm/ when the backend returns config schema.
- Files ending in
.jsonl are exposed as selectable sft.dataset_path options.
- The same JSONL files are exposed as selectable
evaluation.dataset_path options.
- Non-JSONL files are ignored.
- Hidden paths are ignored.
- Dataset files are ignored by Git; only
datasets/llm/README.md should be tracked.
Restart or refresh the backend schema after adding a dataset if the option does not appear.
Frontend Selection
In Simulation or Agent config:
- Select
task.type = llm_peft_sft.
- Open the SFT Dataset fields.
- Choose the JSONL file from
Dataset path, or type it manually.
- Set
Dataset format to match the JSONL rows.
- To compute validation loss/perplexity, enable
evaluation.enable and select a validation JSONL file.
Example values:
sft.dataset_path = ./datasets/llm/train.jsonl
sft.format = prompt_completion
evaluation.dataset_path = ./datasets/llm/validation.jsonl
Verification
Check that the schema exposes the dataset:
curl http://localhost:8000/api/v1/jobs/config/schema
Look for the path in:
sft.dataset_path.options
Then run a small LLM PEFT simulation config and confirm the runtime metrics include llm_dataset.num_records and llm_dataset.client_record_counts.
