arize-dataset

name	arize-dataset
description	INVOKE THIS SKILL when creating, managing, or querying Arize datasets and examples. Covers dataset CRUD, appending examples, exporting data, and file-based dataset creation using the ax CLI.

Arize Dataset Skill

Concepts

• Dataset = a versioned collection of examples used for evaluation and experimentation
• Dataset Version = a snapshot of a dataset at a point in time; updates can be in-place or create a new version
• Example = a single record in a dataset with arbitrary user-defined fields (e.g., question, answer, context)
• Space = an organizational container; datasets belong to a space

System-managed fields on examples (id, created_at, updated_at) are auto-generated by the server -- never include them in create or append payloads.

Prerequisites

Three things are needed: ax CLI, an API key (env var or profile), and a space ID. A project name is also needed but usually comes from the user's message.

Install ax

If ax is not installed, not on PATH, or below version 0.3.0, see ax-setup.md.

Verify environment

Run a quick check for credentials:

macOS/Linux (bash):

ax --version && echo "--- env ---" && if [ -n "$ARIZE_API_KEY" ]; then echo "ARIZE_API_KEY: (set)"; else echo "ARIZE_API_KEY: (not set)"; fi && echo "ARIZE_SPACE_ID: ${ARIZE_SPACE_ID:-(not set)}" && echo "--- profiles ---" && ax profiles show 2>&1

Windows (PowerShell):

ax --version; Write-Host "--- env ---"; Write-Host "ARIZE_API_KEY: $(if ($env:ARIZE_API_KEY) { '(set)' } else { '(not set)' })"; Write-Host "ARIZE_SPACE_ID: $env:ARIZE_SPACE_ID"; Write-Host "--- profiles ---"; ax profiles show 2>&1

Read the output and proceed immediately if either the env var or the profile has an API key. Only ask the user if both are missing. Resolve failures:

• No API key in env and no profile → AskQuestion: "Arize API key (https://app.arize.com/admin > API Keys)"
• Space ID unknown → run ax spaces list -o json to list all accessible spaces and pick the right one, or AskQuestion if the user prefers to provide it directly
• Project unclear → ask, or run ax projects list -o json --limit 100 and present as selectable options

Space ID and Project

Both are needed for most commands. Resolve each:

1. User provides it in the conversation -- use directly via --space-id / --project flags.
2. Env var is set (ARIZE_SPACE_ID, ARIZE_DEFAULT_PROJECT) -- use silently.
3. If missing, AskQuestion once. Tell the user:
   • Run ax spaces list -o json to discover your space ID, or find it in the Arize URL: /spaces/{SPACE_ID}/...
   • Project is the project name as shown in the Arize UI.
   • For convenience, recommend setting env vars so they don't get asked again: export ARIZE_SPACE_ID="U3BhY2U6..." and export ARIZE_DEFAULT_PROJECT="my-project"

Prefer asking the user over searching or iterating through projects and API keys. If you get a 401 Unauthorized, tell the user their API key may not have access to that space and ask them to verify.

List Datasets: ax datasets list

Browse datasets in a space. Output goes to stdout.

ax datasets list
ax datasets list --space-id SPACE_ID --limit 20
ax datasets list --cursor CURSOR_TOKEN
ax datasets list -o json

Flags

Flag	Type	Default	Description
--space-id	string	from profile	Filter by space
--limit, -l	int	15	Max results (1-100)
--cursor	string	none	Pagination cursor from previous response
-o, --output	string	table	Output format: table, json, csv, parquet, or file path
-p, --profile	string	default	Configuration profile

Get Dataset: ax datasets get

Quick metadata lookup -- returns dataset name, space, timestamps, and version list.

ax datasets get DATASET_ID
ax datasets get DATASET_ID -o json

Flags

Flag	Type	Default	Description
DATASET_ID	string	required	Positional argument
-o, --output	string	table	Output format
-p, --profile	string	default	Configuration profile

Response fields

Field	Type	Description
id	string	Dataset ID
name	string	Dataset name
space_id	string	Space this dataset belongs to
created_at	datetime	When the dataset was created
updated_at	datetime	Last modification time
versions	array	List of dataset versions (id, name, dataset_id, created_at, updated_at)

Export Dataset: ax datasets export

Download all examples to a file. Use --all for datasets larger than 500 examples (unlimited bulk export).

ax datasets export DATASET_ID
# -> dataset_abc123_20260305_141500/examples.json

ax datasets export DATASET_ID --all
ax datasets export DATASET_ID --version-id VERSION_ID
ax datasets export DATASET_ID --output-dir ./data
ax datasets export DATASET_ID --stdout
ax datasets export DATASET_ID --stdout | jq '.[0]'

Flags

Flag	Type	Default	Description
DATASET_ID	string	required	Positional argument
--version-id	string	latest	Export a specific dataset version
--all	bool	false	Unlimited bulk export (use for datasets > 500 examples)
--output-dir	string	.	Output directory
--stdout	bool	false	Print JSON to stdout instead of file
-p, --profile	string	default	Configuration profile

Agent auto-escalation rule: If an export returns exactly 500 examples, the result is likely truncated — re-run with --all to get the full dataset.

Export completeness verification: After exporting, confirm the row count matches what the server reports:

# Get the server-reported count from dataset metadata
ax datasets get DATASET_ID -o json | jq '.versions[-1] | {version: .id, examples: .example_count}'

# Compare to what was exported
jq 'length' dataset_*/examples.json

# If counts differ, re-export with --all

Output is a JSON array of example objects. Each example has system fields (id, created_at, updated_at) plus all user-defined fields:

[
  {
    "id": "ex_001",
    "created_at": "2026-01-15T10:00:00Z",
    "updated_at": "2026-01-15T10:00:00Z",
    "question": "What is 2+2?",
    "answer": "4",
    "topic": "math"
  }
]

Create Dataset: ax datasets create

Create a new dataset from a data file.

ax datasets create --name "My Dataset" --space-id SPACE_ID --file data.csv
ax datasets create --name "My Dataset" --space-id SPACE_ID --file data.json
ax datasets create --name "My Dataset" --space-id SPACE_ID --file data.jsonl
ax datasets create --name "My Dataset" --space-id SPACE_ID --file data.parquet

Flags

Flag	Type	Required	Description
--name, -n	string	yes	Dataset name
--space-id	string	yes	Space to create the dataset in
--file, -f	path	yes	Data file: CSV, JSON, JSONL, or Parquet
-o, --output	string	no	Output format for the returned dataset metadata
-p, --profile	string	no	Configuration profile

Supported file formats

Format	Extension	Notes
CSV	.csv	Column headers become field names
JSON	.json	Array of objects
JSON Lines	.jsonl	One object per line (NOT a JSON array)
Parquet	.parquet	Column names become field names; preserves types

Format gotchas:

• CSV: Loses type information — dates become strings, null becomes empty string. Use JSON/Parquet to preserve types.
• JSONL: Each line is a separate JSON object. A JSON array ([{...}, {...}]) in a .jsonl file will fail — use .json extension instead.
• Parquet: Preserves column types. Requires pandas/pyarrow to read locally: pd.read_parquet("examples.parquet").

Append Examples: ax datasets append

Add examples to an existing dataset. Two input modes -- use whichever fits.

Inline JSON (agent-friendly)

Generate the payload directly -- no temp files needed:

ax datasets append DATASET_ID --json '[{"question": "What is 2+2?", "answer": "4"}]'

ax datasets append DATASET_ID --json '[
  {"question": "What is gravity?", "answer": "A fundamental force..."},
  {"question": "What is light?", "answer": "Electromagnetic radiation..."}
]'

From a file

ax datasets append DATASET_ID --file new_examples.csv
ax datasets append DATASET_ID --file additions.json

To a specific version

ax datasets append DATASET_ID --json '[{"q": "..."}]' --version-id VERSION_ID

Flags

Flag	Type	Required	Description
DATASET_ID	string	yes	Positional argument
--json	string	mutex	JSON array of example objects
--file, -f	path	mutex	Data file (CSV, JSON, JSONL, Parquet)
--version-id	string	no	Append to a specific version (default: latest)
-o, --output	string	no	Output format for the returned dataset metadata
-p, --profile	string	no	Configuration profile

Exactly one of --json or --file is required.

Validation

• Each example must be a JSON object with at least one user-defined field
• Maximum 100,000 examples per request

Schema validation before append: If the dataset already has examples, inspect its schema before appending to avoid silent field mismatches:

# Check existing field names in the dataset
ax datasets export DATASET_ID --stdout | jq '.[0] | keys'

# Verify your new data has matching field names
echo '[{"question": "..."}]' | jq '.[0] | keys'

# Both outputs should show the same user-defined fields

Fields are free-form: extra fields in new examples are added, and missing fields become null. However, typos in field names (e.g., queston vs question) create new columns silently -- verify spelling before appending.

Delete Dataset: ax datasets delete

ax datasets delete DATASET_ID
ax datasets delete DATASET_ID --force   # skip confirmation prompt

Flags

Flag	Type	Default	Description
DATASET_ID	string	required	Positional argument
--force, -f	bool	false	Skip confirmation prompt
-p, --profile	string	default	Configuration profile

Workflows

Find a dataset by name

Users often refer to datasets by name rather than ID. Resolve a name to an ID before running other commands:

# Find dataset ID by name
ax datasets list -o json | jq '.[] | select(.name == "eval-set-v1") | .id'

# If the list is paginated, fetch more
ax datasets list -o json --limit 100 | jq '.[] | select(.name | test("eval-set")) | {id, name}'

Create a dataset from file for evaluation

1. Prepare a CSV/JSON/Parquet file with your evaluation columns (e.g., input, expected_output)
2. ax datasets create --name "eval-set-v1" --space-id SPACE_ID --file eval_data.csv
3. Verify: ax datasets get DATASET_ID
4. Use the dataset ID to run experiments

Add examples to an existing dataset

# Find the dataset
ax datasets list

# Append inline or from a file (see Append Examples section for full syntax)
ax datasets append DATASET_ID --json '[{"question": "...", "answer": "..."}]'
ax datasets append DATASET_ID --file additional_examples.csv

Download dataset for offline analysis

1. ax datasets list -- find the dataset
2. ax datasets export DATASET_ID -- download to file
3. Parse the JSON: jq '.[] | .question' dataset_*/examples.json

Export a specific version

# List versions
ax datasets get DATASET_ID -o json | jq '.versions'

# Export that version
ax datasets export DATASET_ID --version-id VERSION_ID

Iterate on a dataset

1. Export current version: ax datasets export DATASET_ID
2. Modify the examples locally
3. Append new rows: ax datasets append DATASET_ID --file new_rows.csv
4. Or create a fresh version: ax datasets create --name "eval-set-v2" --space-id SPACE_ID --file updated_data.json

Pipe export to other tools

# Count examples
ax datasets export DATASET_ID --stdout | jq 'length'

# Extract a single field
ax datasets export DATASET_ID --stdout | jq '.[].question'

# Convert to CSV with jq
ax datasets export DATASET_ID --stdout | jq -r '.[] | [.question, .answer] | @csv'

Dataset Example Schema

Examples are free-form JSON objects. There is no fixed schema -- columns are whatever fields you provide. System-managed fields are added by the server:

Field	Type	Managed by	Notes
id	string	server	Auto-generated UUID. Required on update, forbidden on create/append
created_at	datetime	server	Immutable creation timestamp
updated_at	datetime	server	Auto-updated on modification
(any user field)	any JSON type	user	String, number, boolean, null, nested object, array

Related Skills

• arize-trace: Export production spans to understand what data to put in datasets → use arize-trace
• arize-experiment: Run evaluations against this dataset → next step is arize-experiment
• arize-prompt-optimization: Use dataset + experiment results to improve prompts → use arize-prompt-optimization

Troubleshooting

Problem	Solution
ax: command not found	See ax-setup.md
401 Unauthorized	API key may not have access to this space. Verify the key and space ID are correct. Keys are scoped per space -- get the right one from https://app.arize.com/admin > API Keys.
No profile found	Run ax profiles show --expand to check; set ARIZE_API_KEY env var or write ~/.arize/config.toml
Dataset not found	Verify dataset ID with ax datasets list
File format error	Supported: CSV, JSON, JSONL, Parquet
platform-managed column	Remove id, created_at, updated_at from create/append payloads
reserved column	Remove time, count, or any source_record_* field
Provide either --json or --file	Append requires exactly one input source
Examples array is empty	Ensure your JSON array or file contains at least one example
not a JSON object	Each element in the --json array must be a {...} object, not a string or number

Save Credentials for Future Use

At the end of the session, if the user manually provided any of the following during this conversation (via AskQuestion response, pasted text, or inline values) and those values were NOT already loaded from a saved profile or environment variable, offer to save them for future use.

Credential	Where it gets saved
API key	ax profile at ~/.arize/config.toml
Space ID	macOS/Linux: shell config (~/.zshrc or ~/.bashrc) as export ARIZE_SPACE_ID="...". Windows: user environment variable via [System.Environment]::SetEnvironmentVariable('ARIZE_SPACE_ID', '...', 'User')

Skip this entirely if:

• The API key was already loaded from an existing profile or ARIZE_API_KEY env var
• The space ID was already set via ARIZE_SPACE_ID env var
• The user only used base64 project IDs (no space ID was needed)

How to offer: Use AskQuestion: "Would you like to save your Arize credentials so you don't have to enter them next time?" with options "Yes, save them" / "No thanks".

If the user says yes:

1. API key — Check if ~/.arize/config.toml exists. If it does, read it and update the [auth] section. If not, create it with this minimal content:

   [profile]
   name = "default"
   
   [auth]
   api_key = "THE_API_KEY"
   
   [output]
   format = "table"
   

   Verify with: ax profiles show

2. Space ID — Persist the space ID as an environment variable:

   macOS/Linux — Detect the user's shell config file (~/.zshrc for zsh, ~/.bashrc for bash). Append:

   export ARIZE_SPACE_ID="THE_SPACE_ID"
   

   Tell the user to run source ~/.zshrc (or restart their terminal) for it to take effect.

   Windows (PowerShell) — Set a persistent user environment variable:

   [System.Environment]::SetEnvironmentVariable('ARIZE_SPACE_ID', 'THE_SPACE_ID', 'User')
   

   Tell the user to restart their terminal for it to take effect.