arize-toolkit-cli

name

arize-toolkit-cli

description

Manage Arize platform resources using the arize_toolkit CLI. Use when users want to list, create, update, or delete Arize resources (models, monitors, prompts, evaluators, custom metrics, dashboards, spaces, organizations, users, or data imports) via the command line, configure CLI profiles, or perform platform administration tasks. Triggers on "list models", "create monitor", "arize_toolkit", "CLI", "arize cli", or any request to manage Arize platform resources from the terminal. Always use the "arize_toolkit" command, never "ax".

Arize Toolkit CLI

Manage Arize platform resources from the command line using arize_toolkit.

---

Workflow

1. Check Setup → 2. Resolve Config → 3. Run Command → 4. Summarize Output

---

Step 1: Check Setup

Verify the CLI is installed:

arize_toolkit --version

If not installed:

pip install arize_toolkit[cli]

---

Step 2: Resolve Configuration

The CLI resolves configuration in this priority order:

1. CLI flags: --api-key, --org, --space, --app-url
2. Environment variables: ARIZE_DEVELOPER_KEY, ARIZE_ORGANIZATION, ARIZE_SPACE, ARIZE_APP_URL
3. Config profile: ~/.arize_toolkit/config.toml

Check if a profile exists

arize_toolkit config list

If no profile, initialize one

Ask the user for their API key, organization name, and space name, then create the profile:

arize_toolkit config init --api-key "API_KEY" --org "ORG_NAME" --space "SPACE_NAME"

First-time Arize setup

If the user doesn't have an Arize account yet, walk them through these steps:

1. Create an account — sign up at https://app.arize.com/auth/join
2. Get an API key — after logging in, navigate to Settings → API Keys in the Arize UI (see API key docs)
3. Find organization and space names — from the Arize landing page, use the upper-left dropdown to select an organization, then pick a space. These are the --org and --space values for the CLI.
4. Initialize the CLI profile with the values from steps 2-3:

   arize_toolkit config init --api-key "API_KEY" --org "ORG_NAME" --space "SPACE_NAME"
   

5. For on-prem deployments — add --app-url pointing to the base URL of the Arize instance:

   arize_toolkit config init --api-key "API_KEY" --org "ORG_NAME" --space "SPACE_NAME" --app-url "https://arize.your-company.com"
   

---

Step 3: Run Commands

Global Options (apply to all commands)

Flag	Description
--profile NAME	Use a specific config profile
--json	Output as JSON instead of Rich tables
--api-key KEY	Override API key
--org NAME	Override organization
--space NAME	Override space
--app-url URL	Override app URL

Command Groups

Group	Alias	Description
config		Manage CLI configuration profiles
spaces		Manage Arize spaces
orgs		Manage organizations
users		Manage users and space membership
models	projects	Manage models/projects
monitors		Create and manage monitors
prompts		Manage LLM prompt templates
custom-metrics		Manage custom metrics
evaluators		Manage LLM and code evaluators
dashboards		Manage dashboards
imports		Manage data import jobs (files/tables)
traces		Query and inspect traces and spans

See references/COMMANDS.md for full command reference.

---

Step 4: Summarize Output

• When --json is used, pipe through jq for filtering
• Present results clearly to the user
• For list commands, summarize count and key details
• For create/update/delete, confirm the action completed

---

Common Workflows

List all models in a space

arize_toolkit models list

Get model performance metrics

arize_toolkit models performance accuracy production --model-name "my-model" --granularity day

Create a performance monitor with email alerts

arize_toolkit monitors create-performance "My Monitor" \
  --model "my-model" \
  --environment production \
  --performance-metric accuracy \
  --threshold 0.9 \
  --operator lessThan \
  --email [e-mail]

Copy a monitor to another model

arize_toolkit monitors copy "My Monitor" --model "source-model" --new-model "target-model"

Create a prompt template

arize_toolkit prompts create "my-prompt" \
  --messages '[{"role": "system", "content": "You are helpful."}, {"role": "user", "content": "{{query}}"}]' \
  --input-variable-format mustache

Create a prompt from a file

arize_toolkit prompts create "my-prompt" --messages @prompt_messages.json

List recent traces for a model

arize_toolkit traces list --model-name "my-agent"

Get all spans for a trace

arize_toolkit traces get TRACE_ID --model-name "my-agent"

Export trace spans to CSV with all attributes

arize_toolkit traces get TRACE_ID --model-name "my-agent" --csv spans.csv

Export traces to CSV with filtered attributes

arize_toolkit traces list --model-name "my-agent" --csv traces.csv
arize_toolkit traces get TRACE_ID --model-id "ID" --columns "input.value,output.value" --csv spans.csv

Export data as JSON for scripting

arize_toolkit --json models list | jq '.[].name'

Use a different profile

arize_toolkit --profile staging monitors list --model-name "my-model"

---

Tips

• Use --json flag for scripting and piping to jq
• Use @filepath syntax for complex JSON arguments (schema, messages, templates, code)
• models and projects are interchangeable aliases
• All destructive commands (delete) prompt for confirmation
• spaces switch and spaces create automatically persist the new space/org to your config profile, so subsequent commands use the new space
• Use --help on any command for full usage details: arize_toolkit monitors create-performance --help

---

Troubleshooting

Issue	Solution
command not found	Install with pip install arize_toolkit[cli]
Authentication error	Check API key: arize_toolkit config show
Wrong space/org	Use --space / --org flags or switch profile
Complex JSON args	Use @file.json syntax instead of inline JSON

---

References

• references/COMMANDS.md - Full command reference with all options