run-test

name

run-test

description

Execute Freeplay test runs to evaluate prompt templates and agents against datasets. Always confirm with the user before running — test runs make API calls to LLM providers and may incur costs. Use when the user wants to run a test, execute tests against a dataset, kick off evaluations, or trigger a test run. Do NOT use for analyzing existing test results (use test-run-analysis) or managing datasets (use dataset-management).

Run Freeplay Test

IMPORTANT: Confirmation Required

Always ask for user confirmation before executing any test run. Test runs make API calls to Freeplay and LLM providers, which may incur costs. Never execute a test without explicit user consent.

---

This skill helps users execute Freeplay test runs via the SDK/API to evaluate their AI features against datasets.

Note: This skill is for SDK/API-based test runs only, not UI-based testing from the Freeplay dashboard.

When to use this skill

• "Run a test for my prompt"
• "Execute the test run for customer support"
• "Test my agent against the golden dataset"
• "Run evaluation tests"
• "Kick off a test run"
• "I want to test my prompt changes"

Workflow

Step 1: Identify the Test Run Code Path

Search the codebase for existing test run implementation. Look for:

# Search patterns (run these to find test run implementations)
grep -r "test_runs.create" --include="*.py" .
grep -r "fp_client.test_runs" --include="*.py" .
grep -r "freeplay.*test" --include="*.py" .
grep -r "TestRun" --include="*.py" .