| name | setup-skill-performance |
| description | Generate eval scenarios from a Tessl plugin (a packaged skill bundle), run baseline + with-context evals, and present results. Use when setting up an evaluation pipeline, running benchmarks, generating test scenarios, measuring skill performance or accuracy, scoring how well a skill helps agents solve tasks, or evaluating skill effectiveness before publishing. |
Eval Setup
You handle plugin eval setup — scenario generation from a plugin, running evals, and presenting results.
The user triggers this skill when they have a plugin but no eval scenarios yet, or when they want to generate new scenarios.
Companion skill: After setup is complete, suggest the user run the optimize-skill-performance skill to analyze results, diagnose failures, fix plugin content, and re-verify improvements.
Time expectations: Set these upfront so the user isn't surprised:
- Scenario generation: ~1–2 minutes per scenario
- Eval run: ~10–15 minutes per scenario per agent (each scenario runs twice: baseline + with-context)
- For a first run, aim for 3–5 scenarios with 1 agent to keep total time under 2 hours
Run labels
Every tessl eval run invocation MUST include --label <run-label> so the run is identifiable in tessl eval list. The label is a short, human-readable description of what the run is about — not a structured ID.
Compose <run-label> from whatever helps you recognise the run later when scanning the list. Typical ingredients:
- Eval type — e.g.
activation, baseline, initial evals, verification
- What was being tested or changed — e.g.
description rewrite, ,
