apastra

PromptOps skills for versioning, evaluating, tracing, and shipping AI prompts as disciplined software assets. Agent-as-harness — your IDE agent runs evals, compares baselines, surfaces trace evidence, and gates quality.

Upgrade from local-first evaluation to automated GitHub Actions CI. Installs workflows for PR gating, release promotion, and auto-merge.

Quick setup guide for apastra PromptOps. Create your first prompt spec, dataset, evaluator, and suite in 5 minutes.

Run prompt evaluations using your IDE agent as the harness. Load suites, execute test cases, score results, and compare against baselines.

Generate new prompt specs, datasets, evaluators, and suites from templates. Creates correctly-formatted files that pass schema validation.

Inspect agent traces, hook events, tool-call transcripts, and run artifacts; turn them into PromptOps evidence, eval cases, and artifact references.

Interactive PromptOps eval design — what behavior to pin down, cases, scoring, thresholds, and outcome/step/trace evidence. Deep link to Writing evals once per session; pair with scaffold + apastra-eval.

Establish and manage evaluation baselines for regression detection. A baseline is a known-good scorecard that future runs are compared against.

Validate all promptops files against JSON schemas. Catch formatting errors before running evaluations.