shihongDev/evalyn

evalyn-calibrate

Use when LLM judges need calibration, evaluation metrics seem misaligned with expectations, or annotation and judge tuning is needed

evalyn-eval

Use when building evaluation datasets, selecting metrics, or running evaluations on an LLM agent project with evalyn

evalyn

Use to evaluate an LLM agent with evalyn. Orchestrates the full pipeline: install, instrument, trace, build dataset, suggest metrics, run eval, analyze, calibrate.

evalyn-setup

Use when setting up evalyn evaluation for an LLM agent project, instrumenting agent code, or adding the evalyn decorator

evalyn-analyze

Use when analyzing evalyn evaluation results, investigating failures, comparing runs, or understanding agent performance