skillhone

name	skillhone
description	SkillHone — toolkit for evaluating, optimizing, and managing agent skills. Use when asked to "evaluate a skill", "run probe", "optimize/iterate a skill", "create a new skill experiment", "seed a skill repo", or "run skill benchmarks"; also use when the user mentions a Forgejo-hosted skill repo and wants to measure or improve its quality. Wraps standalone scripts: status, eval, optim, new, seed, serve, synth.
compatibility	Requires Python 3.10+, git, and access to a Forgejo instance (or local FS).

SkillHone

SkillHone is a measurement harness plus an optimization toolkit for agent skills. The important abstraction is not a fixed workflow; it is the evidence trail created when a skill is run against private eval tasks.

The harness has these layers:

• Skill repo: public behavior to improve (SKILL.md, scripts, references).
• Eval repo: private measurement contract (datasets, verifier, synthesis contract, task-local validators).
• Solver workdirs: isolated per-item sandboxes containing artifacts and trajectory.jsonl.
• Observation records: redacted probe results, trajectory diagnosis, compiler/validator diagnosis, issues, PRs, and wiki pages.

When improving a skill, identify which layer explains the failure before changing code. A score drop may point to skill instructions, but it can also be a harness, verifier, compiler, artifact-path, or infrastructure problem.

Standalone scripts live under scripts/. Pick one based on the task in front of you.

Before starting a diagnosis, development, PR review, merge, or optimization cycle on a Forgejo-backed repo, run so you know the current issue/PR state and do not duplicate work or merge the wrong PR.