| name | clawpathy-autoresearch |
| description | Eval-driven skill tuning. Given a task and an LLM-judge rubric, iteratively rewrites a SKILL.md until a downstream executor agent performs well against the judge. Low-code: all evaluation is LLM-as-judge, not deterministic Python. |
| version | 1.0.0 |
| author | Jay Moore |
| license | MIT |
| tags | ["meta","autoresearch","skill-tuning","llm-judge","eval-driven"] |
| inputs | [{"name":"paper_query_or_task","type":"string","description":"Paper title/URL/PMID/DOI, or a freeform task description","required":true}] |
| outputs | [{"name":"workspace/","type":"directory","description":"Tuned skill/SKILL.md plus history.jsonl, snapshots, executor_runs"}] |
| metadata | {"openclaw":{"requires":{"bins":"[Truncated]"},"always":false,"emoji":"🔁","homepage":"https://github.com/ClawBio/ClawBio","os":["darwin","linux"],"trigger_keywords":["auto research","autoresearch","tune a skill","skill tuning","improve a skill","eval-driven","clawpathy","replicate paper","reproduce paper"]}} |
clawpathy-autoresearch
[!note] Vault audit 2026-07-24 — USE-10
Use this for eval-driven tuning that iteratively rewrites an existing SKILL.md against an LLM-judge rubric; to scaffold a new skill from scratch use skill-builder, to auto-draft from observed workflows use autoskill, to package a plugin bundle use plugin-creator. Distinguishing axis: authoring mode (eval-tuning vs manual scaffold vs observation vs plugin packaging).
Eval-driven skill development. The system iteratively rewrites a SKILL.md
so a downstream executor agent performs better at a task class, as judged
by an LLM against a paper/task-specific rubric.
Core idea
propose (sonnet) → execute (sonnet, shell) → judge (opus, rubric)
↑ │
└──────── feedback: verdict + recommended edits ────────┘
- Proposer rewrites SKILL.md based on the last judge verdict.
- Executor runs the new SKILL.md end-to-end inside a workspace.
- Judge scores methodology (primary) and outputs (secondary) against
a per-task rubric. Lower is better; 0 = perfect.
- Keep the new SKILL.md only if it strictly beats the best score; else
revert. Stop on target_score or on
early_stop_n consecutive regressions.
You are the orchestrator
You (the agent reading this) don't run the loop yourself. You dispatch
subagents to build the workspace, then hand off to the Python loop.
Phase 1 — Scout
Dispatch a subagent with prompts/scout.md to research the paper/task.
Report key findings to the user in a few lines.
Phase 2 — Scope (you + user)
Have a conversation. Ask ONE question at a time, multiple-choice where
helpful. Agree on:
- what to reproduce / what success looks like
- which data sources are in-bounds
- what methodology expectations belong in the rubric
- iteration budget and target_score (if any)
Present a summary and get approval.
Phase 3 — Build
Dispatch a builder subagent with prompts/builder.md and the agreed
scope. It writes:
task.jsonrubric.md — the authoritative scoring rubric for the LLM judgereference/ (optional; judge-only)skill/SKILL.md — seed
