discover-academic-skills

name

discover-academic-skills

description

Discover research/academic agent-skills from skills.sh (and user-supplied candidates), filter them through strict hard gates plus a strict academic-relevance gate, score survivors with a rubric, and hand back a ranked report with reasons. Use when the user wants to find, discover, or scout new academic/research skills to add to the hub, evaluate a skill they saw on social media, or refresh the discovery ledger.

Discover Academic Skills

A discovery scout, not an auto-collector. It surfaces candidate research skills with scores and reasons; ingestion is always human-gated (per ADR 0001). The scout never installs or vendors anything.

• Scope: discovery + strict filtering + scoring of academic/research skills from the skills.sh registry and from candidates the user pastes (e.g. from social media). Anything the user accepts is later vendored by hand into collected-skills/ via the normal intake.
• Inputs: none required (runs a default academic query set); optional --candidate owner/repo@skill, --queries, --min-installs, --include-seen.
• Outputs: a ranked Markdown report in scratch/skill-discovery/ (gitignored), the top picks surfaced inline, and new rows appended to the committed discovery-ledger.md.
• Limitations: npx skills find is fuzzy and skills.sh coverage is partial; the strict relevance gate favours precision over recall (it will miss some real-but-unlabelled skills — by design, to cut review burden). The skills.sh JSON API (/api/v1) needs a Vercel OIDC token and is not used; the unauthenticated CLI is the discovery channel.

Prerequisites

• Node / npx — for npx skills find (unauthenticated; no token).
• gh authenticated — for the license hard gate and repo metadata. Without it the license gate is skipped and every candidate is flagged license-unverified for you to check by hand (nothing is silently dropped).

Pipeline (deterministic gates → relevance gate → rubric)

1. Gather + deterministic hard gates (script)

Run the helper — it queries skills.sh, dedupes, suppresses ledger entries, and applies the deterministic gates:

python research-skills-hub/open-paper-skills/discover-academic-skills/scripts/discover.py

Deterministic gates it applies (a failure drops the candidate into the report's "dropped" appendix with a reason — never silently):

• Install floor — < 50 installs dropped (tune with --min-installs; manual --candidate entries bypass this).
• License present — no license → dropped (default all-rights-reserved can't be redistributed). Permissive (MIT/Apache/BSD/…) passes; copyleft passes flagged copyleft-license; unrecognised passes flagged for your check.
• Ledger suppression — anything already decided in discovery-ledger.md is dropped (use --include-seen to override). This is what makes repeated runs quieter over time.
• Hub-overlap hint — candidates whose name overlaps an existing hub skill are flagged possible-overlap:<name> (not dropped — you may want the better one).

The script writes scratch/skill-discovery/<date>-candidates.raw.json with a kept list (for scoring) and a dropped list (for the appendix).

2. Academic-relevance gate — STRICT (agent)

For every kept candidate, read its SKILL.md (open the skills.sh page at the candidate url, or run npx skills use owner/repo@skill to print the skill prompt) and decide, strictly:

Would a researcher use this in a research workflow? It must clearly match the IN list. Borderline or merely dual-use → reject (move to a "dropped — off scope" section with a one-line reason).

IN (pass): literature discovery/reading (arXiv, OpenAlex, Semantic Scholar, PubMed, bioRxiv); citation/reference management (Zotero, BibTeX); academic writing (LaTeX papers, venue templates, submission formatting); research data / experiments / reproducibility / baseline tables; paper-oriented figures and diagrams; systematic review / meta-analysis / bibliometrics; discipline-specific science tooling and ML/AI research workflows.

OUT (reject): general software engineering, web/frontend, DevOps, marketing, generic productivity. Dual-use tools (generic latex, diagram, pdf, search) are rejected unless the SKILL.md gives an explicit research framing.

Precision over recall is intended: a strict gate keeps your review list almost entirely genuine research skills. The vocabulary behind this gate is in ../../CONTEXT.md.

3. Rubric scoring (agent)

Score each survivor on four dimensions, 0–5 each, weighted into a 0–100 composite:

Dimension	Weight	What to judge
Research value & impact	35%	how central/useful to a research workflow (not "is it relevant" — "how much does it matter")
Quality & maturity	25%	SKILL.md clarity/completeness, install count, stars, recency of pushed_at
Safety & trust	25%	read the bundled scripts: does it run arbitrary code, reach the network, need credentials? author track record
Fit & uniqueness	15%	fills a gap vs the hub / complements the OS; overlap with an existing skill lowers this

Tiers by composite: ≥80 strongly recommend · 65–79 consider · 50–64 borderline · <50 low (appendix only). License is not scored (it is a gate/flag).

4. Report + ledger (agent)

Write scratch/skill-discovery/<date>-candidates.md:

• Grouped by tier (strongly recommend → consider → borderline). Each entry: composite score, the four sub-scores, a 2–3 sentence reason, and any flags (copyleft-license, license-<spdx>, possible-overlap:…, license-unverified).
• Appendices: dropped at hard gates and dropped — off scope, each with the one-line reason. Do not silently omit anything.

Then surface every ≥80 candidate inline to the user, and append every surfaced candidate to discovery-ledger.md with disposition surfaced. When the user later accepts/rejects one, update its row to accepted (note where it was vendored) or rejected (note why). Ledger row shape:

| owner/repo@skill | <date> | surfaced | <score> | <one-line reason> |

Accepting a candidate

Accepting is manual and follows the existing cherry-pick intake: read the whole SKILL.md and every bundled script, confirm the license, copy it into collected-skills/<skill>/, register it with the skill-organizer skill (source URL + license), then run ./verify.sh. If a whole repo is worth tracking wholesale, promote it to its own vendored-mirror collection like mattpocock-skills instead (see ADR 0001).

Manual candidate (social-media finds)

Feed a skill you saw elsewhere through the exact same gates and rubric:

python .../discover.py --candidate owner/repo@skill

It skips the install floor but runs the license gate, relevance gate, and rubric like any other candidate.