magpie

name	magpie
description	Survey a local GitHub mirror, score every repo, and maintain a ranked worklist of things worth working on. Use when asked to survey repos, refresh the worklist, pick something to work on, or record an accept/reject decision.

magpie

Magpie flies over a mirror of GitHub repos, picks out the shiny things — work that is small, real, and valuable — and keeps a ranked worklist a human can accept or reject. Every decision is remembered, so the picking gets better over time.

Magpie points, it does not dig: it proposes work, it never executes destructive actions, and code changes it recommends are made as branches and pull requests by whoever accepts the item — never direct pushes.

Where things live

Two layers, deliberately separate:

1. The ledger (this repo) — versioned, pushed, and served over HTTP from the mirror (e.g. http://localhost:5445/melvincarvalho/magpie/):

worklist.json          # current ranked shortlist (the product)
surveys/YYYY-MM-DD.json # full scored output of each survey run
decisions.json         # append-only human verdicts — the taste dataset
scores.json            # cached per-repo signals, diffed between runs

2. The scratchpad (per repo) — <repo>/.git/magpie.json inside each mirrored repo. Files under .git/ are never tracked, committed, or pushed, so any repo can be annotated — including org repos the user doesn't own — with zero risk of leaking upstream. The web server refuses dotfile paths, so this layer is filesystem-only working state: last survey time, computed signals, raw findings. It is a cache; it may be lost when a clone is deleted. Anything durable belongs in the ledger.

The mirror itself defaults to ~/remote/github.com/<owner>/<repo>, maintained by gh-sync (see the productivity repo). Repo directory mtimes equal the repo's last push time — that is the freshness signal, no API calls needed. The full inventory is one fetch: the JSON embedded in <script id="data"> at the mirror root's index.html.

Schemas

Worklist entry (worklist.json is { "generated": iso8601, "items": [...] }):

{
  "id": "jspod-seed-clobber",
  "repo": "JavaScriptSolidServer/jspod",
  "pitch": "seedPodFiles overwrites user index.html on every start — small fix, real users affected",
  "kind": "bug",
  "effort": "small",
  "score": 87,
  "signals": { "pushedDays": 3, "stars": 41, "openIssues": 7 },
  "seeAlso": [
    "http://localhost:5445/JavaScriptSolidServer/jspod/",
    "https://github.com/JavaScriptSolidServer/jspod"
  ]
}

kind: bug | feature | docs | cleanup | security | archive-candidate. effort: small | medium | large. Prefer small.

Decision entry (decisions.json, a JSON array; append-only — push new entries, never rewrite or delete existing ones):

{"ts":"2026-07-08T18:00:00Z","id":"jspod-seed-clobber","repo":"JavaScriptSolidServer/jspod","action":"accepted","note":"good catch, fixing today"}
{"ts":"2026-07-08T18:05:00Z","id":"fund-agent-shell-injection","priority":"pin","note":"security, do first"}

A decision carries an action, a priority, or both — two independent axes:

• action (lifecycle): accepted | rejected | done. Drives filtering (done/rejected leave the worklist) and the status shown on an item.
• priority (ordering): pin | later. pin floats a finding to the top regardless of score; later sinks it to the bottom; absent = normal score order. Orthogonal to action — pinning does not change lifecycle status. Latest entry per finding wins on each axis, so you can re-pin or clear later.

Never rewrite or delete entries; corrections are new entries.

Two ways to append a decision: edit decisions.json directly (then run refresh-worklist.mjs), or POST it to magpie-serve.mjs — a tiny localhost-only write endpoint (127.0.0.1:5446, POST /decision) that appends and regenerates the worklist in one step. It is the ONLY writer of decisions.json and the only reason the read-only mirror can stay read-only. The dashboard's pin/unpin buttons drive it; priority:"normal" clears a pin.

After recording decisions, run node refresh-worklist.mjs — it rewrites worklist.json as the latest survey's ranked findings minus anything done/rejected. Accepted items stay (in flight); later items sink. This is what keeps the worklist and the dashboard in step with reality.

Per-repo scratchpad (.git/magpie.json):

{"surveyed":"2026-07-08T18:00:00Z","signals":{...},"findings":[...],"seeAlso":["https://github.com/<owner>/<repo>"]}

Survey procedure

1. Read taste first. Load every entry of decisions.json. Rejected patterns (e.g. "README polish on dead repos") must not resurface; accepted patterns tell you what scores high.
2. Inventory. Read the mirror index JSON for the repo list and freshness. Do not hit the GitHub API for what the mirror already knows.
3. Score. Value ≈ (people affected × fix impact) / effort. Real signals: recent pushes, stars/forks, open issues, failing CI, TODO and FIXME density, broken links in READMEs, missing licenses, security smells. Staleness alone is NOT a finding — an untouched-for-8-years repo nobody uses scores near zero. An empty local clone is not a sync failure — the upstream is usually just genuinely empty (0 commits); verify the remote actually has commits before claiming gh-sync failed. (Empty clones are excluded from the dashboard inventory, so you should not normally see them.)
4. Dig only where promising. Read code in the top candidates, at filesystem speed, straight from the mirror. Write findings to each repo's .git/magpie.json.
5. Write the ledger. Full output to surveys/<date>.json; the top ~10 as worklist.json — each item's pitch one sentence, concrete, and checkable. Commit both with message survey YYYY-MM-DD.
6. Budget. A survey is read-only and bounded: stop at the configured number of deep reads (default 25 repos) rather than sweeping everything nightly; rotate cohorts between runs so the whole estate gets covered over time. Log what was skipped.

Working the worklist (a pin is a work request)

A pin is not just ordering — it is the human saying "work on this next." When asked to work the magpie worklist (or handed a specific pinned item):

1. Read worklist.json. Take items with priority: "pin" first (they sort to the top), then by score. Skip anything already in flight (status accepted) unless told to continue it.
2. Do the work under the hard rules below — on a branch, delivered as a PR, never a push to a default branch; destructive actions stay proposals.
3. Record progress in decisions.json (via magpie-serve or by editing + refresh-worklist.mjs): accepted with a PR link when you start (it shows as in flight), done when merged. The item then leaves the worklist and appears in the Done tab.

That is the loop end to end: you pin in the UI, an agent (a Claude Code session or a scheduled run) picks up pinned items and turns them into PRs, and the Done tab fills in behind it. Pinning is how a human points; this is how the agent follows.

Hard rules

• Propose, never execute: deletions, archivals, and force-pushes are worklist items for a human, not actions.
• Code changes happen on branches, delivered as PRs. Never push to a default branch.
• Never commit or push anything under .git/. Never commit magpie metadata into a surveyed repo's working tree.
• The mirror is read-only ground truth. Never modify surveyed repos during a survey.
• Respect privacy: repos marked private in the mirror must never be referenced in a ledger that is pushed to a public remote.
• decisions.json is append-only, forever (push entries; never rewrite/delete).