| name | luna-ingest |
| description | Use when ingesting a github OR bitbucket.org repo / issue / PR URL into the luna Obsidian vault — repo URLs fetch metadata + README, follow 1-hop references, classify each (integrate / take-parts / inspire / skip / api_failure) and write a structured tech-ingest note under 30-Resources/Tech/; issue URLs (github.com or bitbucket.org) fetch the issue and write an issue note; bitbucket PR URLs (HIMMEL-329) write a PR note. Triggers on /luna-ingest <url> at user prompt OR programmatic Skill-tool dispatch from inside another runbook (e.g. obsidian-triage:harvest-clips repo-URL dispatch branch). Host-routed: github.com via gh, bitbucket.org via the himmel bitbucket CLI. Rejects twitter / article inputs. |
luna-ingest — chain-following triage (LUNA-5 Wedge B, MVP; skill conversion LUNA-9)
You are running LUNA-5 Wedge B. Take a github OR bitbucket.org URL, follow its README references one hop (github sources only — see below), classify each, and produce a structured tech-ref note in the luna vault.
Scope: github + bitbucket.org URLs. Twitter (x.com/...) and article URLs raise an explicit "not yet" error and point at LUNA-5 v2.
- github.com: repo URLs run Phases 1–4; issue URLs (
/issues/<n>) take the issue branch (HIMMEL-239); PR (/pull/<n>) and discussion URLs remain out of scope.
- bitbucket.org (HIMMEL-329): repo, PR (
/pull-requests/<n>), and issue (/issues/<n>) URLs take the Bitbucket branch below. They dispatch to the himmel bitbucket CLI (node scripts/bitbucket/dist/index.js …) instead of gh.
Forge routing (HIMMEL-329): the URL host selects the path — parallel to how the dev-loop routes on the git origin. github.com (or bare <owner>/<repo>) → the github Phases below, unchanged. bitbucket.org → the Bitbucket branch. Everything in Phases 1–4 and the issue branch is the github path; the Bitbucket branch reuses the shared Phase 1.5 safety pre-filter and Phase 3/4 heuristics but swaps the fetch transport.
Invocation surfaces (LUNA-9):
- User prompt:
/luna-ingest <github-url> [--vault <path>] [--dest <category>] [--limit <N>] [--deep] [--research] [--dry-run] — handled by the thin wrapper at .claude/commands/luna-ingest.md which delegates here.
- Programmatic Skill-tool dispatch (HIMMEL-128 compliant — no headless claude):
Skill { skill: "obsidian-triage:luna-ingest", args: "<github-url> [<flags...>]" } from inside another runbook (e.g. /harvest-clips github-URL branch).
Both surfaces feed the same argument shape into the runbook below. Wherever this document references $ARGUMENTS, treat it as the literal arg string supplied via either invocation path.
Inputs
$ARGUMENTS is <url> [--vault <path>] [--dest <category>] [--limit <N>] [--dry-run].
Parse:
<url> — required. Accepted shapes:
https://github.com/<owner>/<repo> (+ /, /tree/<branch>, /blob/<branch>/<path>)
https://github.com/<owner>/<repo>/issues/<n> — routes to the github issue branch (HIMMEL-239)
- bare
<owner>/<repo> — treated as github
https://bitbucket.org/<ws>/<repo> (+ /, /src/<branch>/...) — routes to the Bitbucket branch (HIMMEL-329)
https://bitbucket.org/<ws>/<repo>/pull-requests/<n> — Bitbucket PR branch
https://bitbucket.org/<ws>/<repo>/issues/<n> — Bitbucket issue branch
--vault <path> — luna vault root. Default: $HOME/Documents/luna. Refuse with rc=2 if the dir does not exist OR is not an Obsidian vault (no .obsidian/ dir).
--dest <category> — vault destination category override. Default: 30-Resources/Tech. Validate against the vault's PARA folders (00-Inbox, 10-Projects/*, 20-Areas/*, 30-Resources/Tech, 30-Resources/Concepts, 30-Resources/Components). Anything else → rc=2.
--limit <N> — max 1-hop refs to follow. Default 250 (LUNA-92: capture all refs by default — the prior [1,25] clamp left high-fanout awesome-lists partial and silently dropped sources). No upper clamp; reject < 1 with rc=1. 250 is a generous safety ceiling for unattended harvest-clips dispatch (which passes no --limit), not a quality cap — the largest list we have hit is 188 refs. Pass --limit <N> to bound a noisy meta-list downward. The LUNA-6 rate-limit contract (api_failure verdict + rc=4 resume) is the backstop if a pathological list exceeds the gh-api hourly budget.
--dry-run — print what would be written + verdicts, touch no files. rc=0.
--allow-unsafe — opt-in flag to ingest a repo flagged by the LUNA-43 safety-keyword pre-filter (Phase 1.5). Without this flag, repos whose name/description/topics match red-flag terms exit rc=6 before fetching the README.
--repo <ws/repo> is NOT a luna-ingest flag — it is the bitbucket CLI's own override, set internally by the Bitbucket branch from the parsed URL. Operators pass the bitbucket.org URL, not --repo.
--deep — LUNA-57: after classification, run the component scan
(tools/component-scan.mjs) over the source repo (always) and over each
1-hop ref verdicted integrate or take-parts. Inventories reusable
components (skills/commands/agents/tools/plugin manifests) into a
## Reusable components section of the Tech note AND a cross-repo-deduped
30-Resources/Components/ library. Off by default (MVP keeps the light path).
--components-dir <rel> — vault-relative Components library root.
Default 30-Resources/Components. Only meaningful with --deep.
--max-components <N> — per-repo component cap (tail-skip beyond). Default 60.
--research — LUNA-64: after classification, web-research the SOURCE repo
(maturity/adoption, notable alternatives, known limitations, community
sentiment, recency) using Claude's own WebSearch/WebFetch tools and inject a
## Research enrichment section into the Tech note. Off by default.
Orthogonal to --deep — they enrich different axes (--deep = reusable
components inventoried from the code; --research = external context about
the repo) and may be combined (--deep --research runs both). This is the
lightweight inline path: NO external Python (research_deep.py), NO
Perplexity/Grok keys, NO vault-wide propagation — those belong to the full
/research-deep command, deferred to a later LUNA-64 follow-up. Source repo
only in MVP (1-hop refs are not researched). The harvest-clips github
dispatch MUST NOT pass --research (cost + scope safety).
Reject x.com/, twitter.com/, or any host that is neither github.com nor bitbucket.org with:
ERR luna-ingest: only github.com and bitbucket.org URLs supported. Twitter + article inputs ship in LUNA-5 v2.
Issue branch — github issue URLs (HIMMEL-239)
When <github-url> matches ^https?://(www\.)?github\.com/<owner>/<repo>/issues/(\d+)$ (query/fragment stripped first), run THIS branch instead of Phases 1–3.6 and the Phase 4 repo synthesis. The repo phases assume a README + 1-hop ref pool; an issue has neither, so this branch is deliberately flat: fetch → safety-filter → write. --deep, --research, and --limit are repo-only flags — if passed alongside an issue URL, ignore them with one stderr line: WARN luna-ingest: --deep/--research/--limit are repo-only; ignored for issue URLs.
I-1 — Fetch. Extract <owner>/<repo> + <n>. Validate <owner>/<repo> with the same Phase 1 regex (leading-char rule included). Then:
gh api "repos/${owner_repo}/issues/${n}" --jq '{title, state, body, user: .user.login, labels: [.labels[].name], comments, created_at, closed_at, html_url}'
- 404 → rc=5, nothing written (issue deleted or repo private/renamed).
- Other
gh api failure → rc=2 with stderr inlined.
- Also fetch repo metadata (
gh api "repos/${owner_repo}" — same Phase 1 jq) for the trust-tier inputs. If THIS call fails but the issue call succeeded, continue with trust_tier: community-thin, trust_tier_reason: repo metadata unavailable → conservative default.
Comments (skip when the issue's comments count is 0):
gh api "repos/${owner_repo}/issues/${n}/comments" --paginate --jq '.[] | {user: .user.login, created_at, body}'
Cap at the 30 most-reacted-or-earliest comments (take the first 30 returned; the API returns oldest-first). If more existed, record tail-skipped: <N-30> comments in § Audit log.
I-2 — Safety pre-filter. Run the Phase 1.5 algorithm unchanged over: issue title, joined labels[], and the first 8 KB of the issue body. Same rc=6 / --allow-unsafe semantics.
I-3 — Write. Destination: <vault>/<dest-category>/<owner>-<repo>-issue-<n>.md (Phase 4 slugify on <owner>-<repo>; same path-safety invariant; rc=3 if the file exists). Compute trust_tier from the REPO owner via the Phase 4 rules (the issue author does not affect the tier). Structure:
---
type: tech-ingest
source: <issue html_url>
source_type: github-issue
ingested_at: <UTC ISO-8601>
last_revalidated: <UTC ISO-8601>
ingest_command: /luna-ingest
issue_state: <open|closed>
issue_author: <user.login>
issue_labels:
- <label>
comment_count: <N>
tags: # HIMMEL-247: REQUIRED — Phase 4 "Tags derivation", issue variant
- github
- github-issue
- <tag>
trust_tier: <tier>
trust_tier_reason: <text>
safety_flag: <term-or-blank>
---
# <repo-name> issue #<n> — <issue title>
## Source
[<owner>/<repo>#<n>](<html_url>) — <state>, opened <created_at> by @<user.login><", closed <closed_at>" when closed>
## Summary
<3-5 bullet TL;DR: what the issue reports, root cause if identified, resolution state>
## Issue body
<the issue body as markdown, verbatim — trim only trailing template boilerplate>
## Key comments
<for each kept comment with substance (skip "+1"/"same here" noise): `> **@<user>** · <created_at>` followed by the comment body or its load-bearing excerpt. If none: `_(no substantive comments)_`>
## Audit log
- Comments fetched: <K> of <N> (tail-skipped: <T>)
- API calls: <2-or-3>
- Trust tier: <tier> (<reason>)
- Safety flag: <term-or-none>
Exit codes are the standard set (0 written, 2 env, 3 exists, 5 source fetch failed, 6 safety-blocked). The harvest-clips github dispatch needs no change: issue clips it dispatches here now land an issue note, and its rc=3 dedup contract holds (back-reference the existing <owner>-<repo>-issue-<n> note).
Bitbucket branch — bitbucket.org URLs (HIMMEL-329)
When the URL host is bitbucket.org, run THIS branch instead of the github
Phases 1–4 / issue branch. It dispatches to the himmel bitbucket CLI —
node <himmel-root>/scripts/bitbucket/dist/index.js <verb> --repo <ws>/<repo>
— the gh analogue for Bitbucket Cloud. <himmel-root> is the himmel primary
checkout; the CLI's dist/ is gitignored, so it must be built once
(cd scripts/bitbucket && npm i && npm run build). Auth comes from the repo-root
env file (BITBUCKET_EMAIL / BITBUCKET_API_TOKEN); a missing build or missing
creds surfaces as a CLI error → rc=2 (env unusable).
B-0 — Parse + route. Strip protocol/query/fragment, then:
bitbucket.org/<ws>/<repo>/pull-requests/<n> → B-PR sub-branch.
bitbucket.org/<ws>/<repo>/issues/<n> → B-issue sub-branch.
- otherwise (
/<ws>/<repo>, optionally /src/<branch>/…) → B-repo sub-branch.
Extract <ws>/<repo> by stripping any /src/…, /pull-requests/…, /issues/…
suffix and a trailing /. Validate <ws>/<repo> with the same Phase 1
regex (^[A-Za-z0-9_-][A-Za-z0-9_.-]*/[A-Za-z0-9_-][A-Za-z0-9_.-]*$, leading-char
rule rejecting ./..); else rc=1. The CLI receives it verbatim via --repo.
B-repo — repository ingest
B-repo-1 Fetch. bitbucket repo get --repo <ws>/<repo> → JSON {name, full_name, description, language, default_branch, url, updated_on, is_private, readme}. CLI rc≠0 (network / auth / repo 404) → rc=5, nothing written
(same contract as github Phase 1 source-fetch failure). readme: null (no
README.md on the default branch) → record README: missing.
B-repo-1.5 Safety pre-filter. Run the Phase 1.5 algorithm unchanged over
name, description, and the first 8 KB of readme. Bitbucket has no
topics[] — skip that field. Same rc=6 / --allow-unsafe semantics and the
same safety_flag propagation.
B-repo-2 Reference extraction + classification. Write readme to a temp file
and run the existing Phase 2 + Phase 3 unchanged — they extract github.com
refs and classify them via gh api. github refs in a Bitbucket README are the
common, supported case. bitbucket.org refs are NOT followed in MVP
(BB→BB ref-following is a v2 follow-up): extract them separately
(grep -oE 'https://bitbucket\.org/[^ )]+') and list them under § Open questions
as bitbucket.org refs not followed (HIMMEL-329 MVP — v2 follow-up). --limit
applies to the github refs as usual.
B-repo-3 Trust tier (bitbucket variant). Bitbucket Cloud exposes no
stargazers_count. Apply the Phase 4 "Trust-tier auto-classification" rules
with these deltas:
- rule 0 (safety short-circuit), rule 1 (
anthropic-official), rule 2
(known-author, owner = <ws> lowercased) — unchanged.
- rule 3 (
community-active) — cannot fire (no star signal) → skip it.
- rule 4 (
community-thin) — default for any non-allowlisted owner.
trust_tier_reason for the rule-4 case: bitbucket: no star signal, updated_on=<YYYY-MM-DD> → community-thin.
B-repo-4 Synthesis. Write the same Phase 4 note shape under
<vault>/<dest>/<ws>-<repo>.md (same slugify + path-safety invariant + rc=3 on
existing), with these frontmatter deltas:
source_type: bitbucket (not github).
- Omit
stars: and topics: — no Bitbucket equivalent; do NOT emit a fake
0 / empty list.
language: from repo get; license: unknown (repo get returns no SPDX).
tags: — first tag is bitbucket (parallel to github's first-tag github);
since there are no topics[], infer 2–4 domain tags from name + description +
README per the Tags-derivation rule 3 fallback (prefer the vault's existing
tag vocabulary). Minimum 2 tags.
- The
## Referenced repos (1-hop) section lists the github refs only;
the ## Open questions section carries the unfollowed-bitbucket-refs note.
--deep source-scan is skipped on a Bitbucket source (the LUNA-57
component-scan.mjs is gh-API-only and cannot scan a Bitbucket repo) — emit a
one-line stderr warn and continue; integrate/take-parts github refs are still
scanned. --research (LUNA-64) is forge-agnostic (WebSearch) and runs normally.
B-PR — pull-request ingest
bitbucket.org/<ws>/<repo>/pull-requests/<n>. Flat fetch → write (like the
github issue branch — a PR has no README + ref pool).
B-PR-1 Fetch. bitbucket pr get <n> --repo <ws>/<repo> → {id, title, state, description, author, source_branch, destination_branch, url, created_on, updated_on}. CLI rc≠0 → rc=5, nothing written. Also fetch bitbucket repo get --repo <ws>/<repo> for the trust-tier owner input; if THAT fails but the PR
fetch succeeded, continue with trust_tier: community-thin,
trust_tier_reason: repo metadata unavailable → conservative default.
B-PR-2 Safety pre-filter. Phase 1.5 algorithm over PR title +
description (first 8 KB). Same rc=6 / --allow-unsafe semantics.
B-PR-3 Write. <vault>/<dest>/<ws>-<repo>-pr-<n>.md (slugify <ws>-<repo>;
rc=3 if it exists). Compute trust_tier from the repo owner via B-repo-3.
Frontmatter mirrors the github issue note with source_type: bitbucket-pr,
pr_state, pr_author, source_branch, destination_branch; tags:
bitbucket, bitbucket-pr, then up to 3 inferred domain tags. Body:
# <repo> PR #<n> — <title>
## Source
[<ws>/<repo> PR #<n>](<url>) — <state>, opened <created_on> by @<author>; <source_branch> → <destination_branch>
## Summary
<3-5 bullet TL;DR of what the PR changes, derived from title + description>
## Description
<the PR description verbatim as markdown; `_(no description)_` if empty>
## Audit log
- API calls: <1-or-2>
- Trust tier: <tier> (<reason>)
- Safety flag: <term-or-none>
B-issue — issue ingest
bitbucket.org/<ws>/<repo>/issues/<n>.
B-issue-1 Fetch. bitbucket issue get <n> --repo <ws>/<repo> → {id, title, state, kind, content, reporter, url, created_on, updated_on}. The CLI exits 3
when the issue tracker is disabled OR the issue is gone (404) → map to rc=5,
nothing written (parallel to the github issue branch's 404 → rc=5). Other CLI
rc≠0 → rc=2. Also fetch repo get for the trust-tier owner (same degrade as
B-PR-1 on failure).
B-issue-2 Safety pre-filter. Phase 1.5 over issue title + content
(first 8 KB). Same rc=6 / --allow-unsafe semantics.
B-issue-3 Write. <vault>/<dest>/<ws>-<repo>-issue-<n>.md (rc=3 if exists).
Frontmatter mirrors the github issue note with source_type: bitbucket-issue,
issue_state, issue_kind, issue_reporter; tags: bitbucket,
bitbucket-issue, then up to 3 inferred domain tags. Body mirrors the github
issue note's # … issue #<n>, ## Source, ## Summary, ## Issue body,
## Audit log (drop ## Key comments — the MVP bitbucket issue read fetches no
comment thread; note that under § Audit log as Comments: not fetched (MVP)).
The harvest-clips dispatch contract is unchanged: a bitbucket.org clip dispatched
here now lands the matching note, and the rc=3 dedup invariant holds (back-ref
the existing <ws>-<repo>[-pr|-issue]-<n> note).
Phase 1 — Source fetch (github URL)
Normalize to <owner>/<repo>:
owner_repo=$(printf '%s' "$URL" | sed -E 's|^https?://github\.com/||; s|/(tree|blob)/.*$||; s|/$||')
Validate it matches ^[A-Za-z0-9_-][A-Za-z0-9_.-]*/[A-Za-z0-9_-][A-Za-z0-9_.-]*$ — the leading char rejects . or .. as owner or repo (which would otherwise survive the looser [A-Za-z0-9_.-]+ form and let gh api "repos/../foo" flow through). Else rc=1.
Fetch:
gh api "repos/${owner_repo}" --jq '{name, full_name, description, html_url, language, stargazers_count, topics, default_branch, pushed_at, license: .license.spdx_id}' > /tmp/luna-ingest-meta.$$.json
If gh api fails (rc≠0): exit 2 with the stderr inlined.
Read README via:
gh api "repos/${owner_repo}/readme" --jq '.content' \
| tr -d '\n' \
| base64 -d > /tmp/luna-ingest-readme.$$.md
If the README endpoint returns 404 (some repos have none): proceed with an empty README; record README: missing in the synthesis.
Phase 1.5 — Safety pre-filter (LUNA-43)
Before continuing to Phase 2, scan the metadata captured in Phase 1 (name, description, topics[]) plus the decoded README for red-flag keywords.
Matching algorithm (single canonical form — pin this exactly, no variants):
- Normalize the searched field first. For each of
name, description, joined topics[], and the first 8 KB of decoded README (interpret 8 KB as 8192 bytes of the UTF-8-encoded slice — truncate at a byte boundary, do not split a multibyte rune), lowercase the text and replace any run of one-or-more [-_\s]+ characters with a single hyphen (-). This canonicalises red_team, red team, red--team, and Red-Team all to red-team.
- Apply each rule below as a case-insensitive substring search against the normalized text. Whichever rule's substring (or AND-pair) appears first → set
safety_flag: <matched-term> (use the first-listed canonical form from the rule). Word-boundary regex (\b) is NOT used — substring after normalization is the only form.
- AND-pair rules require both substrings to appear within 80 normalized characters of each other in the same field (e.g.
red-team .{0,80} agent or agent .{0,80} red-team).
Single-term rules (substring matches alone trigger):
abliteration (also catches abliterated, abliterationist)
censorship-removal
uncensored-model
autonomous-hacking
auto-hack
exploit-kit
malware-as-a-service
maas-toolkit
AND-pair rules (BOTH substrings must appear in the same field within 80 chars):
red-team AND (agent OR framework) — avoids false-positives on legit security-research repos that mention red-team work
jailbreak AND (model OR llm OR bypass-safety) — avoids false-positives on iOS jailbreak tooling
Match rule: if ANY of the rules above (or operator-curated additions, see below) trigger against name OR description OR topics[] OR the first 8 KB of decoded README → set safety_flag: <matched-term> (use the first-listed canonical form of the matched rule, e.g. red-team+agent for the AND-pair) and one of:
- Without
--allow-unsafe: exit rc=6 with message ERR luna-ingest: safety pre-filter matched <term> in <field>. Pass --allow-unsafe to ingest with safety_flag annotation, or skip this repo. — no synthesis written.
- With
--allow-unsafe: continue to Phase 2 but propagate safety_flag into the synthesis frontmatter so downstream readers (and /synthesize-clips) know to treat the page as unknown-risk regardless of license / star count.
Operator customisation: the keyword list above is the LUNA-43 default. Operators can extend it by setting LUNA_INGEST_SAFETY_KEYWORDS=<comma-separated terms> in their shell — additions ONLY (no removal of defaults). Removed defaults require an explicit SKILL.md edit + PR review, by design.
Log line: [safety] matched=<term> field=<name|description|topics|readme> action=<refused|annotated> to stderr regardless of action.
Phase 2 — Reference extraction
Parse the README for github URLs that point at OTHER repos:
grep -oE 'https://github\.com/[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+(\.git)?' /tmp/luna-ingest-readme.$$.md \
| sed -E 's|^https://github\.com/||; s|\.git$||; s|/$||' \
| sort -u \
| grep -v "^${owner_repo}$" \
> /tmp/luna-ingest-refs.$$.txt || true
The (\.git)? suffix on the regex plus the s|\.git$|| substitution dedupes <owner>/<repo> with <owner>/<repo>.git (a common README link form).
Strip refs whose path has more than one / (sub-paths inside the same repo — keep the bare owner/repo only), AND reject any segment that is purely . or .. (would survive the looser ref regex and lead to gh api repos/../something calls that pollute the audit log):
awk -F/ 'NF==2 && $1 !~ /^\.+$/ && $2 !~ /^\.+$/' /tmp/luna-ingest-refs.$$.txt > /tmp/luna-ingest-refs.clean.$$.txt
Apply the --limit cap (the explicit --limit <N> if passed, else the default 250). If more refs exist than the cap, log "tail-skipped: N refs" in the synthesis under § Open questions. With the default 250, a tail-skip now signals a genuinely huge meta-list (re-run with an explicit higher --limit), not the old routine [1,25]-clamp truncation.
LUNA-7 size telemetry: measure the decoded README size with wc -c < /tmp/luna-ingest-readme.$$.md and record it in the synthesis § Audit log. If the size exceeds 512 KB, emit a single stderr warning before Phase 3 starts:
WARN luna-ingest: README is <N> KB (>512 KB threshold). --limit caps API-call count, NOT text size; large monorepo/awesome-list READMEs may produce a noisy ref pool.
Continue regardless — the warning is informational. --limit already bounds the per-ref API-call cost; the size warning just flags "expect noisy classification" for the operator.
Phase 3 — Per-ref classification
For each ref <o>/<r> in the cleaned ref list:
-
Fetch lightweight metadata: gh api "repos/${ref}" --jq '{description, language, stargazers_count, pushed_at, topics, license: .license.spdx_id}'. On failure, distinguish the failure class (LUNA-6):
- 404 / "Not Found" → verdict
skip, verdict_reason: ref_not_found. Continue scanning the rest.
- 403 / 429 / "rate limit" / "API rate limit exceeded" → verdict
api_failure, verdict_reason: rate_limited. Continue scanning; the run will exit rc=4 at the end so the operator knows to retry, but partial progress is preserved.
- Other network/auth errors → verdict
api_failure, verdict_reason: <stderr-short>. Continue; exit rc=4 at end.
api_failure is NOT one of integrate/take-parts/inspire/skip — it's a SEPARATE verdict that exists only to keep partial runs honest. Refs with verdict: api_failure go under § Unclassified (transient) in the synthesis, never § Skip (which implies a deliberate heuristic decision).
-
Assign a verdict per these heuristics (apply in order; first match wins):
integrate — the ref appears in a parent README section titled "Dependencies", "Requires", "Built on", "Uses" OR the topics overlap explicitly with a "package" or "library" tag AND the ref has an SPDX license that's MIT / Apache-2.0 / BSD-3-Clause (permissive).
take-parts — the ref appears under "Related", "Inspired by", "See also", "Forked from" sections AND the ref has ≥10 stars AND its language matches one of the parent's top 2 languages.
inspire — the ref appears under "Reading", "Articles", "Background", "Prior art" OR the ref's language differs from the parent's primary language but its topics overlap.
skip — everything else, OR the ref's pushed_at is more than 24 months old (stale), OR license is restrictive (AGPL, BSL) and the parent suggests integration.
-
Capture a 1-3 sentence rationale citing which README section / topic / license / staleness drove the verdict.
-
Record confidence 0.0–1.0: high (≥0.8) when section heading is unambiguous; medium (0.6–0.8) when section is implied by surrounding prose; low (<0.6) when no section context exists (verdict is best-guess from topics/license alone).
-
Refs with confidence <0.6 are listed in the synthesis under § Weak refs (appendix) instead of the main § Referenced repos section.
Phase 3.5 — Deep component scan (LUNA-57, --deep only)
Skip this entire phase unless --deep was passed.
Run the component scanner once per repo, gh-API only (no clone). The source
repo is ALWAYS scanned; a 1-hop ref is scanned ONLY when its Phase-3 verdict
is integrate or take-parts (the repos actually worth lifting from — refs
verdicted inspire/skip/api_failure are not scanned). Propagate the
source's trust_tier + safety_flag into the scan so every Components/ note
inherits the parent's risk annotation. Compute the source trust_tier inline
here using the Phase-4 "Trust-tier auto-classification" rules — all their
inputs (owner, stargazers_count, pushed_at, plus the safety_flag set
in Phase 1.5) are already available at this point, so there is no need to wait
for the Phase-4 Write step.
For the SOURCE repo (writes the deduped library + returns JSON for the
Tech-note section):
bun "$TOOLS/component-scan.mjs" --repo "$owner_repo" --vault "$VAULT" \
--components-dir "$COMPONENTS_DIR" --trust-tier "$trust_tier" \
--safety-flag "$safety_flag" --max-components "$MAX_COMPONENTS" --emit json \
[--dry-run]
Capture the JSON — its components[] array feeds the Phase-4
## Reusable components section.
For each ref <o>/<r> with verdict ∈ {integrate, take-parts} (library
dedup only; the Tech-note section summarises ref scans by count + link):
bun "$TOOLS/component-scan.mjs" --repo "<o>/<r>" --vault "$VAULT" \
--components-dir "$COMPONENTS_DIR" --trust-tier "$ref_trust_tier" \
--safety-flag "$safety_flag" --max-components "$MAX_COMPONENTS" --emit json \
[--dry-run]
$ref_trust_tier is computed per-ref from that ref's OWN Phase-3 metadata
(description / language / stars / pushed_at / license / topics already
fetched in Phase 3) via the same Phase-4 "Trust-tier auto-classification"
rules; $safety_flag here is the SOURCE's Phase-1.5 flag — refs are not run
through the Phase-1.5 pre-filter, so they conservatively inherit the parent
source's flag.
When the skill was invoked with --dry-run, pass --dry-run to
component-scan too (it then prints the JSON inventory + would-write notes
without touching the vault, rc=0).
$TOOLS = <plugin-root>/tools. The tool is HIMMEL-128 compliant (no
headless claude — it shells gh, not claude). A non-zero exit (rc=4)
means ≥1 component file failed to fetch; record the count under § Open
questions and continue — partial inventories are honest, not fatal.
Phase 3.6 — Research enrichment (LUNA-64, --research only)
Skip this entire phase unless --research was passed. Orthogonal to
Phase 3.5 — if both --deep and --research were passed, both phases run
(Phase 3.5 first, then this one).
This is the lightweight inline research path: you (Claude) run the web
research yourself via the WebSearch / WebFetch tools. Do NOT shell out to the
obsidian-second-brain research_deep.py script, and do NOT call Perplexity /
Grok — those (plus vault-wide /obsidian-save propagation) are the full
/research-deep command, deferred to a later LUNA-64 follow-up. Scope here:
the SOURCE repo only (1-hop refs are not researched in MVP — keeps the
tool-call budget bounded).
- Dry-run short-circuit. If
--dry-run was passed, skip the web queries
entirely (avoids spending tool calls on a no-write run); print
DRY luna-ingest: would run research enrichment for <owner_repo> to stderr
and continue to Phase 4 (the ## Research enrichment section is then
rendered as _(dry-run — research not executed)_; do NOT set
research_enriched — this is a degraded branch).
- Run 3–5 WebSearch queries about the source repo. Seed them from the
repo name + description + primary topics, covering these axes (drop any axis
the repo metadata makes irrelevant):
- maturity / adoption (is it actively used in production? release cadence?)
- notable alternatives / what people compare it to
- known limitations, gotchas, or common complaints
- community sentiment + recency (how recent is the discourse?)
Use WebFetch to pull a specific source when a search result needs
confirmation. Keep total fetches modest (≤5) — this is enrichment, not a
full research report.
- Synthesize the findings into 4–8 bullets for the
## Research enrichment section (rendered in Phase 4). Every external claim carries a
recency marker (the source's date, or "as of ") and a source
URL inline — consistent with the luna vault's AI-first sourcing rule.
- Graceful degradation. If WebSearch/WebFetch is unavailable or returns
nothing usable, do NOT fail the ingest: render the section as
_(research enrichment requested but web search returned no usable results)_, add a
one-line note under § Open questions, and continue. Do NOT set
research_enriched (the note is not enriched). A partial note is better
than no note — same rule as the component-scan partial path. No new exit
code: --research failure never changes rc.
Phase 4 — Synthesis
Compute a destination path under <vault>/<dest-category>/<repo-slug>.md where:
<repo-slug> = lowercase, hyphens, max 60 chars. e.g. kepano/obsidian-skills → kepano-obsidian-skills. Slugify: replace any non-[a-z0-9-] with -, collapse repeats, strip leading/trailing -.
- If
<dest-category> was passed via --dest, use it verbatim. Else default 30-Resources/Tech.
Path safety invariant: before any Write, canonicalise the resolved destination via realpath -m and verify it starts with the canonicalised vault root. Refuse with rc=2 if a .. traversal or symlink escape would land outside the vault. This guards against malformed --dest or weirdly-named refs.
If the target file already exists: refuse with rc=3 unless --force was passed (NOT supported in MVP — rc=3 with a hint to delete + retry).
Trust-tier auto-classification (LUNA-43)
Compute the synthesis's trust_tier field from the source repo's owner (extracted as owner = owner_repo.split('/')[0], then lowercased) before the Write step. All allowlist + literal comparisons in this section are case-insensitive — both sides lowercased before equality.
Apply rules in this order, top-to-bottom, first match wins — do NOT reorder. Every rule short-circuits all later rules; rules 3 and 4 are NOT mutually exclusive on their own (both could match a given repo), but the "first match wins" semantic resolves the ambiguity in favour of rule 3 when both fire.
- Safety short-circuit —
unknown-risk. If safety_flag (populated by Phase 1.5) is non-blank → trust_tier = unknown-risk, skip rules 1-4. This MUST execute before the owner-based rules so the audit-log Trust tier: unknown-risk (reason: safety_flag=<term>) is consistent regardless of who owns the repo.
anthropic-official — lowercased owner ∈ {anthropics, anthropic}.
known-author — lowercased owner ∈ the curated allowlist below. Default allowlist (all entries already lowercase):
- Individual authors:
kepano, mattpocock, karpathy, obra, wshobson, everyinc, nizos, letta-ai
- Orgs:
microsoft, vercel, vercel-labs, github, browser-use, hkuds, smtg-ai, nousresearch, zilliztech, topoteretes, mksglu
- Operator extension:
LUNA_INGEST_TRUSTED_AUTHORS=<comma-separated owners> adds to the allowlist (additions are lowercased before comparison; no removals).
community-active — none of the above matched AND stargazers_count ≥ 100 AND pushed_at within the last 180 days.
community-thin — none of the above matched (default fallback for any non-allowlisted owner that did not satisfy rule 3).
(stargazers_count is the field name returned by gh api repos/<owner_repo> in Phase 1 — do NOT alias it to stars in the rule logic; the frontmatter stars: field is the rendered value, the gh-api source name stays canonical in code paths.)
Populate trust_tier_reason with a 1-line justification matching the tier:
- rule 0 →
safety_flag=<term> → unknown-risk
- rule 1 →
owner=<owner-as-fetched> → anthropic-official
- rule 2 →
owner=<owner-as-fetched> in known-author allowlist
- rule 3 →
stargazers_count=<N>, pushed_at=<YYYY-MM-DD> → community-active
- rule 4 →
stargazers_count=<N>, pushed_at=<YYYY-MM-DD> → community-thin
Operators retro-tagging older Tech pages should rely on the Phase 4 § Audit log row Trust tier: <tier> (<reason>) rather than re-deriving manually.
Tags derivation (HIMMEL-247)
tags: is REQUIRED in every emitted note's frontmatter — the vault's Tech
notes are tag-queried (Dataview / triage tooling), and an untagged note is
invisible to those queries. The original Phase 4 / I-3 templates never
emitted it — every pre-fix tech-ingest note shipped untagged (all 29
type: tech-ingest notes as of 2026-06-11; HIMMEL-247 backfilled them).
The "~90 earlier Tech notes that carry tags" in the ticket are
clipper-pipeline type: tech-resource notes — a different emitter, not
luna-ingest output. Never omit it.
Rules (repo notes):
- First tag is always
github — matches the existing 30-Resources/Tech/
scheme (the dominant tag in that folder).
- Then the repo's
topics[] from the Phase 1 metadata, each normalized the
Phase 1.5 way (lowercase, runs of [-_\s]+ → a single -), deduped,
capped at 5 topic tags. When more than 5 topics exist, pick the most
content-salient ones (what the repo IS, not its stack minutiae), preferring
topics that already appear in the vault's tag vocabulary — NOT blind
API/alphabetical order.
- If the repo has no topics (or fewer than 2 survive normalization), infer
2–4 domain tags from the name + description + README — prefer the vault's
existing tag vocabulary (e.g.
claude-code, ai-agents, mcp,
obsidian, cli, automation, memory, skills) over minting a new
term; check existing Tech-note tags before coining one.
- Minimum 2 tags total — never a bare
tags: with no entries, never
github alone.
Issue notes (I-3) use the variant: github, github-issue, then up to 3
topic tags derived the same way (repo topics[] when the I-1 repo-metadata
fetch succeeded; else inferred from the issue labels + title).
Write the file. Structure:
---
type: tech-ingest
source: <URL>
source_type: github
ingested_at: <UTC ISO-8601>
last_revalidated: <UTC ISO-8601> # LUNA-43: initially same as ingested_at. Reserved for periodic revalidation tooling — currently unused by any sweep script. Future tooling can compute staleness from this field without confusing it with ingested_at (which never moves after first write).
ingest_command: /luna-ingest
verdict_summary:
integrate: <count>
take-parts: <count>
inspire: <count>
skip: <count>
api_failure: <count> # LUNA-6: refs we couldn't classify due to rate-limit / network / auth. NON-ZERO means partial run.
license: <SPDX or unknown>
stars: <N>
language: <primary>
topics:
- <topic>
tags: # HIMMEL-247: REQUIRED — never omit. Derivation rules in "Tags derivation" above.
- github
- <tag>
trust_tier: <tier> # LUNA-43: anthropic-official | known-author | community-active | community-thin | unknown-risk
trust_tier_reason: <text> # LUNA-43: 1-line justification (e.g. "owner=anthropics → anthropic-official"; "owner=kepano in allowlist → known-author"; "stargazers_count=12, pushed_at=2025-04-01 → community-thin"). Field name is `owner=` (the lowercased value extracted in trust-tier rule logic), matching the trust-tier rule wording — NOT `owner_org=`.
safety_flag: <term-or-blank> # LUNA-43: blank if Phase 1.5 found no red-flag terms; matched term otherwise. NON-BLANK forces trust_tier=unknown-risk via rule 0 short-circuit in trust-tier classification.
research_enriched: <true-or-omit> # LUNA-64: set true ONLY when --research ran (Phase 3.6) AND produced real findings. Omit the field entirely when --research was not passed OR enrichment degraded (dry-run short-circuit / web-failure placeholder) — a degraded note must not advertise itself as enriched to downstream readers (/synthesize-clips, revalidation tooling).
---
# <repo-name>
## Source
[<full_name>](<URL>) — <description from gh api>
- **Language:** <language> | **Stars:** <N> | **License:** <SPDX>
- **Last push:** <pushed_at>
- **Topics:** <topics joined by ", ">
## Summary
<3-5 bullet TL;DR derived from README + repo metadata>
## Referenced repos (1-hop)
<for each ref with confidence ≥0.6, sorted by verdict severity integrate > take-parts > inspire > skip>
### <ref-full-name> — <verdict>
- URL: https://github.com/<ref>
- Language: <lang> | Stars: <N> | License: <SPDX>
- **Rationale:** <1-3 sentences>
<if verdict == take-parts>
- **What to lift:** <file/path/area to inspect first>
</if>
## Research enrichment
<Only present when --research was passed (Phase 3.6). 4–8 bullets synthesizing
the WebSearch findings about the SOURCE repo — maturity/adoption, notable
alternatives, known limitations, community sentiment. Each external claim ends
with a recency marker + source URL inline, e.g.:
`- Actively maintained; v2.3 released as of Mar 2026 ([release notes](https://…)).`
On dry-run, render `_(dry-run — research not executed)_`. On web failure,
render `_(research enrichment requested but web search returned no usable
results)_`. If --research was NOT passed, omit this section entirely.>
## Reusable components
<Only present when --deep was passed. Compose from the source-repo scan
JSON. For each component in `components[]`, one bullet:
`- **<name>** (<type>) — <description>. _Reuse:_ <1-2 line how-to-rephrase
for himmel/luna — YOUR judgment from the description + type>.`
Then, for each integrate/take-parts ref that was scanned, one summary line:
`- _Ref <o>/<r>: N components inventoried → see [[30-Resources/Components/]]._`
If --deep was NOT passed, omit this section entirely.>
## Weak refs (low-confidence, appendix)
<for each ref with confidence <0.6 — same structure, condensed>
## Unclassified (transient — re-run to retry)
<for each ref with `verdict: api_failure` — list URL + verdict_reason. LUNA-6: surfaces rate-limited / network-failed refs distinctly from § Skip so operator knows what to retry. Empty section reads `_(none — all refs classified)_`.>
## Action items
<extracted from README headings like "TODO", "Roadmap", "Known issues" — convert each to `- [ ] <item>`. If none found: `_(none — repo README has no TODO/Roadmap section)_`>
## Open questions
<gaps the chain-follow could not resolve. Always include: "Verify integrate-verdict refs by reading their root README before taking a dep on any of them.">
## Audit log
- Refs scanned: <N>
- Refs above confidence floor (0.6): <K>
- Refs marked api_failure (LUNA-6): <F> (if >0: rc=4; re-run to retry classification)
- Tail-skipped: <T> (set if more refs existed than --limit)
- API calls: <N+1>
- README size: <bytes> (LUNA-7: warn if >512 KB; --limit caps API-call count, not text size)
- Trust tier: <tier> (<reason>) (LUNA-43)
- Safety flag: <term-or-none> (LUNA-43: non-blank value reached rc=6 unless --allow-unsafe; document the term in this row regardless)
Phase 5 — Daily-note backref (deferred to v2)
MVP does NOT touch the daily note. v2 picks up the obsidian-triage Phase 5 dedup-by-backreference contract.
Phase 6 — Promotion candidate (deferred to v2)
MVP does NOT write 00-Inbox/promote-<slug>.md. v2 adds this when any verdict ∈ {integrate, take-parts}.
Dry-run mode
When --dry-run is passed:
- Print "DRY luna-ingest: would write "
- Print the synthesized markdown body to stdout, prefixed with 4 spaces
- Print per-ref verdict summary table
- With
--research: skip the Phase 3.6 web queries (print DRY luna-ingest: would run research enrichment for <owner_repo>); the ## Research enrichment section renders _(dry-run — research not executed)_.
- rc=0, touch no files in the vault
Exit codes
- 0: synthesized + written (or dry-run completed) — all refs classified
- 1: usage / input error (bad URL, missing required arg)
- 2: env unusable (vault not found, gh api / bitbucket CLI unreachable or unbuilt for source repo, dest category invalid)
- 3: target file exists (no overwrite in MVP)
- 4: partial run — synthesis written but ≥1 ref was marked
api_failure (rate-limit / network / auth). Re-run later to retry the failed refs; the dedup-by-existing-file invariant means the existing synthesis stays unless --force. (LUNA-6)
- 4 (extended): also returned when
--deep component scan reported ≥1
component-fetch failure (partial inventory). Synthesis is still written.
- 5: source fetch failed —
gh api (github Phase 1) or the bitbucket CLI (HIMMEL-329 Bitbucket branch) failed for the SOURCE repo / PR / issue (network / rate limit / 404 / issues-disabled) — nothing written, retry the whole call
- 6: safety-blocked — Phase 1.5 matched a red-flag keyword in name/description/topics/README (github) or name/description/README (bitbucket — no topics), and
--allow-unsafe was NOT passed. Nothing written. Re-run with --allow-unsafe to ingest with safety_flag annotation (which exits rc=0, NOT rc=6, on success — --allow-unsafe plus a match is a known-risk acknowledged ingest, not an error). (LUNA-43)
--research (LUNA-64) never changes the exit code: a web-search failure
degrades gracefully (annotated section + § Open questions note) and the
synthesis is still written at the rc the rest of the run would have produced.
Out of scope (v2 follow-ups)
- Twitter URL input via
/x-read
- Article URL input via
/defuddle + /research
- github PR (
/pull/<n>) + discussion URL ingestion (github issue URLs landed in HIMMEL-239; bitbucket repo/PR/issue landed in HIMMEL-329)
- bitbucket-source → bitbucket-ref 1-hop following (HIMMEL-329 MVP follows only github refs from a bitbucket README)
- bitbucket PR comment-thread + issue comment ingestion (the MVP bitbucket PR/issue notes carry no comment thread)
--deep component scan of a bitbucket SOURCE repo (component-scan is gh-API-only)
- Dynamic destination routing based on verdict mix
- Daily-note backref (Phase 5)
- Promotion candidate (Phase 6)
- Refs-of-refs (hop ≥2)
- Batch input (
@file-of-urls.txt)
- Reverse-chain lookup
Each is a separate ticket under LUNA-5 (file as sub-tasks before starting v2).