okf-ingest

name

okf-ingest

description

Bring an existing source into Open Knowledge Format (OKF) compliance — a code repository, a documentation set, an existing/partial knowledge bundle, or a "second brain" (Obsidian vault, Logseq graph, Notion export, Markdown notes). Use when the user wants to convert / ingest / migrate / onboard existing content into OKF, make a repo or docs "OKF-compliant", or turn notes into a knowledge bundle. The skill ANALYZES the source, SUMMARIZES what it found, RECOMMENDS whether to convert fully, partially (derive a knowledge bundle), or not at all, then QUERIES the user with concrete options and IMPLEMENTS the choice non-destructively. Triggers on "okf-ingest", "convert to OKF", "ingest into OKF", "make this OKF-compliant", "migrate my notes/repo/vault to OKF".

okf-ingest — convert existing sources into OKF

This skill turns material that already exists into a conformant OKF bundle. It is the counterpart to the okf skill: okf is the format authority (spec, templates, validator, produce/consume workflows); okf-ingest is the migration engine on top of it.

Always defer to the okf skill for the format itself. Read its reference/spec.md for the normative rules, use its templates/ when emitting files, and validate output with its scripts/validate_okf.py. If the okf skill is installed as a sibling, it is at ../okf/ relative to this skill directory; otherwise invoke it via the Skill tool. Do not restate or fork the spec here.

The non-negotiable shape of this skill

Run these five steps in order. Step 4 (query) is mandatory — never convert anything until the user has explicitly chosen scope and mode.

1. Analyze the source.
2. Summarize findings back to the user.
3. Recommend a conversion verdict per segment (full / partial / none).
4. Query the user with concrete, selectable options (AskUserQuestion).
5. Implement exactly what they selected — non-destructively by default — then validate and report.

Input: a path or a URL

The typical invocation is "use the okf-ingest skill on this: <target>", where <target> is either a local path or a URL. Resolve it to local files before analyzing:

• Local path (file or directory) — use directly.
• Git repo URL (GitHub/GitLab/etc.) — clone it shallowly into the scratchpad (git clone --depth 1 <url> <scratch>/<name>) and analyze the clone. Use gh for private GitHub repos if available.
• A single web page / raw file URL — fetch it (WebFetch, or gh api for GitHub contents). For a docs site, fetch the entry point and crawl a bounded set of linked pages the user cares about; don't spider the whole web.
• A hosted bundle / archive (tarball, zip) — download into the scratchpad and expand before analyzing.

Acquire into the scratchpad, never into the user's project, and note where you put it. If acquisition needs credentials you don't have, ask the user to run the clone/login themselves (e.g. via the ! prefix) and point you at the result.

1. Analyze

Locate the target path(s) the user named (default: the current directory; or the acquired copy if the input was a URL). Then:

• Run the bundled inventory script for a deterministic scan:

  python <skill_dir>/scripts/analyze_source.py <target> --json
  

  (Try python, then py, then the full interpreter path. It needs no dependencies.) It reports file/extension counts and sizes, markdown stats (how many have frontmatter, how many use [[wikilinks]]), detected source-type signals, top-level structure, and segments that should likely be excluded (build artifacts, vendored deps, binaries, secrets).
• Supplement the scan by reading the high-signal files yourself: top-level README, any docs/ entry points, existing frontmatter samples, config files that identify the source type (mkdocs.yml, package.json, .obsidian/, etc.), and a representative sample of content files.
• Classify the source type using reference/source-adapters.md.

2. Summarize

Give the user a tight summary (not a file dump): source type, total volume, which parts are knowledge-bearing vs. operational/ephemeral, existing structure and metadata you can reuse, the natural concept groupings you see, and what you'd exclude. Keep it scannable.

3. Recommend

Apply reference/decision-framework.md to reach a verdict per segment, not just one global verdict. The three verdicts:

• Full conversion — the source is already knowledge-as-markdown (existing/ partial bundle, Obsidian/Logseq/Notion, a curated docs site). Map ~1:1 into OKF.
• Partial / derived conversion — a code repository or mixed repo. Do not convert source files into concepts; instead extract the knowledge about them (architecture overview, module/service catalog, API & schema docs, domain glossary, runbooks) into a new sidecar bundle, and convert only the genuinely knowledge-bearing files (README, docs/, ADRs).
• Not at all (or thin index only) — ephemeral, highly volatile, low-knowledge, generated, or already well-served elsewhere. Recommend skipping or at most a shallow index.

State your recommendation with a one-line rationale for each segment. A real source is usually a mix (e.g. "convert docs/ fully, derive a bundle from src/, skip dist/ and node_modules/").

4. Query (mandatory)

Use AskUserQuestion to let the user decide. Build the options from your analysis; put your recommended choice first and mark it "(Recommended)". Ask only what actually changes the work — typically:

• Scope — which segments to include, and at what verdict (accept the recommended mix / convert everything fully / pick a custom subset / cancel).
• Mode — Sidecar (new bundle beside the source, source untouched — default and safest) vs. In-place (add frontmatter / restructure the source itself — only for material the user owns and ideally under version control).
• Depth — catalog (index-style concepts pointing at sources) vs. full concept docs (frontmatter + summarized body per concept) vs. derived extraction (synthesize new knowledge from code/config).
• Output location — bundle root path (default knowledge/ or okf/).

If the source is small and the recommendation is obvious, you may collapse this to a single confirming question — but still confirm before writing.

5. Implement

Follow the matching recipe in reference/source-adapters.md. Across all conversions:

• Emit concept docs from the okf skill's templates/concept.md. Infer frontmatter: choose a precise type, derive title/description, carry over existing tags, set timestamp (use the real current date), and set resource to the source file path or URL so provenance is preserved. Add a # Citations entry pointing back to the origin.
• Rewrite links into OKF form: convert [[wikilinks]] and source-relative links into bundle-relative (/...) markdown links. Tolerate/leave broken links (they're allowed) but list them in the report.
• Generate index.md at every directory level and a root log.md with an **Initialization** entry describing the ingest. Put okf_version: "0.1" only on the root index.md.
• Be non-destructive by default. Sidecar mode writes a new tree and never touches the source. For in-place mode, confirm first, operate only on user-owned/VCS-tracked files, and never delete source content — only add frontmatter / move files as agreed.
• Never ingest secrets, credentials, large binaries, or generated artifacts. If the scan flags any, exclude them and say so.

Then validate (okf skill's validate_okf.py) and report: the output tree, a conversion log (converted / skipped / needs-human-review), and any broken links or low-confidence concepts the user should review.

Notes

• Prefer doing less destructively over doing more: when unsure whether something is knowledge-bearing, leave it out of the bundle and note it as a candidate.
• For very large sources, propose a phased ingest (one segment first) rather than converting everything in one pass.