teach

# /bedrock:teach — External Source Ingestion into the Second Brain ## Plugin Paths Entity definitions and templates are in the plugin directory, not at the vault root. Use the "Base directory for this skill" provided at invocation to resolve paths: - Entity definitions: `<base_dir>/../../entities/` - Templates: `<base_dir>/../../templates/{type}/_template.md` - Plugin CLAUDE.md: `<base_dir>/../../CLAUDE.md` (already injected automatically into context) Where `<base_dir>` is the path provided in "Base directory for this skill". --- ## Vault Resolution Resolve which vault to teach. This skill can be invoked from any directory. **Step 1 — Parse `--vault` flag:** Check if the input arguments include `--vault <name>`. If found, extract the vault name and remove it from the arguments (the remaining text is the source URL/path). **Step 2 — Resolve vault path:** 1. **If `--vault <name>` was provided:** Read the vault registry at `<base_dir>/../../vaults.json`. Find the entry matching the name. If not found: error — "Vault `<name>` is not registered. Run `/bedrock:vaults` to see available vaults." If found: set `VAULT_PATH` to the entry's `path` value. Store the resolved vault name as `VAULT_NAME`. 2. **If no `--vault` flag — CWD detection:** Read `<base_dir>/../../vaults.json`. Check if the current working directory is inside any registered vault path (CWD starts with a registered vault's absolute path). If multiple match, use the longest path (most specific). If found: set `VAULT_PATH` to the matching vault's `path`. Store its name as `VAULT_NAME`. 3. **If CWD detection fails — default vault:** From the registry, find the vault with `"default": true`. If found: set `VAULT_PATH` to the default vault's `path`. Store its name as `VAULT_NAME`. 4. **If no resolution:** Error — "No vault resolved. Available vaults:" followed by the registry listing. "Use `--vault <name>` to specify, or run `/bedrock:setup` to register a vault." **Step 3 — Validate vault path:** ```bash test -d "<VAULT_PATH>" && echo "exists" || echo "missing" ``` If missing: error — "Vault path `<VAULT_PATH>` does not exist on disk. Run `/bedrock:setup` to re-register." **Step 4 — Read vault config:** ```bash cat <VAULT_PATH>/.bedrock/config.json 2>/dev/null ``` Extract `language` and other relevant fields for use in later phases. **From this point forward, ALL vault file operations use `<VAULT_PATH>` as the root.** - Graphify output: `<VAULT_PATH>/graphify-out/` - When delegating to `/bedrock:preserve`, pass `--vault <VAULT_NAME>` --- ## Overview This skill receives an external source (URL or local path), fetches its content to a temporary directory, converts non-markdown files to markdown via docling, runs the `/graphify` extraction pipeline on the tmp content, and delegates entity persistence (plus graphify-output merge) to `/bedrock:preserve`. **You are a fetcher and orchestrator agent.** Your job is to: 1. Ensure docling is installed (auto-install if missing) 2. Classify the input and fetch content to `/tmp` 3. Convert fetched files to markdown via docling (when applicable) 4. Invoke `/graphify` to extract a knowledge graph into a per-run temp directory 5. Delegate graph merge and entity writes to `/bedrock:preserve` 6. Clean up temporary files You do NOT classify entities, create vault files, write to the vault directly, or merge graph state. All extraction is done by `/graphify`. All writes (including the graphify-output merge into the vault's cumulative `graphify-out/`) are done by `/bedrock:preserve`. Follow the phases below in order, without skipping steps. --- ## Phase 0 — Ensure docling is installed Before any fetch or conversion, verify that the `docling` CLI is available. If missing, install it silently using the same fallback chain `/bedrock:setup` uses for graphify, emitting a single status line before proceeding. ```bash if command -v docling >/dev/null 2>&1; then echo "Phase 0: docling already installed — proceeding." else echo "Phase 0: docling not found — installing silently (one-time setup, may take a few minutes for model download)." # Step 1 — pipx (preferred, isolated) if command -v pipx >/dev/null 2>&1; then pipx install docling >/dev/null 2>&1 || true fi # Step 2 — pip (fallback if pipx unavailable or failed) if ! command -v docling >/dev/null 2>&1; then if command -v pip3 >/dev/null 2>&1; then pip3 install --user docling >/dev/null 2>&1 || true elif command -v pip >/dev/null 2>&1; then pip install --user docling >/dev/null 2>&1 || true fi fi # Final re-probe if ! command -v docling >/dev/null 2>&1; then echo "ERROR: docling install failed. Run /bedrock:setup to install it, or install manually: pipx install docling" exit 1 fi echo "Phase 0: docling installed." fi ``` **Failure mode:** If install fails (no `pipx`/`pip`, network outage, permission denied), abort the skill with the error above. Do NOT fetch or mutate anything. Direct the user to `/bedrock:setup`. **No user prompt:** this step is silent — one status line on success, one error line on failure. --- ## Phase 1 — Fetch ### 1.1 Classify the input The user provides an argument. Classify it in the following priority order. URL-type routing is unchanged; local files no longer have an extension allowlist — any existing file is accepted, and Phase 1.5 decides whether to run docling on it. | Input | Detected type | Fetch method | |---|---|---| | URL containing `confluence` or `atlassian.net` | confluence | Read `skills/confluence-to-markdown/SKILL.md`, follow instructions, save output to tmp | | URL containing `docs.google.com` | gdoc | Read `skills/gdoc-to-markdown/SKILL.md`, follow instructions, save output to tmp | | URL containing `github.com` | github-repo | `git clone --depth 1` to tmp + GitHub MCP enrichment (docling never runs on GitHub repos) | | URL starting with `http://` or `https://` (any other) | remote-binary | Download raw bytes to tmp via `curl`/WebFetch; Phase 1.5 decides conversion | | Local file path (any existing file) | local-file | Copy to tmp; Phase 1.5 decides conversion | | Local directory path | local-dir | Copy directory to tmp | | No match above | manual | Ask the user: "Could not identify the source type. Paste the content or provide a valid URL/path." | If no argument was provided: ask the user "What source do you want to ingest? Provide a URL (Confluence, Google Docs, GitHub, or any HTTP(S) URL) or a local file path (any file type — docling will convert it to markdown if supported)." ### 1.2 Create temporary directory All content is fetched to a temporary directory. This is the single input path for `/graphify`. ```bash TEACH_TMP="/tmp/bedrock-teach-$(date +%s)" mkdir -p "$TEACH_TMP" echo "Temporary directory: $TEACH_TMP" ``` Store the path for use in subsequent phases. ### 1.3 Fetch content Execute the fetch strategy for the detected type. All content lands in `$TEACH_TMP/`. #### 1.3.1 GitHub repository For GitHub URLs (e.g.: `https://github.com/acme-corp/billing-api`): 1. Extract `owner/repo` and `repo-name` from the URL 2. Clone the repository (shallow): ```bash git clone --depth 1 <url> "$TEACH_TMP/<repo-name>" ``` 3. GitHub MCP enrichment — call directly in main context (NOT via subagent — MCP permissions are not inherited): - `mcp__plugin_github_github__get_file_contents` → read the repo's README.md - `mcp__plugin_github_github__list_commits` → last 10 commits - `mcp__plugin_github_github__list_pull_requests` → last 5 PRs (state=all, sort=updated) 4. Compile MCP results into a single markdown file and save as `$TEACH_TMP/<repo-name>/_github_metadata.md` > **Best-effort:** If any MCP call fails, continue with what was obtained. Do NOT block ingestion. #### 1.3.2 Confluence For Confluence URLs: 1. Read the internal skill at `<base_dir>/../confluence-to-markdown/SKILL.md` 2. Follow its instructions to parse the URL, choose layer (MCP → API → browser), and extract content 3. Save the returned Markdown content to `$TEACH_TMP/<slug>.md` - `<slug>` is derived from the page title or URL path (kebab-case, lowercase) If all three layers (MCP, API, browser) are unavailable: warn the user with the guidance message from the fetcher module and abort this source type. #### 1.3.3 Google Docs / Sheets For Google Docs or Sheets URLs: 1. Read the internal skill at `<base_dir>/../gdoc-to-markdown/SKILL.md` 2. Follow its instructions to parse the URL, detect document type (Doc vs Sheet), choose layer (MCP → API/public export → browser), and extract content 3. The fetcher saves output to `/tmp/gdoc_{docId}.md` or `/tmp/gsheet_{docId}.md` 4. Copy the output file to `$TEACH_TMP/<slug>.md` - `<slug>` is derived from the document title or URL path (kebab-case, lowercase) If all three layers (MCP, API/public export, browser) are unavailable: warn the user with the guidance message from the fetcher module and abort this source type. #### 1.3.4 Remote URL (generic) For any other HTTP/HTTPS URL, download the raw bytes so docling can operate on binary formats (PDF, DOCX, PPTX, XLSX, images, etc.) that WebFetch cannot return faithfully as text: 1. Try `curl` first for true binary fidelity: ```bash curl -fsSL -o "$TEACH_TMP/<filename-derived-from-url>" "<url>" ``` - `<filename-derived-from-url>` preserves the URL's basename (including extension) when available; fall back to `<slug>.bin` if no extension is present. 2. If `curl` is unavailable or the URL returns an HTML page (by Content-Type), fall back to WebFetch and save the response text as `$TEACH_TMP/<slug>.md`. If both attempts fail: warn "Could not fetch URL. Check if the URL is accessible." and abort. Phase 1.5 decides whether the downloaded file goes through docling, based on the file extension. #### 1.3.5 Local file (any format) For local files: 1. Verify the file exists using Read (or `test -f`). 2. Copy to tmp preserving the filename: ```bash cp "<local-path>" "$TEACH_TMP/" ``` No extension-based filtering — any existing file is accepted. Phase 1.5 decides conversion. #### 1.3.6 Local directory For local directories: 1. Verify the directory exists 2. Copy to tmp (excluding heavy directories): ```bash rsync -a --exclude='.git' --exclude='node_modules' --exclude='bin' --exclude='obj' \ --exclude='.vs' --exclude='TestResults' --exclude='packages' \ "<local-dir>/" "$TEACH_TMP/$(basename <local-dir>)/" ``` ### 1.4 Phase 1 result At the end of this phase, you should have: - **`$TEACH_TMP`**: directory with all fetched content (local path for graphify) - **`source_url`**: original URL or file path provided by the user - **`source_type`**: `confluence`, `gdoc`, `github-repo`, `remote-binary`, `local-file`, `local-dir`, or `manual` Report: "Phase 1 complete: Content fetched to `$TEACH_TMP`. Source type: `<source_type>`." --- ## Phase 1.5 — Docling Conversion For every fetched file in `$TEACH_TMP` that is not a GitHub repo and is not already markdown output from Confluence/GDoc fetchers, check whether docling supports the file type and, if so, convert it to markdown in place. GitHub repos (`source_type == "github-repo"`) skip this phase entirely and flow straight to graphify. ### 1.5.1 Docling-supported extensions Docling supports conversion for the following file types (as of the version installed by Phase 0 / `/bedrock:setup`). Compare by lowercase file extension: ``` .pdf .docx .pptx .xlsx .html .htm .md .adoc .png .jpg .jpeg .tiff .bmp .epub ``` - `.md` is listed here because docling passes markdown through largely unchanged. In practice, running docling on `.md` is a no-op we skip to save time — treat `.md` as already-markdown. - `.txt` and `.csv` are NOT in docling's supported list (they are plain-text already); skip docling and pass through raw. ### 1.5.2 Routing and failure rules For each file under `$TEACH_TMP` (excluding files inside `<repo-name>/` subdirectories of a `github-repo` source — skip those entirely): 1. **Skip by type — already markdown or plain text:** if extension is `.md`, `.txt`, or `.csv`, leave the file untouched and record status `passed-through` for the report. Graphify handles these natively. 2. **Skip by routing — not docling-supported:** if the extension is not in the supported list above AND is not `.md`/`.txt`/`.csv`, leave the file untouched and record status `passed-through` with a note `(type not supported by docling)`. Graphify decides what to do with the raw file. 3. **Run docling:** otherwise, invoke docling and replace the source file with the converted markdown. Docling writes to the working directory by default; use `--to md` and `--output` to target a predictable path: ```bash cd "$TEACH_TMP" docling --from <auto> --to md --output "$TEACH_TMP" "<relative-file-path>" ``` Docling produces `<stem>.md` alongside the source. After a successful run: - Remove the original binary: `rm "<relative-file-path>"`. - Record status `converted` for the report with the new markdown filename. 4. **Failure fallback:** if docling exits non-zero for a file: - If the source file's extension is `.md`, `.txt`, or `.csv` (already handled by rule 1, so this branch is defensive): leave the original file in place, record status `failed-fallback (raw passthrough)`, and continue with other files. - Otherwise (binary format like `.docx`, `.pdf`, etc.): **abort the entire skill**. Clean up `$TEACH_TMP` (`rm -rf "$TEACH_TMP"`) and emit a clear error: `ERROR: docling failed to convert <file>. Aborting ingestion. Temp directory cleaned up.` Do NOT proceed to graphify or preserve. ### 1.5.3 Phase 1.5 result At the end of Phase 1.5: - `$TEACH_TMP` contains markdown files (either originals or docling-converted). - You have a per-file status map to surface in Phase 4's report: - `converted`: ran docling successfully - `passed-through`: skipped docling (markdown/plain text or unsupported type) - `failed-fallback`: docling failed but file was text-native; continued with raw file Report: "Phase 1.5 complete: N converted, M passed-through, P failed-fallback." --- ## Phase 2 — Extract ### 2.1 Invoke /graphify into a per-run temp directory Use the Skill tool to invoke `/graphify`, directing its output to a per-run temp directory (**not** the vault). The vault's cumulative `graphify-out/` is updated by `/bedrock:preserve`'s