reflect

## Runtime Configuration (Step 0 — before any processing) Read these files to configure domain-specific behavior: 1. **`ops/derivation-manifest.md`** — vocabulary mapping, platform hints - Use `vocabulary.notes` for the notes folder name - Use `vocabulary.note` / `vocabulary.note_plural` for note type references - Use `vocabulary.reflect` for the process verb in output - Use `vocabulary.topic_map` / `vocabulary.topic_map_plural` for MOC references - Use `vocabulary.cmd_reweave` for the next-phase suggestion - Use `vocabulary.inbox` for the inbox folder name 2. **`ops/config.yaml`** — processing depth, pipeline chaining - `processing.depth`: deep | standard | quick - `processing.chaining`: manual | suggested | automatic If these files don't exist, use universal defaults. **Processing depth adaptation:** | Depth | Connection Behavior | |-------|-------------------| | deep | Full dual discovery (MOC + semantic search). Evaluate every candidate. Multiple passes. Synthesis opportunity detection. Bidirectional link evaluation for all connections. | | standard | Dual discovery with top 5-10 candidates. Standard evaluation. Bidirectional check for strong connections only. | | quick | Single pass — either MOC or semantic search. Accept obvious connections only. Skip synthesis detection. | ## EXECUTE NOW **Target: $ARGUMENTS** Parse immediately: - If target contains `[[note name]]` or note name: find connections for that {vocabulary.note} - If target contains `--handoff`: output RALPH HANDOFF block at end - If target is empty: check for recently created {vocabulary.note_plural} or ask which {vocabulary.note} - If target is "recent" or "new": find connections for all {vocabulary.note_plural} created today **Execute these steps:** 1. Read the target {vocabulary.note} fully — understand its claim and context 2. **Throughout discovery:** Capture which {vocabulary.topic_map_plural} you read, which queries you ran (with scores), which candidates you evaluated. This becomes the Discovery Trace — proving methodology was followed, not reconstructed. 3. Run Phase 0 (index freshness check) 4. Use dual discovery in parallel: - Browse relevant {vocabulary.topic_map}(s) for related {vocabulary.note_plural} - Run semantic search for conceptually related {vocabulary.note_plural} 5. Evaluate each candidate: does a genuine connection exist? Can you articulate WHY? 6. Add inline wiki-links where connections pass the articulation test 7. Update relevant {vocabulary.topic_map}(s) with this {vocabulary.note} 8. If task file in context: update the {vocabulary.reflect} section 9. Report what was connected and why 10. If `--handoff` in target: output RALPH HANDOFF block **START NOW.** Reference below explains methodology — use to guide, not as output. --- # Reflect Find connections, weave the knowledge graph, update {vocabulary.topic_map_plural}. This is the forward-connection phase of the processing pipeline. ## Philosophy **The network IS the knowledge.** Individual {vocabulary.note_plural} are less valuable than their relationships. A {vocabulary.note} with fifteen incoming links is an intersection of fifteen lines of thought. Connections create compound value as the vault grows. This is not keyword matching. This is semantic judgment — understanding what {vocabulary.note_plural} MEAN to determine how they relate. A {vocabulary.note} about "friction in systems" might deeply connect to "verification approaches" even though they share no words. You are building a traversable knowledge graph, not tagging documents. **Quality over speed. Explicit over vague.** Every connection must pass the articulation test: can you say WHY these {vocabulary.note_plural} connect? "Related" is not a relationship. "Extends X by adding Y" or "contradicts X because Z" is a relationship. Bad connections pollute the graph. They create noise that makes real connections harder to find. When uncertain, do not connect. ## Invocation Patterns ### /reflect (no argument) Check for recent additions: 1. Look for {vocabulary.note_plural} modified in the last session 2. If none obvious, ask user what {vocabulary.note_plural} to connect ### /reflect [note] Focus on connecting a specific {vocabulary.note}: 1. Read the target {vocabulary.note} 2. Discover related content 3. Add connections and update {vocabulary.topic_map_plural} ### /reflect [topic area] Synthesize an area: 1. Read the relevant {vocabulary.topic_map} 2. Identify {vocabulary.note_plural} that should connect 3. Weave connections, update synthesis ### /reflect --handoff [note] External loop mode for /ralph: - Execute full workflow as normal - At the end, output structured RALPH HANDOFF block - Used when running isolated phases with fresh context per task ## Workflow ### Phase 0: Verify Index Freshness Before using semantic search, verify the index is current. This is self-healing: if {vocabulary.note_plural} were created outside the pipeline (manual edits, other skills), reflect catches the drift before searching. 1. Try `mcp__qmd__status` to get the indexed document count for the target collection 2. **If MCP unavailable** (tool fails or returns error): fall back to bash: ```bash LOCKDIR="ops/queue/.locks/qmd.lock" while ! mkdir "$LOCKDIR" 2>/dev/null; do sleep 2; done qmd_count=$(qmd status 2>/dev/null | grep -A2 '{vocabulary.notes_collection}' | grep 'documents' | grep -oE '[0-9]+' | head -1) rm -rf "$LOCKDIR" ``` 3. Count actual files: ```bash file_count=$(ls -1 {vocabulary.notes}/*.md 2>/dev/null | wc -l | tr -d ' ') ``` 4. If the counts differ, sync the index: ```bash qmd update && qmd embed ``` Run this check before proceeding. If stale, sync and continue. If current, proceed immediately. ### Phase 1: Understand What You Are Connecting Before searching for connections, deeply understand the source material. For each {vocabulary.note} you are connecting: 1. Read the full {vocabulary.note}, not just title and description 2. Identify the core claim and supporting reasoning 3. Note key concepts, mechanisms, implications 4. Ask: what questions does this answer? What questions does it raise? **What you are looking for:** - The central argument (what is being claimed?) - The mechanism (why/how does this work?) - The implications (what follows from this?) - The scope (when does this apply? When not?) - The tensions (what might contradict this?) **If a task file exists** (pipeline execution): read the task file to see what the extraction phase discovered. The reduce notes, semantic neighbor field, and classification provide critical context about why this {vocabulary.note} was extracted and what it relates to. ### Phase 2: Discovery (Find Candidates) Use dual discovery: {vocabulary.topic_map} exploration AND semantic search in parallel. These are complementary, not sequential. **Capture discovery trace as you go.** Note which {vocabulary.topic_map_plural} you read, which queries you ran (with scores), which searches you tried. This becomes the Discovery Trace section in output — proving methodology was followed, not reconstructed after the fact. **Primary discovery (run in parallel):** **Path 1: {vocabulary.topic_map} Exploration** — curated navigation If you know the topic (check the {vocabulary.note}'s Topics footer), start with the {vocabulary.topic_map}: - Read the relevant {vocabulary.topic_map}(s) - Follow curated links in Core Ideas — these are human/agent-curated connections - Note what is already connected to similar concepts - Check Tensions and Gaps for context - What do agent notes reveal about navigation? {vocabulary.topic_map_plural} tell you what thinking exists and how it is organized. Someone already decided what matters for this topic. **Path 2: Semantic Search** — find what {vocabulary.topic_map_plural} might miss **Three-tier fallback for semantic search:** **Tier 1 — MCP tools (preferred):** Use `mcp__qmd__deep_search` (hybrid search with expansion + reranking): - query: "[{vocabulary.note}'s core concepts and mechanisms]" - limit: 15 **Tier 2 — bash qmd with lock serialization:** If MCP tools fail or are unavailable: ```bash LOCKDIR="ops/queue/.locks/qmd.lock" while ! mkdir "$LOCKDIR" 2>/dev/null; do sleep 2; done qmd query "[note's core concepts]" --collection {vocabulary.notes_collection} --limit 15 2>/dev/null rm -rf "$LOCKDIR" ``` The lock prevents multiple parallel workers from loading large models simultaneously. **Tier 3 — grep only:** If both MCP and bash fail, log "qmd unavailable, grep-only discovery" and rely on {vocabulary.topic_map} + keyword search only. This degrades quality but does not block work. Evaluate results by relevance — read any result where title or snippet suggests genuine connection. Semantic search finds {vocabulary.note_plural} that share MEANING even when vocabulary differs. A {vocabulary.note} about "iteration cycles" might connect to "learning from friction" despite sharing no words. **Why both paths:** {vocabulary.topic_map} = what is already curated as relevant semantic search = neighbors that have not been curated yet Using only search misses curated structure. Using only {vocabulary.topic_map} misses semantic neighbors outside the topic. Both together catch what either alone would miss. **Secondary discovery (after primary):** **Step 3: Keyword Search** For specific terms and exact matches: ```bash grep -r "term" {vocabulary.notes}/ --include="*.md" ``` Use grep when: - You know the exact words that should appear - Searching for specific terminology or phrases - Finding all uses of a named concept - The vocabulary is stable and predictable **Choosing between semantic and keyword:** | Situation | Better Tool | Why | |-----------|-------------|-----| | Exploring unfamiliar territory | semantic | vocabulary might not match meaning | | Finding synonyms or related framings | semantic | same concept, different words | | Known terminology | keyword | exact match, no ambiguity | | Verifying coverage | keyword | ensures nothing missed | | Cross-domain connections | semantic | concepts bridge domains, words do not | | Specific phrase lookup | keyword | faster, more precise | **Step 4: Description Scan** Use ripgrep to scan {vocabulary.note} descriptions for edge cases: - Does this extend the source {vocabulary.note}? - Does this contradict or create tension? - Does this provide evidence or examples? Flag candidates with a reason (not just "related"). **Step 5: Link Following** From promising candidates, follow their existing links: - What do THEY connect to? - Are there clusters of related {vocabulary.note_plural}? - Do chains emerge that your source {vocabulary.note} should join? This is graph traversal. You are exploring the neighborhood. ### Phase 3: Evaluate Connections For each candidate connection, apply the articulation test. **The Articulation Test:** Complete this sentence: > [[note A]] connects to [[note B]] because [specific reason] If you cannot fill in [specific reason] with something substantive, the connection fails. **Valid Relationship Types:** | Relationship | Signal | Example | |-------------|--------|---------| | extends | adds dimension | "extends [[X]] by adding temporal aspect" | | grounds | provides foundation | "this works because [[Y]] establishes..." | | contradicts | creates tension | "conflicts with [[Z]] because..." | | exemplifies | concrete instance | "demonstrates [[W]] in practice" | | synthesizes | combines insights | "emerges from combining [[A]] and [[B]]" | | enables | unlocks possibility | "makes [[C]] actionable by providing..." | **Reject if:** - The connection is "related" without specifics - You found it through keyword matching alone with no semantic depth - Linking would confuse more than clarify - The relationship is too obvious to be useful **Agent Traversal Check:** Ask: **"If an agent follows this link, what do they gain?"** | Agent Benefit | Keep Link | |---------------|-----------| | Provides reasoning foundation (why something works) | YES | | Offers implementation pattern (how to do it) | YES | | Surfaces tension to consider (trade-off awareness) | YES | | Gives concrete example (grounds abstraction) | YES | | Just "related topic" with no decision value | NO | The vault is built for agent traversal. Every connection should help an agent DECIDE or UNDERSTAND something. Connections that exist only because they feel "interesting" without operational value are noise. **Synthesis Opportunity Detection:** While evaluating connections, watch for synthesis opportunities — two or more {vocabulary.note_plural} that together imply a higher-order claim not yet captured. Signs of a synthesis opportunity: - Two {vocabulary.note_plural} make complementary arguments that combine into something neither says alone - A pattern appears across three or more {vocabulary.note_plural} that has not been named - A tension between two {vocabulary.note_plural} suggests a resolution claim When you detect a synthesis opportunity: 1. Note it in the output report 2. Do NOT create the synthesis {vocabulary.note} during reflect — flag it for future work 3. Describe what the synthesis would argue and which {vocabulary.note_plural} contribute ### Phase 4: Add Inline Connections Connections live in the prose, not just footers. **Inline Links as Prose:** The wiki link IS the argument. The title works as prose when linked. Good patterns: ```markdown Since [[other note]], the question becomes how to structure that memory for retrieval. The insight that [[throughput matters more than accumulation]] suggests curation, not creation, is the real work. This works because [[good systems learn from friction]] — each iteration improves the next. ``` Bad patterns: ```markdown This relates to [[other note]]. See also [[throughput matters more than accumulation]]. As discussed in [[good systems learn from friction]], systems improve. ``` If you catch yourself writing "this relates to" or "see also", STOP. Restructure so the claim does the work. **Where to add links:** 1. Inline in the body where the connection naturally fits the argument 2. In the relevant_notes YAML field with context phrase 3. BOTH when the connection is strong enough **Relevant Notes Format:** ```yaml relevant_notes: - "[[note title]] — extends this by adding the temporal dimension" - "[[another note]] — provides the mechanism this claim depends on" ``` Context phrases use standard relationship vocabulary: extends, grounds, contradicts, exemplifies, synthesizes, enables. **Bidirectional Consideration:** When adding [[A]] to [[B]], ask: should [[B]] also link to [[A]]? Not always. Relationships are not always symmetric: - "extends" often is not bidirectional - "exemplifies" usually goes one direction - "contradicts" is often bidirectional - "synthesizes" might reference both sources Add the reverse link only if following that path would be useful for agent traversal. **Reweave Task Filtering (when adding bidirectional links):** When you edit an older {vocabulary.note} to add a reverse link, you MAY flag it for full reconsideration via reweave. But SKIP reweave flagging if ANY of these apply: | Skip Condition | Rationale | |----------------|-----------| | Note has >5 incoming links | Already a hub — one more link does not warrant full reconsideration | | Note has `type: tension` in YAML | Structural framework, not content that evolves | | Note was reweaved in current batch | Do not re-reweave what was just reweaved | | Note is a {vocabulary.topic_map} | {vocabulary.topic_map_plural} are navigation, not claims to reconsider | **Check incoming links:** ```bash grep -r '\[\[note name\]\]' {vocabulary.notes}/*.md | wc -l ``` If >= 5, skip reweave flagging. ### Phase 5: Update {vocabulary.topic_map_plural} {vocabulary.topic_map_plural} are synthesis hubs, not just indexes. **When to update a {vocabulary.topic_map}:** - New {vocabulary.note} belongs in Core Ideas - New tension discovered - Gap has been filled - Synthesis insight emerged - Navigation path worth documenting **{vocabulary.topic_map} Size Check:** After updating Core Ideas, count the links: ```bash grep -c '^\- \[\[' "{vocabulary.notes}/[moc-name].md" ``` If approaching the split threshold (configurable, default ~40): note in output "{vocabulary.topic_map} approaching split threshold (N links)"