| name | librarians-assistant |
| description | Wiki remediation agent — fixes broken links, orphans, non-reciprocal wikilinks, frontmatter, and tags identified by the Wiki Librarian audit. Runs after librarian at 08:50 AM daily. |
| trigger | /librarians-assistant |
Librarians Assistant
Loads: wiki/scratchpad/agent-sheets/librarians-assistant.md for full directives
MCP tools reference: wiki/scratchpad/jobs/mcp-tools-reference.md — all 22 tools documented with args, behaviors, and known issues
Wiki root: /home/ty/Documents/LLM-WIKI
Runs after: Wiki Librarian (48a3a009a820) — reads its carryover for context
Bootstrap
- Read this skill file (you just did)
- Read your agent sheet:
wiki/scratchpad/agent-sheets/librarians-assistant.md
- Read the librarian's carryover:
wiki/scratchpad/jobs/reports/librarian/carryover.md
- Read the batch progress:
wiki/scratchpad/jobs/reports/librarian/batch-progress.md (if it exists)
- Verify MCP availability before using your tools — TWO steps:
/home/ty/Repositories/ai_workspace/project-synapse-mcp/.venv/bin/python3 -c "from synapse_mcp.zettelkasten.insight_engine import InsightEngine; print('OK')" 2>/dev/null && echo "PACKAGE OK" || echo "PACKAGE MISSING"
This confirms `synapse_mcp` is in the project-synapse-mcp venv — but `InsightEngine` import does NOT guarantee `wiki_lint`, `wiki_cluster_pages`, etc. are registered MCP handlers.
```bash
# Step 2: Make an actual MCP call to verify tools are registered
# Try: debug_test, wiki_lint, or wiki_list_pages
If Step 1 succeeds but Step 2 fails (tool not found), MCP server is running but the tool wasn't registered. Use filesystem fallback.
If MCP is unavailable: Use direct filesystem fixes instead of MCP tools (see fallback below).
6. Execute per the agent sheet
Your Tools (all via project-synapse MCP)
wiki_lint() — detect remaining broken links, orphans, missing frontmatterwiki_read_page — read a page to fix itwiki_write_page — fix frontmatter, add wikilinks, normalize tagswiki_search — find related pages for orphan linkingwiki_cluster_pages() — find same-cluster pages for cross-linking
MCP Unavailable — Filesystem Fallback
When the MCP availability probe (python3 -c "from synapse_mcp.zettelkasten.insight_engine import InsightEngine; print('OK')") returns UNAVAILABLE, use direct filesystem operations instead:
- Broken wikilinks: grep for
\[\[ patterns, resolve via filesystem read/write
- Orphan pages: analyze wikilink graph with Python script at
wiki/scratchpad/full_audit.py
- Frontmatter: use
grep -l "^---" wiki/**/*.md | xargs python3 -c "..." for bulk operations
- Reciprocal link gaps: compute wikilink graph, find asymmetric edges, add return links
The wiki/scratchpad/full_audit.py script is the primary audit tool in MCP-unavailable environments — run it first to get current state.
Key Paths
Wiki: /home/ty/Documents/LLM-WIKI/
Agent sheet: wiki/scratchpad/agent-sheets/librarians-assistant.md
Librarian carryover: wiki/scratchpad/jobs/reports/librarian/carryover.md
Batch progress: wiki/scratchpad/jobs/reports/librarian/batch-progress.md
Assistant carryover: wiki/scratchpad/jobs/reports/librarians-assistant/carryover.md
Workflow
- Read carryover from last librarian run — understand what was already fixed and what remains
- Read batch-progress.md if it exists — know where previous assistant run stopped
- Always verify MCP availability first (see Bootstrap step 5)
- Run fixes in priority order (from carryover's "Next session focus")
- Write progress every 15-20 fixes to batch-progress.md
- When done (50+ fixes or hard blocker): update assistant carryover with open items
Known Pitfalls
CRITICAL: All cron jobs need full enabled_toolsets
If any tool is missing from enabled_toolsets, the call silently fails and the job shows last_status: ok while the work was skipped. See references/cron-toolsets.md for the complete toolset table, error → missing-tool mapping, and job ID reference.
CRITICAL: execute_code is NOT available to cron agents
The "Primary Frontmatter Fix Workflow" section shows terminal heredoc as the fallback — this pattern is unreliable in cron context (triggers pending_approval). Preferred fallback order:
wiki_write_page (MCP) — best for wiki pages, bypasses filesystemwrite_file (toolset) — direct overwrite, reliable for targeted fixes- Terminal heredoc — only if both above are unavailable; risks
pending_approval
When using write_file to fix frontmatter, read the file first with read_file, then write corrected content with write_file. Prefer wiki_write_page (MCP) over filesystem writes wherever possible.
generate_insights CLI — use correct pattern
The Zettelkasten engine times out at 300s via MCP. For CLI insight generation, use:
timeout --kill-after=10s 580s uv run python /home/ty/Repositories/ai_workspace/project-synapse-mcp/scripts/generate_insights.py --topic general --print --max-runtime 540
Exit codes: 0=success, 3=timeout (file may be valid), 1=init failure. Do NOT use ~/.hermes/scripts/generate_insights.sh — that script does not exist.
concept-index.md is auto-generated — its broken links should NOT be manually fixed; fix the generation pipeline insteadlog.md links are structural false positives — they contain dynamically generated wikilink references, ignore them in broken link audits- Compound-tag noise in news sources:
[['news', 'geopolitics', ...]] style tag arrays are noise, not real wikilinks — strip them, don't create links for them (18+ instances)
- Insight generation is unreliable:
generate_insights() times out after 300s; if it fails, skip and note in carryover — do NOT retry the same cycle
- Orphan counts differ by method: MCP
wiki_lint() reports ~155 orphans (Neo4j graph), while full_audit.py filesystem scan may report 0 — these are not comparable; use MCP results when available
- Top authorities need depth, not just links: efhf, maximum-occupancy-principle, project-synapse, edm-framework are load-bearing pages — when linking to them, add substantive content, not just wikilinks
Broken link triage — filesystem pre-scan beats full_audit.py for content focus
full_audit.py reports broken links in ALL files including scratchpad/report noise. To focus on real wiki content:
cd /home/ty/Documents/LLM-WIKI && python3 -c "
import re, os
# Only scan high-value content dirs (not scratchpad)
dirs = ['wiki/concepts', 'wiki/entities', 'wiki/synthesis', 'wiki/sources']
all_md = {}
for root, _, files in os.walk('wiki'):
for f in files:
if f.endswith('.md'):
all_md[f[:-3].lower()] = os.path.join(root, f)
broken = []
for d in dirs:
if not os.path.exists(d): continue
for root, _, files in os.walk(d):
for f in files:
if not f.endswith('.md'): continue
path = os.path.join(root, f)
with open(path) as fh: content = fh.read()
links = re.findall(r'\[\[([^\]|]+)', content)
for link in links:
slug = link.strip().lower().replace(' ', '-').replace('/', '-')
if slug not in all_md:
broken.append((path, link))
# Group by link target
from collections import Counter
counter = Counter([link for _, link in broken])
for link, count in counter.most_common(20):
print(f' {link}: {count} references')
"
wiki/sources/ always has the most broken link targets (131 at 2026-06-08) — these are entity references from Zettelkasten insight generation, not content gaps. wiki/concepts/ and wiki/entities/ should always have 0 broken links if prior cycles ran correctly.
Template wikilink examples vs. real broken links
Some pages contain intentional template syntax in their documentation/body. These appear as broken links but are NOT real issues:
synapse-llm-wiki-operating-guide.md contains [[page-slug]], [[slug]], [[Display]] — these are template examples in the Wikilink Rules section showing correct wikilink syntax. Do NOT create stubs for these.
To detect this case: if a wikilink target looks like a generic placeholder (contains "slug", "page-", "display") and the containing file is a documentation/operating-guide page, verify the context before creating a stub. The correct scan already excludes code blocks, but these template examples appear as inline prose examples, not in code blocks.
Insight pages have a specific dual-frontmatter pattern
The Zettelkasten engine produces insight pages with TWO frontmatter blocks concatenated:
- A minimal auto-generated block (created, updated, summary, tags)
- A full block (created, updated, type, summary, tags, sources, status, confidence)
After running the duplicate-frontmatter cleaner, verify the result — extra --- delimiters can remain between blocks, leaving the file still malformed. Always do a head -10 check after cleaning insight pages. If double --- delimiters appear, use write_file with the fully merged content.
write_file safety
write_file OVERWRITES the entire file. Use it only when:
- You have freshly read the full file (not offset-limited)
- You are constructing the content from scratch (e.g., merging duplicate blocks)
- You have verified the current state before overwriting
Do NOT use write_file as a substitute for patch when fixing small targeted issues — patch is safer for surgical edits.
Current cleaner limitation
The duplicate frontmatter cleaner (fix_frontmatter_missing_closing) removes the duplicate field lines but CAN leave extra --- blank---- sequences between blocks, requiring a second pass or manual write_file. The cleaner function detects body start by first # heading, but doesn't handle the case where the "cleaned" output still has two --- delimiters with blank lines between them. Workaround: after cleaner, check grep -c '^---' file.md. If count > 2, use write_file to manually merge.
- P0: elevate source → concept when wikilinks outnumber content: If a wikilink target only exists as a
wiki/sources/articles/<slug>.md but is referenced 20+ times from wiki/concepts/ and wiki/entities/, create a full wiki/concepts/<slug>.md concept page rather than a stub. Use the source article content as the basis; add connections to related concepts/entities.
- engineering-internal-awareness wikilink: Already resolved this session —
wiki/concepts/engineering-internal-awareness.md created (2026-05-30) to handle all 19 references in log.md + scratchpad files. Previous guidance to "ignore" is now replaced with actual fix.
Reference Files — USE FULL PATHS
Path rule: Reference files are in the skill directory, NOT relative to workdir. Always use absolute paths.
| File | Full path |
|---|
fix-patterns.md | /home/ty/.hermes/skills/agent-sheets/librarians-assistant/references/fix-patterns.md |
stub-fix-log.md | /home/ty/.hermes/skills/agent-sheets/librarians-assistant/references/stub-fix-log.md |
Never use relative paths like references/fix-patterns.md — the agent's workdir is /home/ty/Documents/LLM-WIKI, so relative paths resolve to the wrong location.
If a file read fails, search for it first:
find /home/ty/.hermes/skills/agent-sheets/librarians-assistant/ -name "*.md" 2>/dev/null
Known Stub List
Known Stub List (verified 2026-05-30 — do not recreate)
wolfram-physics-project, aseke-framework, extraction-quality-audit, catastrophic-forgetting, in-context-learning, emergence, agentic-oversight, institutional-capture, geopolitics, evaluation, agent-onboarding, scaling-laws, titans, reasoning — all verified existing.
Complex Synthesis Pages — Do Not Auto-Clean (Accurate Description)
These 8 pages had duplicate field lines within a single frontmatter block (e.g., created, updated, type, summary each appear twice), and no closing --- delimiter — frontmatter ran directly into body content. The skill's bulk cleaner code was wrong about the block structure.
Status (2026-06-08): ALL 8 FIXED.
cross-layer-drift-falsification.md — FIXEDcodegraph-hermes-integration-plan.md — FIXEDlibrarian-report-2026-05-09.md — FIXEDself-prompting-via-production-stage-architecture.md — FIXEDessan-internal-representation.md — FIXEDwiki-indexing-theory.md — FIXEDresearch-brief-2026-05-09.md — had no frontmatter at all — FIXED (added from scratch)research-brief-2026-05-13.md — had no frontmatter at all — FIXED (added from scratch)
Additional insight pages fixed (2026-06-08 session):
wiki/synthesis/insights/para-knowledge-architecture-cohesion-insight.md — dual block mergedwiki/synthesis/insights/francesca-albanese-sanctions-case-insight.md — dual block merged
The Zettelkasten engine produces insight pages with a specific dual-frontmatter pattern. See "Insight pages have a specific dual-frontmatter pattern" pitfall above for cleanup verification steps.
Correct Bulk Duplicate Frontmatter Cleaner
The split('\n---\n') approach FAILS for files missing the closing ---. Use this instead:
import re
def fix_frontmatter_missing_closing(path):
"""Fix frontmatter with duplicate fields and NO closing '---' delimiter."""
with open(path, 'r') as f:
lines = f.read().split('\n')
if lines[0] != '---':
return False, "no opening ---"
fm_end = -1
for i in range(1, len(lines)):
if lines[i].startswith('# ') or (lines[i].strip() and not re.match(r'^[\w-]+:', lines[i])):
fm_end = i - 1
break
while fm_end >= 0 and lines[fm_end].strip() == '':
fm_end -= 1
fm_lines = lines[0:fm_end+1]
body_lines = lines[fm_end+1:]
fields = {}
for line in fm_lines[1:]:
m = re.match(r'^(\w[\w-]*):(.*)', line)
if m:
fields[m.group(1)] = line
order = ['created', 'updated', 'type', 'summary', 'tags', 'sources', 'status', 'confidence']
new_fm_lines = ['---']
for key in order:
if key in fields:
new_fm_lines.append(fields[key])
new_fm_lines.append('---')
with open(path, 'w') as f:
f.write('\n'.join(new_fm_lines) + '\n' + '\n'.join(body_lines))
return True, f"cleaned, kept {len(fields)} fields"
Also handles: files that have frontmatter but no closing --- (the frontmatter just runs into body). Detect by: if content.startswith('---') and '---' not in content[3:content.find('\n# ')].
Known Broken Wikilink Targets (need stub creation)
mesa-optimization — referenced in wiki/concepts/reward-hacking.md but page does not exist. Stub created (2026-06-08) with note "mesa-optimization (stub — page not yet created)"; a full stub page should be created if this link is referenced elsewhere.
PDF Path Contamination — Detection and Cleanup
Problem: Other agents (arxiv-agent) may write PDFs to the wrong directory when workdir changes. Files land in /home/ty/Documents/LLM-WIKI/ instead of /home/ty/Documents/paper-research/. This causes the wiki folder to grow with binary files that shouldn't be there.
Detection:
find /home/ty/Documents/LLM-WIKI -name "*.pdf" -type f 2>/dev/null
If any results appear, PDFs have migrated to the wrong location.
Cleanup (move to correct location):
mv /home/ty/Documents/LLM-WIKI/2605.22781v1.pdf /home/ty/Documents/paper-research/
mv /home/ty/Documents/LLM-WIKI/2605.22786v1.pdf /home/ty/Documents/paper-research/
find /home/ty/Documents/LLM-WIKI -name "*.pdf" -exec mv {} /home/ty/Documents/paper-research/ \; 2>/dev/null
Prevention (for arxiv-agent): Always verify after download: ls -la /home/ty/Documents/paper-research/{id}.pdf confirms the file landed in the right place, not the current workdir. See references/obsidian-vault.md for the full post-mortem.
Reference Files
references/obsidian-vault.md — Obsidian vs wiki separation, PDF path contamination post-mortemreferences/stub-fix-log.md — verified stubs from prior sessions (do not recreate)references/fix-patterns.md — UNVERIFIED — this file was listed but may not exist on disk; verify before loading
Wikilink Normalization Pattern
2026-05-31 (this session)
- 54 frontmatter completions (40 concept + 4 people + 14 project/entity pages) using batch Python approach
- Verified efhf ↔ maximum-occupancy-principle already reciprocal (skip creation)
- Verified markovian-dev-agency → hermes-agent and hermes_agent → load-bearing-reasoning already have return links
- MCP: UNAVAILABLE (confirmed — norm for this environment)
2026-06-08 (assistant run after librarian)
- 9 fixes: 1 broken wikilink + 8 frontmatter fixes (6 pages missing closing
---, 2 pages missing frontmatter entirely)
- MCP: OK (project-synapse-mcp venv confirmed working)
- Wiki content confirmed clean — broken links (165) ALL in scratchpad/report files; orphans (158) ALL in scratchpad/system files
- Key discovery:
maximum-occupancy-principle.md actually EXISTS at wiki/concepts/maximum-occupancy-principle.md — prior sessions listed it as non-existent incorrectly
2026-05-30 (prior session)
- 4 wikilinks normalized (llm-wiki-pattern.md, meta-harness.md)
- 31 duplicate frontmatter pages cleaned (16 concept/entity + 11 synthesis + 4 type:missing)
- 3 alias stubs verified clean: reasoning.md, llama-nas.md, rz-nas.md
- 8 complex synthesis pages with 26-34 blocks identified — do NOT auto-clean, need individual review
MCP Status — Correct Understanding
MCP probe — TWO steps, not one:
Step 1 (package import — necessary but NOT sufficient):
/home/ty/Repositories/ai_workspace/project-synapse-mcp/.venv/bin/python3 -c "from synapse_mcp.zettelkasten.insight_engine import InsightEngine; print('OK')" 2>/dev/null && echo "PACKAGE OK" || echo "PACKAGE MISSING"
This confirms the synapse_mcp package is in the project-synapse-mcp venv — NOT the hermes-agent venv.
Step 2 (tool registration — the actual test):
After the import succeeds, make an actual MCP call: debug_test or wiki_lint. If it returns, tools are registered. A clean InsightEngine import does NOT guarantee wiki_lint, wiki_cluster_pages, wiki_hits_analysis are available — those are registered MCP handlers in the server itself, not standalone Python modules.
If MCP calls succeed: Use the MCP tools freely — wiki_lint() (~5s), wiki_cluster_pages() (~10s), wiki_hits_analysis() (~10s), wiki_write_page (primary wiki write channel).
If MCP is down (import fails): Fall back to filesystem via python3 wiki/scratchpad/full_audit.py.
IMPORTANT: generate_insights() times out at 300s. Do NOT use the MCP tool in cron context. Use the CLI wrapper:
timeout --kill-after=10s 580s uv run python /home/ty/Repositories/ai_workspace/project-synapse-mcp/scripts/generate_insights.py --topic general --print --max-runtime 540
Exit codes: 0=success, 3=timeout (file may be valid), 1=init failure. Do NOT use ~/.hermes/scripts/generate_insights.sh — that path does not exist.
Primary Frontmatter Fix Workflow (MCP Unavailable)
When filling missing type/sources/status/confidence on 10+ pages, use batch Python via terminal heredoc — NOT execute_code (which is not available to cron agents):
cd /home/ty/Documents/LLM-WIKI && python3 << 'PYEOF'
import re, os
pages = [
("wiki/concepts/page1.md", ["type", "sources", "status", "confidence"]),
("wiki/concepts/page2.md", ["sources", "status"]),
]
for path, missing in pages:
if not os.path.exists(path):
print(f"SKIP: {path} does not exist")
continue
with open(path) as f:
content = f.read()
fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
if not fm_match:
print(f"NO FM: {path}")
continue
fm_text = fm_match.group(1)
body = content[fm_match.end():]
updates = {"sources": "[]", "status": "active", "confidence": "0.8"}
for key, val in updates.items():
if key not in fm_text and key in missing:
fm_text += f"\n{key}: {val}"
with open(path, "w") as f:
f.write(f"---\n{fm_text}\n---{body}")
print(f"OK: {os.path.basename(path)}")
print("Done.")
PYEOF
Batch in groups of 10-15. Verify sample files before continuing.
Reciprocal Link Pre-Verification
Before creating reciprocal links, check if they already exist:
grep -l "\[\[target-page\]\]" wiki/concepts/*.md wiki/entities/*.md wiki/synthesis/*.md 2>/dev/null
Common pairs that are ALREADY reciprocal (skip creation):
efhf.md ↔ maximum-occupancy-principle.md — efhf.md sources field + MOP connections sectionmarkovian-dev-agency.md → hermes-agent.md — already linked in both directionshermes_agent.md → load-bearing-reasoning.md — reasoning.md has return link
Delivery Rule
- Delivery: origin (Discord thread)
- Report: "Fixed N broken links, resolved N orphans, normalized N pages, added N wikilinks"
- If nothing left to do:
[Librarians-assistant — no open items]
- Do NOT write a report file — output goes to cron delivery
STEP 7 — Kanban Review (Self-Answer Open Questions)
After writing your carryover, load the kanban-morning-review skill:
- Read your carryover at
wiki/scratchpad/jobs/reports/librarians-assistant/carryover.md
- For each item in the What Remains section, attempt to answer it from available context:
- Search wiki via
search_files or mcp_project_synapse_wiki_search for existing content on the topic
- Check other carryovers for related filled concepts
- Use
mcp_project_synapse_synapse_recall for relevant episodic facts
- If the answer exists → write it directly to the carryover under "Resolved this cycle" (do NOT create a kanban task)
- If genuinely unanswerable → use
hermes kanban add to create the task (the skill handles this)
- Patch carryover to remove any items that are now resolved
