qmd-search-maintenance

name	qmd-search-maintenance
description	Maintain QMD (Quick Markdown Search) index, generate vector embeddings, and troubleshoot platform-specific issues on macOS Apple Silicon. Covers external API pipelines (Xunfei, OpenAI) and local fallbacks (sentence-transformers, node-llama-cpp).
category	wiki

QMD Search Maintenance

Maintain and troubleshoot QMD's vector search index, including embedding generation and platform-specific issues.

Quick Reference

qmd status                    # View index health
qmd search "<query>" -c wiki-core   # BM25 full-text search
qmdx query "<query>" -c wiki-core --no-rerank  # Hybrid search (Xunfei patch)
qmd embed -c wiki-core              # Generate embeddings (local model — broken on macOS M5)

A concise quick-start guide for daily use is at ~/wiki/GETTING_STARTED.md — three search modes, flag reference, performance table.

Important: There are TWO qmd installations on this system. Only the Volta one has the patch.

Installation	Version	Path	Patched?
Volta (via npm)	v2.5.3	~/.volta/bin/qmd	✅ llm.js patched
Homebrew	v0.9.0	/opt/homebrew/bin/qmd	❌ No patch, do not use

Always use ~/.volta/bin/qmd for any command that needs embedding. The qmdx alias already points there.

# Alias in ~/.zshrc:
alias qmdx='QMD_XUNFEI_EMBED=true ~/.volta/bin/qmd'

# Or wrapper script: ~/.hermes/scripts/qmd-xunfei

Problem: node-llama-cpp Fails on macOS 26.5.2 / Apple M5

Symptom: qmd embed hangs on [node-llama-cpp] ggml_metal_library_init_from_source: error compiling source. Root cause: node-llama-cpp v3.18.1 cannot compile Metal shaders at runtime on this macOS version. Even with Xcode + Metal Toolchain installed, Metal compilation hangs. CPU fallback: Setting QMD_FORCE_CPU=1 or --no-gpu does NOT fix it — node-llama-cpp still attempts Metal shader compilation during backend init, and the CPU-only prebuilt binary is unavailable for this platform.

Solution: External Embedding API Pipeline

Bypass node-llama-cpp entirely using a two-phase pipeline.

QMD embedding fingerprint (in DB)

After inserting vectors, update model + fingerprint to match the configured embed model:

-- Get the correct fingerprint
cd ~/wiki && node --input-type=module -e "
import { getEmbeddingFingerprint, DEFAULT_EMBED_MODEL } from '/Users/jinguo/.volta/tools/image/packages/@tobilu/qmd/lib/node_modules/@tobilu/qmd/dist/store.js';
console.log(getEmbeddingFingerprint('hf:Qwen/Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf'));
"

-- Update all vectors
UPDATE content_vectors SET model = 'hf:Qwen/Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf',
  embed_fingerprint = '<fingerprint>',
  total_chunks = 1;

Pitfall: qmd-embed-phase2.js writes total_chunks = <global total> (e.g. 13086) instead of 1 per doc. This makes QMD's getHashesNeedingEmbedding report every doc as pending because chunk_count (1) < expected_chunks (13086). Fix: UPDATE content_vectors SET total_chunks = 1.

Query-time Xunfei Patch (for qmd query)

The most impactful fix: patch QMD's llm.js to intercept embed() and embedBatch() so that query-time embedding uses the Xunfei API instead of local llama.cpp. This makes qmd query fully functional.

What was patched

File: ~/.volta/tools/image/packages/@tobilu/qmd/lib/node_modules/@tobilu/qmd/dist/llm.js

Added two new methods to the LlmCpp class:

• xunfeiEmbed(text) — single text embedding via Xunfei API
• xunfeiEmbedBatch(texts) — batch embedding (batch size 20, sequential, returns {embedding: number[], model: string})

Modified embed() and embedBatch() to check process.env.QMD_XUNFEI_EMBED at the top — if set, delegate to Xunfei methods instead of loading llama.cpp.

How to apply the patch

The patch was applied directly to dist/llm.js. The exact changes:

1. Before embed(text, options) — add guard:

async embed(text, options = {}) {
    if (process.env.QMD_XUNFEI_EMBED)
        return this.xunfeiEmbed(text);
    // ... original code ...

2. Before embedBatch(texts, options) — add guard:

async embedBatch(texts, options = {}) {
    if (this._ciMode)
        throw new Error("LLM operations are disabled in CI (set CI=true)");
    if (process.env.QMD_XUNFEI_EMBED)
        return this.xunfeiEmbedBatch(texts);
    // ... original code ...

3. xunfeiEmbed method — uses Node.js https module (dynamic import("https") in ESM):

async xunfeiEmbed(text) {
    const https = await import("https");
    const token = process.env.XUNFEI_EMBEDDING_TOKEN;
    if (!token) return null;
    const payload = JSON.stringify({model:"xop3qwen8bembedding", input:[text], dimensions:1024});
    return new Promise((resolve) => {
        https.request({hostname:"maas-api.cn-huabei-1.xf-yun.com",path:"/v2/embeddings",
            method:"POST", headers:{"Authorization":`Bearer ${token}`, "Content-Type":"application/json",
            "Content-Length":Buffer.byteLength(payload)}, timeout:60000},
        (res) => { /* parse response */ });
    });
}

Usage

# One-off (use full path — do NOT use Homebrew's /opt/homebrew/bin/qmd)
QMD_XUNFEI_EMBED=true ~/.volta/bin/qmd query "your question" -c wiki-core --no-rerank

# Using the zsh alias (from ~/.zshrc):
qmdx query "your question" -c wiki-core --no-rerank

# Using wrapper script:
~/.hermes/scripts/qmd-xunfei query "your question" -c wiki-core --no-rerank

The wrapper script at ~/.hermes/scripts/qmd-xunfei:

#!/bin/bash
export QMD_XUNFEI_EMBED=true
exec ~/.volta/bin/qmd "$@"

Pitfall: Two qmd installations: The PATH has ~/.volta/bin before /opt/homebrew/bin, but which qmd may resolve to the wrong one depending on shell context. Always use ~/.volta/bin/qmd explicitly in scripts and the alias. The Homebrew install (v0.9.0) is unpatched and qmd query will fail with "Dimension mismatch" (vectors_vec expects 1024d but Homebrew's embed returns 768d from a different model path).

Pitfall: --no-rerank is the correct flag, not --skip-rerank: QMD v2.5.3's CLI defines --no-rerank (yargs boolean). Using --skip-rerank is silently ignored — the flag is undefined, so reranking runs with llama.cpp (28-35s). The output still shows "Reranking N chunks... (1ms)" if llama.cpp was recently loaded, but the 1ms is misleading (model cached, not skipped). --no-rerank avoids the "Reranking" log line entirely and uses RRF-only (~1ms). If you see "Reranking N chunks..." in the output, you did NOT pass --no-rerank correctly.

RRF vs LLM Reranker

Aspect	RRF (--no-rerank)	LLM Reranker (default)
What it does	Reciprocal Rank Fusion — score += weight / (60 + rank) per result list, top-1 bonus +0.05	Cross-encoder: Qwen3-Reranker-0.6B(query + chunk) → relevance score, blended with RRF position
Model loading	None (pure math)	node-llama-cpp → 35s with Metal error recovery
Quality	Top-1 relevance 93-100%	Slight improvement on borderline cases
Speed	~1ms	~35s
Verdict	Recommended for daily use	Only for edge cases

The difference is small for strong matches. RRF-fused BM25 + vector search already catches the most relevant results. The reranker mainly rescues borderline candidates where FTS and vector search disagree.

Performance

Step	Before (local llama.cpp)	After (Xunfei API)
Query expansion	0ms ✅	0ms ✅
Embedding 3 queries	Hang (Metal compile)	1ms 🚀
Vector search	N/A	≤1ms
Reranking	25.9s (Metal error)	≤1ms (--no-rerank) or same

Caveats

• Reranking still uses llama.cpp: The reranker model (qwen3-reranker-0.6b) still goes through node-llama-cpp. Use --no-rerank to bypass (results are RRF-fused from BM25 + vector search, still high quality).
• qmd vsearch not fixed: The vector search command runs query expansion first (same Metal init path). The same patch technique would work, but was not applied.
• Patching compiled JS is fragile: Any npm update or bun install that rebuilds QMD will overwrite the patch. Re-apply with bash ~/.hermes/scripts/qmd-patch-xunfei.sh (or bash <(cat ~/.hermes/skills/wiki/qmd-search-maintenance/scripts/qmd-patch-xunfei.sh) for the hermetic copy under this skill).
• ESM vs CJS: llm.js is ESM, so require("https") fails. Must use await import("https").

import requests
API_KEY = os.environ["XUNFEI_EMBEDDING_TOKEN"]  # APPID:APISECRET
headers = {"Authorization": f"Bearer {API_KEY}"}
payload = {
    "model": "xop3qwen8bembedding",
    "input": ["text1", "text2"],
    "dimensions": 1024
}
resp = requests.post("https://maas-api.cn-huabei-1.xf-yun.com/v2/embeddings",
                     headers=headers, json=payload, timeout=60)
embeddings = [x["embedding"] for x in sorted(resp.json()["data"], key=lambda x: x["index"])]

Key details:

• URL: https://maas-api.cn-huabei-1.xf-yun.com/v2/embeddings
• Auth: Authorization: Bearer <full_key> where key = APPID:APISECRET
• Model: xop3qwen8bembedding
• Dimension: 1024
• Rate limit: 20 QPS
• Request body: {"model": "...", "input": [...], "dimensions": 1024}

Script location: ~/.hermes/scripts/qmd-embed-xunfei.py

• Concurrent (ThreadPoolExecutor, 10 workers)
• Batch size 20
• Saves to /tmp/qmd-embed-xunfei.json

Alternative: sentence-transformers (Python, local):

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("Qwen/Qwen3-Embedding-0.6B", device="mps")  # or "cpu"
embs = model.encode(texts, normalize_embeddings=True)

• Installed: intfloat/multilingual-e5-small, sentence-transformers/all-MiniLM-L6-v2
• PyTorch MPS works on M5 for most models (Qwen3-Embedding-0.6B may OOM with large batches)

Phase 2: Insert into QMD Database

Script location: ~/.hermes/scripts/qmd-embed-phase2.js

Requires running from QMD's node_modules context:

NODE_PATH=/Users/jinguo/.volta/tools/image/packages/@tobilu/qmd/lib/node_modules/@tobilu/qmd/node_modules \
  node ~/.hermes/scripts/qmd-embed-phase2.js /tmp/qmd-embed-data.json

What it does:

1. Opens QMD's SQLite DB at ~/.cache/qmd/index.sqlite
2. Loads sqlite-vec-darwin-arm64/vec0.dylib
3. Creates/validates vectors_vec virtual table with correct dimension
4. Inserts into content_vectors (metadata) + vectors_vec (vector data)

Database schema:

• content_vectors(hash, seq, pos, model, embed_fingerprint, total_chunks, embedded_at) — metadata
• vectors_vec(hash_seq, embedding) — sqlite-vec virtual table (FLOAT[N] distance_metric=cosine)

Quick Status (current macOS 26.5.2 / M5)

Command	Status	Notes
qmd search	✅ Works	BM25 full-text, no model needed
qmd query	✅ Works with Xunfei patch	QMD_XUNFEI_EMBED=true qmd query "..." --no-rerank
qmd vsearch	❌ Hangs	Same root cause — needs the same patch as query
qmd embed	⚠️ Works with DEVELOPER_DIR	Requires DEVLOPER_DIR + Xcode Metal Toolchain; default model fails 98% of docs on CPU
qmd mcp --http	⚠️ Partial	GET /health ✅, POST /search ✅, POST /query ❌, POST /mcp ❓
qmd status	✅ Fixed	After DB fingerprint fix, shows "Tip: N docs need embeddings"
Xunfei API + sqlite-vec	✅ Working	Two-step pipeline, ~15 vec/s, 11ms search

Root cause: node-llama-cpp v3.18.1 on macOS 26.5.2 / Apple M5 hangs during ggml_metal_library_init_from_source. The CPU-only prebuilt binary is unavailable for this platform.

Fix: DEVELOPER_DIR env var (Proven workaround discovered 2026-07-05)

Setting DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer allows node-llama-cpp to find the Xcode Metal Toolchain compiler (at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/metal), bypassing the CLT-only xcode-select path.

DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer qmd embed -c wiki-core --max-docs-per-batch 50 --no-gpu

Requirements:

• Xcode.app installed (not just CLT)
• Metal Toolchain component: DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer xcodebuild -downloadComponent MetalToolchain
• DEVELOPER_DIR inherited by child processes — set in calling shell or ~/.zshenv

Cannot use sudo xcode-select -s: Blocked by macOS security. DEVELOPER_DIR env var is the correct workaround.

Limitation: Even with DEVELOPER_DIR, the default embeddinggemma-300M-Q8_0.gguf model fails >98% of documents on CPU. Use the Xunfei API pipeline for reliable full-corpus embedding.

CLI Flag Correction: --no-rerank, NOT --skip-rerank

Critical: QMD v2.5.3 defines the flag as --no-rerank (yargs boolean \"no-rerank\": { type: \"boolean\", default: false }). The flag --skip-rerank is undefined and silently ignored.

Your input	What QMD does
--no-rerank	✅ Skips LLM reranker, uses RRF-only (~1ms)
--skip-rerank	❌ Silently ignored — reranker runs (35s)
(no flag)	Default: reranker runs (35s)

How to tell if it worked: with --no-rerank, there is NO "Reranking N chunks..." line in the output. If you see that line, the flag was not recognized.

When to Use Each Embedding Approach

| Approach | Speed | Reliability | Cost | Notes | |---|---|---|---|---|---| | Xunfei API (recommended for initial full embed) | ~15 vec/s (concurrent 10) | ✅ Production | ¥ token plan | API key in .zshenv; needs internet | | sentence-transformers (MPS) | Varies | ⚠️ OOM with large batches | Free | Batch size ≤8, needs PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 | | sentence-transformers (CPU) | ~250 vec/min | ✅ Reliable | Free | Slow but reliable for small wikis |

Xunfei API Details

• URL: https://maas-api.cn-huabei-1.xf-yun.com/v2/embeddings
• Auth: Authorization: Bearer <APPID:APISECRET> (simple Bearer, NOT HMAC)
• Model: xop3qwen8bembedding (1024-dim output)
• Rate limit: 20 QPS, currency 20
• Request format: {"model": "...", "input": ["text"], "dimensions": 1024}
• Response: {"data": [{"embedding": [...], "index": 0}]}
• Env var: XUNFEI_EMBEDDING_TOKEN in ~/.zshenv (format APPID:APISECRET)

Reference script: The wiki-book repo has ~/wiki-book/scripts/build-vectorize-xunfei.py which uses the same API for Cloudflare Vectorize. Use it as a reference for Xunfei API integration patterns (concurrent workers, batch sizing, error handling).

Direct Vector Search (Bypassing QMD Frontend)

qmd vsearch hangs because it initializes node-llama-cpp for query expansion before doing vector lookup. Use sqlite-vec directly in two steps:

Step 1: Generate query embedding

source ~/.zshenv
NODE_PATH=$HOME/.volta/tools/image/packages/@tobilu/qmd/lib/node_modules/@tobilu/qmd/node_modules \
  node -e '
const KEY=process.env.XUNFEI_EMBEDDING_TOKEN,https=require("https"),fs=require("fs");
const BODY=JSON.stringify({model:"xop3qwen8bembedding",input:["YOUR QUERY"],dimensions:1024});
https.request({hostname:"maas-api.cn-huabei-1.xf-yun.com",path:"/v2/embeddings",method:"POST",
  headers:{"Authorization":"Bearer "+KEY,"Content-Type":"application/json","Content-Length":Buffer.byteLength(BODY)},timeout:15000},
r=>{let b="";r.on("data",c=>b+=c);r.on("end",()=>{
const e=JSON.parse(b).data[0].embedding;
const B=Buffer.alloc(4096);for(let i=0;i<1024;i++)B.writeFloatLE(e[i],i*4);
fs.writeFileSync("/tmp/qvec.bin",B);
console.log("Embedding saved");
});}).on("error",e=>console.error("Error:",e.message)).end(BODY);
'

Step 2: Search using saved embedding

NODE_PATH=$HOME/.volta/tools/image/packages/@tobilu/qmd/lib/node_modules/@tobilu/qmd/node_modules \
  node -e '
const path=require("path"),fs=require("fs");
const V=path.join(process.env.HOME,".volta/tools/image/packages/@tobilu/qmd/lib/node_modules/@tobilu/qmd/node_modules/sqlite-vec-darwin-arm64/vec0.dylib");
const B=fs.readFileSync("/tmp/qvec.bin");
const Database=require("better-sqlite3"),db=new Database(process.env.HOME+"/.cache/qmd/index.sqlite");
db.loadExtension(V);
const t=Date.now();
const rows=db.prepare("SELECT v.hash_seq,v.distance,d.path,d.title FROM (SELECT hash_seq,distance FROM vectors_vec WHERE embedding MATCH ? AND k = 5)v JOIN documents d ON d.hash=substr(v.hash_seq,1,64) AND d.active=1 ORDER BY v.distance ASC").all(B);
db.close();console.log("Search:",Date.now()-t+"ms\n");
rows.forEach((r,i)=>{const s=((1-r.distance)*100).toFixed(1);const p=r.path.replace("/Users/jinguo/wiki/","").replace(".md","");console.log((i+1)+". ["+s+"%] "+r.title);});
'

Why separate steps?: Node.js native addons (better-sqlite3) can hang when loaded inside an https callback from a .js file. The two-step pattern avoids this by isolating the embedding generation (HTTP) from the database query (native addon).

Shell quoting pitfall: Inside node -e '...' (single-quoted), single quotes inside JavaScript terminate the shell string. Avoid using single quotes in SQL; skip the d.collection='wiki-core' filter (all docs are wiki-core), or use String.fromCharCode(39) to build the string at runtime.

Scripts

Phase 1: Generate Embeddings (Xunfei API)

• scripts/qmd-embed-xunfei.py — Python, concurrent (10 workers), batch 20, saves to /tmp/qmd-embed-xunfei.json
• API key: $XUNFEI_EMBEDDING_TOKEN (format APPID:APISECRET, set in ~/.zshenv)

Phase 2: Insert into QMD Database

• scripts/qmd-embed-phase2.js — Node.js, reads Phase 1 JSON, writes to content_vectors + vectors_vec
• Run from QMD's node_modules context:

  NODE_PATH=$HOME/.volta/tools/image/packages/@tobilu/qmd/lib/node_modules/@tobilu/qmd/node_modules \
    node scripts/qmd-embed-phase2.js /tmp/qmd-embed-xunfei.json
  

Combined Search (Xunfei + sqlite-vec)

• scripts/qmd-vector-search.py — Python wrapper: calls Xunfei API → spawns Node.js sqlite-vec query
• Usage: source ~/.zshenv && python3 scripts/qmd-vector-search.py '<query>' [limit]

Same scripts also deployed to ~/.hermes/scripts/ for direct use:\n- ~/.hermes/scripts/qmd-embed-xunfei.py\n- ~/.hermes/scripts/qmd-embed-phase2.js\n- ~/.hermes/scripts/qmd-vector-search.py — Python wrapper combining Phase 1 + Phase 2 into one search command. Usage: source ~/.zshenv && python3 ~/.hermes/scripts/qmd-vector-search.py '<query>' [limit]

QMD MCP Server

QMD exposes an MCP server for AI agent integration. Two transport modes:

HTTP Mode (for custom integration)

qmd mcp --http --port 8006

Endpoints (verified working):

• GET /health — returns {"status":"ok","uptime":N}
• POST /search — BM25 search: {"searches":[{"query":"...","type":"lex","limit":5}]}
• POST /query — hybrid search (❌ hangs on macOS 26.5.2/M5 — same Metal bug)
• POST /mcp — MCP JSON-RPC via StreamableHTTP transport (requires session header)

The HTTP server uses Hono internally with @modelcontextprotocol/sdk. Default port 8005 (wiki-book deploy) or 8006 (local).

Stdio Mode (for agent MCP client integration)

qmd mcp
# Agent connects via stdio (Claude Code, Cursor, Hermes MCP client, etc.)

QMD registers tools: search, query, get, status. Configure as Hermes MCP server:

mcp:
  servers:
    qmd:
      command: qmd
      args: [mcp]

Note: On macOS 26.5.2/M5, only search (BM25) works reliably. query hangs due to Metal bug. For vector search, use the sqlite-vec direct query approach instead.

Archived Sub-Skills

The following standalone skills have been consolidated into this umbrella:

• qmd-embed-xunfei — Xunfei embedding API patch for QMD's llm.js. Content absorbed into the "Query-time Xunfei Patch" section above. Reference copy at references/qmd-embed-xunfei.md.
• qmd-embed-mps-workaround — Deprecated PyTorch MPS approach for embedding generation. All content (two-phase pipeline, batch size pitfall, scripts) has been absorbed above. The standalone SKILL.md is preserved at references/qmd-embed-mps-workaround.md.

Known Issues

• qmd vsearch still tries to init node-llama-cpp even with vectors in DB (Metal hang persists for vector search too — not yet patched like query is).
• qmd status after DB fix: shows "Tip: N docs need embeddings" where N is small (e.g. 15). This is accurate — only those documents genuinely lack vectors. If it shows a huge number (e.g. 12550), the DB fingerprint or total_chunks is wrong (see DB fix section).
• Patching dist/llm.js is fragile: any npm update or bun install that rebuilds QMD will overwrite the patch. Re-apply after updates.
• Vectors ARE usable from the database directly; only QMD's own search commands are blocked.

Docker Build for Linux (to bypass Metal)

Building QMD inside Docker to use CPU-only mode on Linux was attempted but hit two blockers:

1. Pre-packaged qmd-pkg fails: The qmd-pkg/ directory contains macOS arm64 native addons (better_sqlite3.node). On Linux, these fail with "Exec format error". Cannot cross-architecture.

2. npm install in Docker fails: Installing @tobilu/qmd from npm requires native addon compilation (tree-sitter, better-sqlite3). Debian/Ubuntu containers need python3 make g++ build tools. Alpine containers need python3 build-base. The npm proxy configuration must be passed explicitly (--build-arg HTTP_PROXY=...), and host.docker.internal works on macOS Docker Desktop for proxy access.

   Even with build tools installed, the CPU-only prebuilt binary for node-llama-cpp is unavailable for Linux ARM64, and the build process may fail with xpm/cmake errors. This approach is not practical on macOS Docker Desktop.

3. Alternative: Use the Xunfei API pipeline (Phase 1/Phase 2 scripts) instead of Docker. It's faster, more reliable, and doesn't require cross-platform binary management.