kl

# Knowledge Graph Query (kl) CLI tool for querying a spatio-temporal knowledge graph built from DingTalk workplace messages and other exported sources. It contains entities and facts, plus optional experimental multi-resolution communities. Discover capabilities before checking status or querying. **Architecture**: kl CLI is a thin HTTP client talking to kl-server (port 8200). The server keeps Qdrant + SQLite warm in memory. All commands require the server to be running. ## Mandatory capability discovery **Before running `status` or choosing any query command, call:** ```bash kl capabilities --json ``` On Windows use the invocation form documented below, but keep `capabilities --json` as the first CLI arguments. Parse `commands` and use only entries whose `enabled` value is `true`; never infer availability from this static skill text. In particular, `global-search`, `community`, `members`, and the `communities` search collection are experimental and normally disabled. `ask` is always available and is the primary retrieval command. If discovery fails because the server is not running, start the server, then retry capability discovery before querying. ## Invoking `kl` (any directory, macOS & Windows) The examples below write `kl ...` / `./kl ...` for brevity, but the repo can be installed **anywhere**, so never hardcode a path. **Discover the repo root at runtime, then invoke `kl` relative to it.** Discover the repo root (call it `KL_REPO`): - This skill file lives at `<repo>/skills/kl/SKILL.md`, so the repo root is **three directories up from this file** — if you know this file's path, use its `../../..`. - Otherwise walk up from the current directory until you find the marker files (the `kl` script + `kl_cli.py`): ```bash # macOS / Linux KL_REPO="$PWD" while [ "$KL_REPO" != "/" ] && [ ! -f "$KL_REPO/kl_cli.py" ]; do KL_REPO="$(dirname "$KL_REPO")" done ``` ```powershell # Windows (PowerShell) $KL_REPO = (Get-Location).Path while ($KL_REPO -and -not (Test-Path (Join-Path $KL_REPO 'kl_cli.py'))) { $KL_REPO = Split-Path $KL_REPO -Parent } ``` Then invoke it (works from any directory): - **macOS / Linux** — the `kl` wrapper `cd`s into its own directory, so calling it via `$KL_REPO` works from anywhere: ```bash "$KL_REPO/kl" status "$KL_REPO/kl" ask "..." ``` Equivalent direct form: `"$KL_REPO/.venv/bin/python" "$KL_REPO/kl_cli.py" status`. - **Portable rule of thumb:** if `./kl` fails (wrong directory, or Windows), fall back to invoking the venv interpreter on `kl_cli.py` with the same arguments — the venv Python is at `.venv/bin/python` on macOS/Linux and `.venv/Scripts/python.exe` on Windows. On Windows, see the dedicated section below for bash-specific syntax and encoding requirements. ## Invoking on Windows (read this if you are on win32) **AIAssist agents run in bash on Windows, not PowerShell.** The macOS/Linux examples above (`$KL_REPO/kl`) and old PowerShell snippets (`& "$KL_REPO\.venv\..."`) **neither works in the agent's bash shell**. Use the patterns below instead. ### 1. Always set PYTHONUTF8=1 Windows defaults to GBK / cp1252 for console encoding. Without `PYTHONUTF8=1`, any `print()` or log line containing Chinese characters (group names, entity names, fact text) will crash with `UnicodeEncodeError` or produce garbled output you cannot read. Set it as the **first thing** before any `kl` command: ```bash export PYTHONUTF8=1 ``` Or prefix every invocation inline: ```bash PYTHONUTF8=1 "$KL_REPO/.venv/Scripts/python.exe" "$KL_REPO/kl_cli.py" status ``` ### 2. Use bash syntax: forward slashes, not backslashes The venv Python lives at `.venv/Scripts/python.exe` (note: `Scripts`, not `bin`). In bash, use **forward slashes** throughout — backslashes are escape characters in bash, so `"$KL_REPO\.venv\Scripts\..."` will break. ```bash # Discover the repo root (agent runs in bash, not PowerShell) KL_REPO="$PWD" while [ "$KL_REPO" != "/" ] && [ ! -f "$KL_REPO/kl_cli.py" ]; do KL_REPO="$(dirname "$KL_REPO")" done # All subsequent calls use forward slashes PYTHONUTF8=1 "$KL_REPO/.venv/Scripts/python.exe" "$KL_REPO/kl_cli.py" status PYTHONUTF8=1 "$KL_REPO/.venv/Scripts/python.exe" "$KL_REPO/kl_cli.py" ask "你的问题" --pretty ``` ### 3. Environment variables: use `export`, not `set` The `.env` file in the repo uses Windows CMD `set` syntax, which bash cannot `source`. Instead, export each variable explicitly in bash before starting the server or querying: ```bash export PYTHONUTF8=1 export KL_DATA_DIR="$KL_REPO/data" export KL_DWS_EXPORT_DIR="$KL_REPO/dataset/<your-export-dir>" export KL_EMBED_BASE_URL=<your-embed-endpoint>/v1 export KL_EMBED_MODEL=Qwen3-Embedding-0.6B export KL_EMBED_API_KEY=<your-key> export KL_EMBEDDING_DIM=1024 export KL_EMBED_SEND_DIMENSIONS=0 export KL_LLM_BASE_URL=<your-llm-endpoint> export KL_LLM_MODEL=qwen3.6-flash export ANTHROPIC_AUTH_TOKEN=<your-token> export DISABLE_AIOHTTP_TRANSPORT=True export KL_SERVER_PORT=8200 ``` ### 4. Start the server in bash (not via `./kl`) The `./kl` wrapper script may not work on Windows. Start the server directly. **Prerequisite:** all environment variables from section 3 must be exported in the same shell session first — the server needs `KL_EMBED_*`, `KL_LLM_*`, etc. to function: ```bash # (env vars from section 3 must already be exported in this shell) PYTHONUTF8=1 "$KL_REPO/.venv/Scripts/python.exe" "$KL_REPO/kl_server.py" & sleep 15 # Qdrant warmup PYTHONUTF8=1 "$KL_REPO/.venv/Scripts/python.exe" "$KL_REPO/kl_cli.py" status ``` ### 5. Quick reference: the complete pattern ```bash # One-time per session: discover repo + set encoding export PYTHONUTF8=1 KL_REPO="$PWD" while [ "$KL_REPO" != "/" ] && [ ! -f "$KL_REPO/kl_cli.py" ]; do KL_REPO="$(dirname "$KL_REPO")" done export KL_REPO # Environment variables (adjust paths/values for your setup) export KL_DATA_DIR="$KL_REPO/data" export KL_DWS_EXPORT_DIR="$KL_REPO/dataset/<your-export-dir>" export KL_EMBED_BASE_URL=<your-embed-endpoint>/v1 export KL_EMBED_MODEL=Qwen3-Embedding-0.6B export KL_EMBED_API_KEY=<your-key> export KL_EMBEDDING_DIM=1024 export KL_EMBED_SEND_DIMENSIONS=0 export KL_LLM_BASE_URL=<your-llm-endpoint> export KL_LLM_MODEL=qwen3.6-flash export ANTHROPIC_AUTH_TOKEN=<your-token> export DISABLE_AIOHTTP_TRANSPORT=True export KL_SERVER_PORT=8200 # Start server (if not already running) PYTHONUTF8=1 "$KL_REPO/.venv/Scripts/python.exe" "$KL_REPO/kl_server.py" & sleep 15 # Query PYTHONUTF8=1 "$KL_REPO/.venv/Scripts/python.exe" "$KL_REPO/kl_cli.py" status PYTHONUTF8=1 "$KL_REPO/.venv/Scripts/python.exe" "$KL_REPO/kl_cli.py" ask "session打标进度" --pretty ``` ### 6. Request timeout The CLI HTTP timeout defaults to **120 seconds** (configurable via `KL_CLI_TIMEOUT`). If `kl ask` times out, it usually means Phase-2 LLM synthesis is slow — the server may still complete the request; check `kl status` for server health. To allow more time: ```bash export KL_CLI_TIMEOUT=180 # 3 minutes ``` ## End to End (build → ingest → query) First-time setup from a raw DWS export to answering questions. Steps 1–4 are the one-time **build**; after that you only use step 5 (query), and re-run the ingest when new data arrives. ```bash # 1. BUILD: install deps + point at your endpoints and data (env-only, no # secrets are baked into the repo). Put the config in a local .env file # (gitignored) and source it before starting the server: pip install -r requirements.txt set -a; source .env; set +a # load KL_* + ANTHROPIC_AUTH_TOKEN ``` The repo ships a `.env` at the project root (gitignored — never commit it) with working endpoints. Nothing auto-loads it, so you must `source` it as above. It sets: ```bash # data + export export KL_DATA_DIR=./data # where knowledge.db + qdrant land export KL_DWS_EXPORT_DIR=/path/to/dws_export # the exported source folders # embedding: PAI-EAS Qwen3-Embedding-0.6B (1024-dim). URL MUST end in /v1; # KL_EMBEDDING_DIM MUST match the model (1024) — it is baked into the Qdrant # collections, so changing models later requires wiping + re-embedding. export KL_EMBED_BASE_URL=<your-embed-endpoint>/v1 export KL_EMBED_MODEL=Qwen3-Embedding-0.6B export KL_EMBED_API_KEY=<embedding-api-key> # real value lives only in .env export KL_EMBEDDING_DIM=1024 export KL_EMBED_SEND_DIMENSIONS=0 # extraction/synthesis LLM: qwen3.6-flash (Anthropic-compatible base; # do NOT append /v1 — litellm adds /v1/messages itself). export KL_LLM_BASE_URL=https://example.com/apps/anthropic export KL_LLM_MODEL=qwen3.6-flash export ANTHROPIC_AUTH_TOKEN=<api-key> # real value lives only in .env ``` ```bash # 2. SERVE: start the retrieval server first (ingest runs inside it). ./kl start kl status # wait for "status": "ready" # 3. INGEST: drive the whole build through the running server. This runs # Phase A (chunk+embed every source) → Phase B (LLM extract + build+embed # the graph) → the improve step (ENTITY_SIMILAR/FACT_SIMILAR edges + # communities L0–L3), then hot-swaps the new graph in. Non-blocking. kl ingest -d "$KL_DWS_EXPORT_DIR" # omit -d to use $KL_DWS_EXPORT_DIR kl status # poll: "Ingest: running NN% (phase)" # 4. COMMUNITY SUMMARIES (optional but recommended): kl ingest builds the # community structure but NOT their summaries/vectors. These two steps add # 1–2 sentence summaries and make `search -c communities` work. python -m kl_graph.periodic.community_summarizer # 1–2 sentence summary/community python scripts/embed_communities.py # embed summaries (search -c communities) # 5. QUERY: ask questions (see Retrieval Patterns below). kl ask "who decided to use e2b for sandbox" -k 5 kl context <fact_id> # ground the answer in source ``` Notes: - `kl ingest` **upserts** into the current DB (deterministic UUID5 ids mean re-ingesting never duplicates entities/facts); it has no wipe flag, so for a truly clean rebuild `rm -rf "$KL_DATA_DIR"` before step 2. `--no-improve` skips the community/PageRank step; `-c N` raises extraction concurrency. - For a **cold/scripted build without a running server**, the offline equivalents are `python -m scripts.ingest --fresh-db` (Phase A + B, `--fresh-db` wipes `knowledge.db` — which now holds the extraction cache, so the cache is cleared too) followed by `python -m scripts.improve --skip-llm-judge`. `kl ingest` is the preferred path when the server is up. - Community columns (`community_L0..L3`) come from the improve step; without it `community`/`members` are empty and `entity` shows no community labels. ## Commands ```bash # Lifecycle kl capabilities --json # REQUIRED FIRST: live enabled/disabled query commands kl status # Server status + DB stats (+ ingest progress) kl start # Start kl-server (retrieval) kl start embedding [--model P] [--dp N] [--tp N] [--port 8100] [--gpu-util 0.4] kl stop # Stop both servers kl stop embedding # Stop only the embedding server kl ingest [-c N] [-d PATH] [--no-improve] # in-server background ingest (Phase A + B) # All commands require kl-server running (kl start) kl entity "<name>" # Entity lookup by name (substring); shows id + # communities + top edges + facts about it (id + text) # + ENTITY_SIMILAR neighbors ("similar", with names) kl entity --id <id> # Same, but look up by entity id (exact or prefix). # Add --no-similar to skip the ENTITY_SIMILAR block. kl facts <entity_id> # Facts ABOUT an entity (id + text), confidence-sorted kl facts --fact-id <id> # A single fact by its id (exact/prefix); minimal — # use `kl context` for full source provenance kl expand <entity_id> # [DEPRECATED] ENTITY_SIMILAR neighbors — same as the # "similar" block of `kl entity --id <id>` kl community [-l L0|L1|L2|L3] [-t entity|fact] [--id N] # capability-gated kl members <id> [-l L1] [-t entity] # capability-gated kl context <fact_id> # Source message + context + entities kl timeline "<entity>" [--from YYYY-MM-DD] [--to YYYY-MM-DD] kl stats # Detailed statistics kl search "<query>" [-c chunks|messages|facts|entities|communities] [-k 10] # vector ANN, one collection kl ask "<question>" [-k 10] [--phase2|--no-phase2] [--entity NAME] [--entity-type TYPE] [--fact-type TYPE] [--seed-k 6] [--radius 1] [--max-nodes 40] kl global-search "<question>" [--user "<name>"] [--json] # experimental; use only when capability says enabled kl hop -n <node_id> -c '<cursor_json>' # expand one node one hop deeper (no LLM/embed) ``` **Ingest is two phases** (`kl ingest` for in-server background ingest, or `python -m scripts.ingest` for a cold/scripted build): **Phase A** loads + chunks + embeds *every* source folder (chat + wiki/mail/ work/…) into SQLite `chunks` + Qdrant — no LLM, dense/BM25 recall works after this. **Phase B** runs LLM entity/fact extraction over all chunks, then builds and embeds the graph (the only LLM-billed phase; results are cached per chunk).