edit-carton-kv

name

edit-carton-kv

description

WHAT: the development-flow for changing the CartON KV / CartonObj capability — the embedded-structured-KV-in-concept-descriptions feature (a <CartonObj name=..>{ JSON-with-bare-refs }</CartonObj> fence in n.d): the carton_kv parser/normalizer, edit_carton_obj (surgical single-leaf body edits + remove_fence), validate_carton_obj (schema + did-you-mean refs), the auto_link_description fence-opacity, fence-preservation (carry_forward_fences), ref-expansion, and the is_schema schema registry. It is the COMPLETE distributed edit-set (lib ↔ utils wrappers ↔ MCP tools ↔ the linker ↔ the daemon's live parse path ↔ the schema graph) + deploy + the only valid E2E test. WHEN: when editing carton_kv.py, the edit_carton_obj / validate_carton_obj / get_concept-expand MCP tools, the auto_link_description fence-opacity masking, the daemon fence/desc-mode parse path (parse_queue_file_to_concepts / batch_create_concepts_neo4j), or the is_schema schema-registry; or adding ANY new CartonObj capability (any of).

edit-carton-kv — dev-flow for the CartON KV (CartonObj) capability

CartON KV = structured key/value objects embedded INSIDE a concept's description (n.d) as <CartonObj name=X schema=Y>{ JSON-with-bare-refs }</CartonObj> fences (a bare Title_Underscore token = a carton concept REF; a quoted string = literal data). The capability is DISTRIBUTED across a pure stdlib library, neo4j-bound wrappers, MCP tools, the auto-linker, the observation daemon's live parse path, and the schema graph — AND it runs as the INSTALLED package, not the source. Editing one place desyncs it (the canonical failure: the remove_fence fix landing in a path the daemon never calls, while the ACTIVE path still dropped it). This skill is the complete edit-set + the only valid test. Never edit one place only.

Canonical source (monorepo ONLY — never /home/GOD/carton_mcp, never site-packages, never /home/GOD/core): /home/GOD/gnosys-plugin-v2/knowledge/carton-mcp/. Read each file FULLY before editing (line refs below are grounded against the current code on read; re-verify them — code drifts).

Part 1 — How you edit (the ONION: lib first, then wrapper, then MCP tool)

Every capability is a PURE carton_kv.py LIBRARY function FIRST (stdlib only, no neo4j / no MCP — so it is unit-testable standalone), THEN a carton_utils.py graph-bound WRAPPER if it needs neo4j, THEN a thin server_fastmcp.py MCP TOOL that interprets/formats. Never add an MCP tool without the lib fn under it, and never put graph logic in the lib.

• carton_kv.py — the pure core. Fence find/parse/normalize: find_carton_objs / get_carton_obj (carton_kv.py:208 / :259), the string-context-aware span scanner scan_json_span (:83), bare-ref ↔ strict-JSON converters refs_to_strict_json / body_from_obj / parse_fence_body (:122 / :166 / :180). The minimal-diff splice basis replace_carton_obj_body (:292) and the op-applier apply_carton_obj_op (:451, ops get/set/append/remove — _VALID_OPS at :424). Whole-fence delete remove_carton_obj (:305). Fence-preservation core carry_forward_fences (:318). Ref-expansion core expand_refs_in_description (:398). Schema/ref pure parts extract_refs / deref_for_validation / validate_against_schema (:527 / :547 / :559).
• carton_utils.py — the neo4j-bound wrappers: edit_carton_obj (:48), check_kv_refs (the fuzzy did-you-mean, :154), register_kv_schemas (:180), validate_carton_obj (:223), expand_carton_refs (:262). The canonical wiki-link strip primitives strip_wiki_links / deep_strip_wiki_links (:14 / :36) and the shared read-strip site _execute_neo4j_query (:1119, strip at :1134).
• server_fastmcp.py — the MCP tools (thin): edit_carton_obj ( — interprets the STRING into a ref / JSON / literal at ), (), ( — gained /, applies expansion at ), ().

Part 2 — What you must ALSO edit (the coherence edit-set — ALL of it, every time)

1. ONION DISCIPLINE. A new capability lands lib (carton_kv.py) + a unit test FIRST; add a carton_utils.py wrapper only if it touches neo4j; expose a server_fastmcp.py MCP tool only as a thin interpret/format shim. Do not skip a layer; do not leak graph logic down into the pure lib.

2. LINKER FENCE-OPACITY (one chokepoint covers all callers). auto_link_description (add_concept_tool.py:386) MASKS each whole <CartonObj>…</CartonObj> span with a Private-Use sentinel char (chr(0xE000+i), :412) BEFORE calling _auto_link_core, then restores each fence VERBATIM (:418-419). This single hook covers linker_thread (observation_worker_daemon.py:1155) and every other linker caller. Root cause it fixed: _auto_link_core has bracket-strip regexes (add_concept_tool.py:466-467, re.sub(r"\[…\]", …) / re.sub(r"[\[\]]", "", …)) that EAT JSON array brackets (["clone","install"]), plus it would linkify Title_Case words in the open tag. Any storage/linker change MUST preserve this masking — and the opacity test (test_linker_fence_opacity.py) must stay green.

3. DAEMON WRITE PATH + THE ONE LIVE PARSE PATH (load-bearing). n.d is written by batch_create_concepts_neo4j (observation_worker_daemon.py:119) — the SET n.d = CASE … END at :254-281 implements desc_update_mode (append/prepend/replace/skip), and the fence-preservation guard runs there at :229-247 (calls carry_forward_fences with removed_fences). There is exactly ONE live parse path: parse_queue_file_to_concepts (:444), called from the worker's main loop (the UNWIND batch at :1383 calls it → batch_create_concepts_neo4j at :1398); it forwards removed_fences at :486, and batch_create_concepts_neo4j reads removed_fences into the concept row at . Every fence/desc-mode change lands HERE. () is (verified 2026-06-10: only its def plus comments reference it). Do NOT maintain it "in lockstep"; do not route new behavior through it. The canonical bug this history guards: the fix originally landed only in (which the daemon never calls), so the ACTIVE path still dropped and the guard carried a removed fence back. The cure is to put fence/desc-mode handling in , the path the daemon actually runs. (Regression guard: in targets specifically.)

Part 3 — How you test it (the ONLY valid E2E gate — "lib tests passed" is NOT the gate)

Lib gate (necessary, NOT sufficient): run the 4 lib test files, all green — python3 test_carton_kv.py, python3 test_carton_kv_schema.py, python3 test_edit_carton_obj.py, python3 test_linker_fence_opacity.py (from the repo dir; the latter three import the INSTALLED package, so pip-install FIRST).

The REAL E2E gate — through the MCP surface (after pip-install + daemon-restart-if-needed + reconnect_mcp carton):

1. add_concept a concept whose description carries a <CartonObj> fence with bare refs → read the RAW n.d from neo4j (query_wiki_graph MATCH (c:Wiki {n:$n}) RETURN c.d) and confirm the fence is BYTE-INTACT (opacity: open tag not linkified, JSON array brackets not eaten).
2. edit_carton_obj SET one leaf + REMOVE one leaf → confirm ONLY those changed; prose + sibling fences byte-identical (re-read raw n.d). Allow for the async daemon (the edit queues a replace; wait for the queue to drain).
3. validate_carton_obj against a GOOD and a BAD payload → good = VALID; bad-key = the exact key reported.
4. Edit/add a fence introducing an UNKNOWN bare ref → the write is REFUSED with did-you-mean, AND query_wiki_graph shows NO stub node was created for that name.
5. get_concept with expand_refs=True at depth=0 (raw), depth=1, depth=2, and a deliberate cycle → expansion is render-only; re-read raw n.d and confirm the STORED description is UNCHANGED.

A (success) from a tool, "it imported", or "the lib tests passed" is NOT the gate. The raw n.d in neo4j + the absence of a stub node + the unchanged-after-expansion n.d are the gate.

The invariant

The CartON KV capability is distributed across carton_kv lib ↔ carton_utils wrappers ↔ server_fastmcp MCP tools ↔ the add_concept_tool linker (fence-opacity) ↔ the observation_worker_daemon's ONE live parse path (parse_queue_file_to_concepts + batch_create_concepts_neo4j; process_queue_file is dead code with zero callers) ↔ the SOUP schema graph — and it runs as the INSTALLED package, not the source. Edit one place only and it desyncs (the canonical failure: the remove_fence fix landed in the dead path while the live one dropped it). Do the whole Part-2 set + deploy (Part-2 #7) + the Part-3 E2E gate, every time.

(Knowledge/dev-flow skill — no subagent dispatch, so no RELIABILITY block.)