| name | wiki-data-file-verification |
| description | Verify wiki data-file patches: exec-load + regression. |
| version | 1.0.0 |
| related_skills | ["hermes-tool-corruption-pitfalls","newsletter-link-extractor"] |
Wiki Data-File Verification
Post-edit verification methodology for the wiki's data-only scripts and derived artifact files. Complementary to hermes-tool-corruption-pitfalls (that skill catalogs HOW tools corrupt; this skill is the verification protocol for WHEN you patch).
When to use
- After patching any exec-loaded data-only script — the set-literal files
SKIP_DOMAINS / KNOWN_TIER1_DOMAINS (newsletter-tldr-extractor.py, detect_leaked_domains.py, ...)
- After adding/removing lines in derived artifact files (candidates.md, hold-review-tracker.md, index.md)
- After any patch response carries a sibling-subagent "modified by ..." warning (concurrent cron jobs share these files — the warning is informational, but the patch result deserves re-verification)
Why grep is not enough
Grep proves the patch text landed. It does NOT prove:
- Parse integrity — an injected entry can break the set literal (comment apostrophes, trailing-comma
,\n, SyntaxError) so the file exec-loads with a partial/empty set
- Behavior — the matching logic must actually catch the intended URLs (partial-domain entries like
jobs.ashbyhq fail endswith(), need the startswith(sd + '.') check)
- No over-breadth — the new entry must not wrongly filter legitimate TIER1 candidates
Verification protocol (data-only scripts)
Write an ad-hoc hermes-verify-* script to the OS temp dir (tempfile path, not ~), run once, delete after:
exec(open(path).read(), g) — parse-integrity proof. Crash here = file corrupt → re-patch.- Assert each NEW entry is in the set AND the version-marker string is present in the file (drift detection).
- Behavioral regression: run the real matching logic against the session's actual URLs — every leak URL must be caught, every surviving TIER1 candidate must NOT be caught.
- Cross-check the derived artifact (e.g. candidates.md) against the other data file's set → 0 leaks.
Reference matching logic (includes the partial-domain pitfall fix):
def is_skipped(url, skip):
m = re.match(r"https?://([^/]+)", url)
if not m: return False
d = m.group(1).lower()
if d.startswith("www."): d = d[4:]
for s in skip:
if d == s or d.endswith("." + s): return True
if "." in s and not s.endswith(".") and d.startswith(s + "."): return True
return False
Artifact-editing rule
Pipeline leak reports can list URLs that never landed in the artifact (blacklisted/deduped at write time — e.g. v1.6.57 reported 3 sans.org leak URLs but only 2 were in candidates.md). Drive removals from the artifact's actual lines, not from the report's verbatim list. Don't hunt for URLs that aren't there, and don't be alarmed by count mismatches.
Post-patch integrity combo (sibling-warning case)
When a patch response carries "modified by sibling subagent" (concurrent cron jobs share these files), the combined integrity proof before trusting the patch is:
- 3-copy md5 match: runtime (
~/wiki/scripts/) + ~/wiki/skills/<skill>/scripts/ + ~/.hermes/skills/<skill>/scripts/
- exec-load with membership asserts (the protocol above)
Only then re-run downstream verification (e.g. Step 8 leak recheck).
Related
hermes-tool-corruption-pitfalls — corruption-mode catalog (patch fuzzy-match, two-tree script drift, fictional-changelog pattern). Both skills are user-owned; this one is the protocol, that one is the catalog.newsletter-link-extractor — the pipeline that owns SKIP_DOMAINS / KNOWN_TIER1_DOMAINS and candidates.md
