cleanup

name	cleanup
description	Audit and clean HQ structures, indexes, and stale content.
allowed-tools	Task, Read, Glob, Grep, Bash, Write, Edit, AskUserQuestion

/cleanup - HQ Maintenance

Audit HQ for policy violations, migrate outdated structures, and fix inconsistencies.

User's input: $ARGUMENTS

Modes

• No args / --audit: Report issues only (default, safe)
• --migrate: Convert old formats to new (prd.json → README.md)
• --fix: Auto-fix simple issues (git cleanup, archive stale files)
• --reindex: Regenerate ALL INDEX.md files from disk (full rebuild)
• --consolidate-learnings: Deduplicate, merge, and reorganize learned rules across all target files
• --consolidate-insights: Deduplicate, merge, and flag stale insights in workspace/insights/ and companies/*/knowledge/insights/

The Job

1. Run audit checks
2. Report findings
3. If --migrate or --fix: propose changes, ask confirmation, execute

---

Audit Checks

1. Project Structure

Policy: Personal/HQ projects live in personal/projects/; company projects live in companies/{co}/projects/. Each project should have a README.md.

# Find projects with only prd.json (no README)
for dir in personal/projects/*/ companies/*/projects/*/; do
  if [[ -f "${dir}prd.json" && ! -f "${dir}README.md" ]]; then
    echo "MIGRATE: $dir has prd.json but no README.md"
  fi
done

# Find projects outside approved project folders
find . -path './.git' -prune -o -name "prd.json" -print 2>/dev/null | grep -vE '^\./(personal/projects|companies/[^/]+/projects)/'

Violations:

• prd.json without README.md → needs migration
• prd.json outside personal/projects/ or companies/{co}/projects/ → needs relocation

2. Worker Registry

Policy: Each worker directory has a worker.yaml with required fields (worker.id, worker.type, worker.description). core/workers/registry.yaml is auto-generated from these — no need to check for "unindexed" workers, but DO check that every worker dir has a valid worker.yaml.

# Find worker dirs missing worker.yaml or required fields
for dir in core/workers/public/*/ companies/*/workers/*/; do
  [[ -d "$dir" ]] || continue
  yaml="${dir}worker.yaml"
  if [[ ! -f "$yaml" ]]; then
    echo "MISSING worker.yaml: $dir"; continue
  fi
  for field in id type description; do
    val=$(yq -r ".worker.${field} // \"\"" "$yaml" 2>/dev/null)
    [[ -z "$val" ]] && echo "MISSING worker.${field}: $yaml"
  done
done

# Force a fresh registry regen
bash core/scripts/generate-workers-registry.sh

3. Deprecated Directories

Policy: No apps/ directory (use personal/projects/, companies/{co}/projects/, or core/workers/)

# Check if apps/ still exists
if [[ -d "apps" ]]; then
  echo "DEPRECATED: apps/ directory still exists"
  ls apps/
fi

4. Git Status

Policy: Clean working tree, no orphaned deletions

git status --short

Issues:

• Deleted files not committed
• Untracked new files (should commit or ignore)
• Modified submodules

Note: Knowledge folders are symlinks to repos in repos/public/ and repos/private/ (gitignored). Symlinks themselves should be tracked by HQ git. Knowledge file changes are invisible to HQ git (they live in their own repos).

4b. Knowledge Repo Status

Policy: Knowledge repos should be clean (committed)

bash -c '
shopt -s nullglob
for symlink in core/knowledge/public/* core/knowledge/private/* personal/knowledge/* companies/*/knowledge; do
  [ -L "$symlink" ] || continue
  repo_dir=$(cd "$symlink" && git rev-parse --show-toplevel 2>/dev/null) || continue
  dirty=$(cd "$repo_dir" && git status --porcelain)
  [ -z "$dirty" ] && continue
  echo "DIRTY: $symlink → $repo_dir"
done
'

With --fix: Auto-commit dirty knowledge repos:

(cd "$repo_dir" && git add -A && git commit -m "chore: cleanup commit")

5. Stale Threads & Checkpoints

Policy: Archive manual threads/checkpoints older than 30 days. Purge auto-checkpoints older than 14 days.

# Auto-checkpoints older than 14 days (purge, not archive)
find workspace/threads -name "T-*-auto-*.json" -mtime +14 2>/dev/null

# Stale manual threads (new format, 30 days)
find workspace/threads -name "*.json" -not -name "*-auto-*" -mtime +30 2>/dev/null

# Stale checkpoints (legacy format)
find workspace/checkpoints -name "*.json" -mtime +30 2>/dev/null

6. Worker State Machine

Policy: Workers should have state_machine section (Loom pattern)

# Find workers without state_machine
for f in core/workers/*/worker.yaml core/workers/public/dev-team/*/worker.yaml; do
  if [[ -f "$f" ]] && ! grep -q "state_machine:" "$f"; then
    echo "MISSING: $f lacks state_machine section"
  fi
done

7. Orphaned Skills

Policy: Skills only in .claude/skills/<name>/SKILL.md format (commands tree is gone)

# Find old SKILL.md format
find . -name "SKILL.md" -not -path "./repos/*"

8. Stale INDEX.md Files

Policy: INDEX.md files should exist and match directory contents. See core/knowledge/public/hq-core/index-md-spec.md for spec.

Expected locations:

• personal/projects/INDEX.md
• companies/{product}/knowledge/INDEX.md
• companies/{company}/knowledge/INDEX.md
• core/knowledge/public/INDEX.md
• core/workers/public/INDEX.md
• core/workers/private/INDEX.md
• workspace/orchestrator/INDEX.md
• workspace/reports/INDEX.md
• workspace/social-drafts/INDEX.md

For each:

1. Check if INDEX.md exists → flag MISSING if not
2. Count entries in INDEX table vs actual directory contents → flag STALE if mismatch

With --reindex or --fix: Regenerate all INDEX.md files from disk per spec.

9. Manifest Completeness

Policy: Every company in manifest.yaml should have non-null values for all fields.

# Check for null values in manifest
grep -n "null" companies/manifest.yaml

Violations: Company with knowledge: null, empty settings when settings dir has files, etc.

With --fix: For each company with knowledge: null:

1. Create knowledge repo: repos/private/knowledge-{company}/ → git init → initial README
2. Create symlink: companies/{company}/knowledge → ../../repos/private/knowledge-{company}
3. Update manifest.yaml: replace null with companies/{company}/knowledge/

10. qmd Collection Completeness

Policy: Every company with a knowledge symlink should have a qmd collection. HQ itself should have 4 sub-collections: hq-infra, hq-workers, hq-knowledge, hq-projects (not a monolithic hq).

# Check companies with knowledge but empty qmd_collections
grep -B10 "qmd_collections: \[\]" companies/manifest.yaml | grep "^[a-z]"
# Check HQ sub-collections exist
for c in hq-infra hq-workers hq-knowledge hq-projects; do
  qmd ls "$c" 2>/dev/null | head -1 | grep -q . || echo "MISSING: $c"
done

With --fix: Create qmd collection for each missing company. If any hq-* sub-collection is missing, run core/scripts/migrate-qmd-collections.sh.

---

Migration: prd.json → README.md

For each project with only prd.json:

1. Read prd.json
2. Extract fields:
   • name → title
   • description → overview
   • metadata.goal → Goal line
   • metadata.successCriteria → Success line
   • userStories[] → User Stories section
3. Generate README.md
4. Keep prd.json as backup (rename to prd.json.bak)

Template:

# {name}

**Goal:** {metadata.goal}
**Success:** {metadata.successCriteria}

## Overview
{description}

## User Stories

### US-001: {story.title}
**Description:** {story.description}

**Acceptance Criteria:**
{story.acceptanceCriteria as checklist}

## Non-Goals
{if present}

## Technical Considerations
{if present}

---

Fix Actions

Git Cleanup

# Stage deleted files
git add -u

# Commit cleanup
git commit -m "chore: cleanup orphaned files"

Purge Stale Auto-Checkpoints

# Delete auto-checkpoints older than 14 days (no archive — they're lightweight)
find workspace/threads -name "T-*-auto-*.json" -mtime +14 -delete 2>/dev/null
echo "Purged $(find workspace/threads -name "T-*-auto-*.json" -mtime +14 2>/dev/null | wc -l) auto-checkpoints"

Archive Stale Threads & Checkpoints

mkdir -p archives/threads archives/checkpoints
find workspace/threads -name "*.json" -not -name "*-auto-*" -mtime +30 -exec mv {} archives/threads/ \;
find workspace/checkpoints -name "*.json" -mtime +30 -exec mv {} archives/checkpoints/ \;

Relocate Misplaced Projects

# Move apps/{name}/prd.json to personal/projects/{name}/
mkdir -p personal/projects/{name}
mv apps/{name}/prd.json personal/projects/{name}/

Regenerate INDEX.md Files (--reindex)

Delegate to the umbrella regenerator — same script the handoff pipeline uses.

bash core/scripts/rebuild-all-indexes.sh

Covers all 9 INDEX classes: threads, orchestrator, companies, projects, company-knowledge, public-knowledge, workers, reports, social-drafts. Per-class scripts live at core/scripts/rebuild-{class}-index.sh — each is pure bash + jq (zero Claude context), writes its Generated: {TS} header per core/knowledge/public/hq-core/index-md-spec.md, and logs wrote {path} (N entries) to stderr. Umbrella emits JSON array of regenerated paths on stdout.

---

Output Format

Audit Report

HQ Cleanup Audit
================

✓ Worker registry: 15 workers indexed
✗ Project structure: 8 issues
  - personal/projects/customer-cube: prd.json without README.md
  - personal/projects/deel-analytics: prd.json without README.md
  ...
✗ Deprecated directories: apps/ still exists (4 items)
✗ Git status: 3 uncommitted changes
✓ Checkpoints: all recent
✗ INDEX.md: 2 stale, 1 missing
  - personal/projects/INDEX.md: 30 entries vs 33 actual (stale)
  - workspace/reports/INDEX.md: missing

Summary: 14 issues found
Run `/cleanup --migrate` to convert prd.json files
Run `/cleanup --fix` to clean git and archive stale files
Run `/cleanup --reindex` to regenerate all INDEX.md files
Run `/cleanup --consolidate-learnings` to dedup and reorganize learned rules
Run `/cleanup --consolidate-insights` to dedup and flag stale insights

After Migration

Migrated 8 projects to README.md format:
- personal/projects/customer-cube/README.md (created)
- personal/projects/deel-analytics/README.md (created)
...

Original prd.json files renamed to prd.json.bak
Run `/cleanup --fix` to commit changes

---

Consolidate Learnings (--consolidate-learnings)

Dedup, merge, and reorganize learned rules across all target files.

Step 1: Collect all rules

Scan these locations and extract every rule:

Location	How to find
.claude/CLAUDE.md ## Learned Rules	Read section, parse - **{name}**: entries
Worker yamls ## Learnings	grep -rl "## Learnings" core/workers/ → read each
Skill mds ## Rules	grep -rl "## Rules" .claude/skills/ → read each
Learning event log	ls workspace/learnings/*.json → read rules[] from each

Build a master list: {rule_text, source_file, section, date_added}.

Step 2: Cross-file dedup

For each rule in the master list:

qmd vsearch "{rule_text}" --json -n 10

Flag:

• Exact duplicates (similarity > 0.85 across different files): keep the most specific (worker > skill > global), remove the other
• Near-duplicates (0.6–0.85): merge into one rule with combined context, remove the weaker copy
• Contradictions: flag for user review (don't auto-resolve)

Step 3: Deprecate stale rules

For each rule, check if its references still exist:

• Rule mentions a worker → does core/workers/public/{id}/worker.yaml or companies/{co}/workers/{id}/worker.yaml exist?
• Rule mentions a slash command → does .claude/skills/{name}/SKILL.md exist?
• Rule mentions a tool/API → is it still in use? (best effort)

Flag stale rules for user review. Don't auto-delete — present as candidates.

Step 4: Reorganize scope

If a scoped rule (worker/command) has been superseded by a broader global rule covering the same behavior, remove the scoped copy (the global rule covers it).

If a global rule only applies to one worker/command, demote it to the scoped file and remove from CLAUDE.md (frees global cap space).

Step 5: Apply changes

For each proposed change (remove/merge/demote/promote), apply to target files using Edit tool.

Step 6: Reindex

qmd update && qmd embed

Step 7: Report

Learning Consolidation
======================
Rules scanned: {total}
  - CLAUDE.md: {n}
  - Workers: {n} across {m} files
  - Commands: {n} across {m} files

Actions taken:
  ✓ Removed {n} duplicates
  ✓ Merged {n} near-duplicates
  ✓ Demoted {n} global → scoped
  ✓ Promoted {n} scoped → global
  ⚠ {n} stale rules flagged (review below)
  ⚠ {n} contradictions flagged (review below)

Stale rules:
  - {rule} in {file} — references deleted worker {id}
  ...

Contradictions:
  - {rule_a} vs {rule_b} — {explanation}
  ...

---

Consolidate Insights (--consolidate-insights)

Deduplicate, merge, and flag stale insights across all insight directories.

Step 1: Collect all insights

Scan these locations:

Location	How to find
workspace/insights/global/	ls workspace/insights/global/*.md
workspace/insights/tools/	ls workspace/insights/tools/*.md
workspace/insights/concepts/	ls workspace/insights/concepts/*.md
companies/*/knowledge/insights/	ls companies/*/knowledge/insights/*.md 2>/dev/null

Build master list: {title, slug, scope, confidence, created, file_path} from YAML frontmatter.

Step 2: Cross-file dedup

For each insight:

qmd vsearch "{insight title + first sentence}" --json -n 10

Flag:

• Exact duplicates (similarity > 0.85): keep the more detailed version, remove the other
• Near-duplicates (0.6–0.85): merge into one insight with combined context, remove the weaker copy
• Cross-scope overlap: company insight that duplicates a global insight → keep company version (more specific)

Step 3: Flag stale insights

Insights older than 90 days with confidence: medium are stale candidates:

# Find medium-confidence insights older than 90 days
for f in workspace/insights/**/*.md companies/*/knowledge/insights/*.md; do
  [ -f "$f" ] || continue
  confidence=$(grep "^confidence:" "$f" | awk '{print $2}')
  created=$(grep "^created:" "$f" | awk '{print $2}')
  [ "$confidence" = "medium" ] && echo "STALE CANDIDATE: $f (created: $created)"
done

Present stale candidates for user review. Don't auto-delete.

Step 4: Apply changes

For each proposed change (remove/merge), apply using Edit tool. Update updated date on merged insights.

Step 5: Reindex

qmd update && qmd embed

Step 6: Report

Insight Consolidation
=====================
Insights scanned: {total}
  - Global: {n}
  - Tools: {n}
  - Concepts: {n}
  - Company-scoped: {n} across {m} companies

Actions taken:
  ✓ Removed {n} duplicates
  ✓ Merged {n} near-duplicates
  ⚠ {n} stale insights flagged (review below)

Stale candidates:
  - {title} in {file} — medium confidence, created {date}
  ...

---

Rules

• --audit is safe: Never modifies files, only reports
• Always ask before destructive actions: deletions, moves
• Backup before migration: rename, don't delete
• Commit after changes: keep git clean

---

Current HQ Policies

Reference for what we're enforcing:

Area	Policy
Projects	Live in personal/projects/{name}/ or companies/{co}/projects/{name}/ with README.md
PRD format	Markdown README.md (not prd.json)
Workers	Each has worker.yaml with worker.id/type/description; registry auto-generates
Worker FSM	state_machine: section in worker.yaml (Loom pattern)
Apps	Deprecated - migrate to personal/projects/, companies/{co}/projects/, or core/workers/
Skills	.claude/skills/<name>/SKILL.md format
Threads	Primary session persistence (workspace/threads/)
Auto-checkpoints	Lightweight, purge after 14 days (T-*-auto-*.json)
Checkpoints	Legacy format, archive after 30 days
Metrics	Append to workspace/metrics/metrics.jsonl
Git	Clean working tree
Knowledge repos	Symlinks in knowledge/ and companies/*/knowledge/ point to repos; all repos committed
INDEX.md	Exist at 10 key dirs, match contents (see spec)
Manifest	All companies have non-null knowledge, settings, repos
qmd	All companies with knowledge have a qmd collection
Learnings	No cross-file duplicates, stale rules flagged, scoped > global