| name | build-latentspace |
| description | Rebuild the D3.js post map by embedding all blog posts with OpenAI and reducing to 2D with deterministic PCA. Run when posts are added, removed, or labels need refreshing. |
| user-invocable | true |
| allowed-tools | Bash Read |
Build Latent Space
Rebuild data/latentspace.json — the single data source for the D3.js latent space visualization on the landing page.
What it does
- Collects all eligible posts from
note/_posts and testbed/_posts (excludes _ prefixed files and empty posts)
- Embeds each post using OpenAI
text-embedding-3-small
- Reduces embeddings to 2D with deterministic PCA
- Assigns a label based on
emoji frontmatter
- Extracts
connected links from related frontmatter
- Writes the result to
data/latentspace.json
Incremental mode: cached embeddings in data/embeddings.json avoid repeat OpenAI calls. Coordinates are recomputed from the full embedding matrix with deterministic PCA.
Prerequisites
OPENAI_API_KEY environment variable must be set- Python dependencies:
openai, beautifulsoup4, lxml, numpy, scikit-learn, tiktoken
Run
SKILL_DIR="${CLAUDE_SKILL_DIR:-$(git rev-parse --show-toplevel)/.agents/shared/skills/build-latentspace}"
python3 "${SKILL_DIR}/scripts/build_post_map.py"
Verify the output looks correct:
python3 -c "import json; data=json.load(open('data/latentspace.json')); print(f'{len(data)} posts'); print({d['label'] for d in data}); print(f'{sum(len(d[\"connected\"]) for d in data)} connections')"
Output schema
{
"title": "GAN",
"slug": "gan",
"url": "/gan/",
"category": "note",
"label": "Brain",
"connected": ["conditional-gan", "building-gan"],
"x": 0.3421,
"y": 0.7812
}
Label mapping
Labels are derived from the emoji frontmatter field of each post:
| Emoji file | Label |
|---|
| brain.png | Brain |
| books.png | Books |
| pin.png | Pin |
| eyes.png | Eyes |
| wordballoon-with-dots.png | Wordballoon |
| robot.png | Robot |
| storm.png | Storm |
| (testbed posts) | Testbed |
| (no emoji) | Note |
