Raw-Knowledge

Curate raw sources (YouTube transcripts and web articles) into a typed-relationship ontology of concept and entity pages. Compounds knowledge, deduplicates, maintains index.md and log.md. Use when sources need processing into the wiki, or for query/lint/digest operations.

Fetch a web article from a URL and save it as an immutable raw source page in clean markdown. Primary extractor is markitdown; falls back to the obscura stealth headless browser for JS-heavy or bot-protected pages (Medium, Substack, Cloudflare). Use when a user provides an article/blog/docs URL to ingest into the knowledge base.

Extract a transcript from a YouTube URL and save it as an immutable raw source page. Uses a resilient fallback chain (youtube-transcript-api -> yt-dlp subtitles -> whisper audio). Use when a user provides a YouTube link to ingest into the knowledge base.