| name | kabigon |
| description | Load URL content as text or markdown with kabigon. Use this for extracting content from YouTube videos, social posts, news articles, PDFs, GitHub files, generic web pages, and audio/video URLs through kabigon's automatic Pipeline planning. |
How to load URL content
Use the automatic path unless you are debugging or intentionally comparing Loaders. Kabigon matches the URL to a source-specific Pipeline, builds an Execution plan from Targeted loaders plus allowed Fallback loaders, and returns the first successful Loader result.
# CLI usage
uvx kabigon https://www.youtube.com/watch?v=dQw4w9WgXcQ
uvx kabigon https://x.com/howie_serious/status/1917768568135115147
uvx kabigon https://reddit.com/r/python/comments/xyz/...
uvx kabigon https://github.com/user/repo/blob/main/README.md
uvx kabigon https://example.com/document.pdf
import kabigon
text = kabigon.load_url_sync("https://www.youtube.com/watch?v=dQw4w9WgXcQ")
import asyncio
import kabigon
text = await kabigon.load_url("https://x.com/user/status/123")
# List supported loader names.
uvx kabigon --list
import kabigon
plan = kabigon.explain_plan("https://www.youtube.com/watch?v=dQw4w9WgXcQ")
loaders = kabigon.available_loaders()
Advanced loader selection
Use --loader only when debugging, comparing Loaders, or intentionally bypassing automatic Pipeline planning. A comma-separated loader list runs in the exact order provided.
uvx kabigon --loader playwright https://example.com
uvx kabigon --loader httpx https://example.com
uvx kabigon --loader firecrawl https://example.com
uvx kabigon --loader bbc https://www.bbc.com/news/articles/example
uvx kabigon --loader cnn https://www.cnn.com/2025/01/01/world/example/index.html
uvx kabigon --loader youtube https://www.youtube.com/watch?v=dQw4w9WgXcQ
uvx kabigon --loader youtube-ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ
uvx kabigon --loader ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ
uvx kabigon --loader twitter https://x.com/howie_serious/status/1917768568135115147
uvx kabigon --loader truthsocial https://truthsocial.com/@realDonaldTrump/posts/115830428767897167
uvx kabigon --loader reddit https://reddit.com/r/confession/comments/1q1mzej/im_a_developer_for_a_major_food_delivery_app_the/
uvx kabigon --loader ptt https://www.ptt.cc/bbs/Gossiping/M.1746078381.A.FFC.html
uvx kabigon --loader reel https://www.instagram.com/reel/CuA0XYZ1234/
uvx kabigon --loader github https://github.com/anthropics/claude-code/blob/main/plugins/ralph-wiggum/README.md
uvx kabigon --loader pdf https://example.com/document.pdf
# Run loaders in explicit order. Example: try YouTube transcripts first,
# then fall back to `youtube-ytdlp` audio transcription if captions are missing.
uvx kabigon --loader youtube,youtube-ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ
Supported sources
- YouTube videos:
youtube, then youtube-ytdlp audio transcription when needed.
- Social posts:
twitter, truthsocial, reddit, ptt, and reel.
- News articles:
bbc and cnn article-aware extraction.
- Code and documents:
github file/page content and pdf text extraction.
- Generic web pages:
playwright, httpx, or firecrawl.
- Audio/video URLs:
ytdlp generic audio transcription.
Failure behavior
- A Loader that cannot handle the input raises a not-applicable attempt so the Load chain can continue.
- Content extraction, timeout, and configuration failures are reported in the Load chain failure details.
- Prefer
kabigon.explain_plan(url) when you need to understand why a URL will use a specific Pipeline or Loader order.
Configuration notes
FIRECRAWL_API_KEY is required for the firecrawl loader.FFMPEG_PATH can point to a custom FFmpeg binary for Whisper and yt-dlp transcription loaders.
Troubleshooting
