website2markdown

Convert web pages to clean Markdown using the Website2Markdown API (md.genedai.me). Use when you need to fetch, read, extract, or analyze web page content — especially when WebFetch fails, returns incomplete content, or the target is JS-heavy/paywalled. Supports 21 platform adapters (WeChat, Zhihu, Feishu, Twitter/X, etc.), batch processing, structured extraction, and deep crawling.

설치로 이동

저장소: Digidai/website2markdown-skills
최근 소스 활동: 2026년 3월 27일 03:24
감지된 SKILL.md 언어: 영어
스타: 2
포크: 0

설치 방법

기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.

Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.

직접 명령은 검토 Prompt를 거치지 않습니다. 실행하기 전에 소스를 확인하세요.

npx skills add https://github.com/Digidai/website2markdown-skills --skill website2markdown

명령은 한 줄로 유지됩니다. 복사하기 전에 가로로 스크롤해 전체 내용을 확인하세요.

로컬 사본을 원하시나요? SkillsMP에서 현재 제공할 수 있는 파일을 다운로드하세요.

소스 파일 검토

설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.

SKILL.md

소스 지침 · 읽기 전용 미리보기

검색으로 돌아가기 설치로 돌아가기

User provides a URL
  │
  ├─ Single page, simple site ──▶ Try WebFetch first
  │     └─ WebFetch fails? ──▶ Use website2markdown
  │
  ├─ Chinese platform / JS-heavy / paywalled ──▶ Use website2markdown directly
  │
  ├─ Multiple URLs needed ──▶ Use website2markdown batch API
  │
  ├─ Need structured fields (title, author, price) ──▶ Use extract API
  │
  └─ Need to crawl entire site/section ──▶ Use deepcrawl API

curl -s "https://md.genedai.me/https://example.com/page?raw=true"

curl -s "https://md.genedai.me/example.com/page?raw=true"

curl -s -H "Accept: text/markdown" "https://md.genedai.me/https://example.com/page"

Parameter	Output	When to use
`?raw=true`	Plain Markdown	Default for reading content
`?format=json&raw=true`	JSON `{url, title, markdown, method, timestamp}`	When you need metadata
`?format=html&raw=true`	Cleaned HTML	When you need HTML structure
`?format=text&raw=true`	Plain text (no formatting)	When you need raw text only

Parameter	Purpose	Example
`raw=true`	Return plain text instead of HTML preview	Always use this
`selector=<css>`	Extract specific elements (max 256 chars)	`selector=article`, `selector=.content`
`force_browser=true`	Force headless Chrome rendering	JS-heavy or anti-bot sites
`no_cache=true`	Bypass KV cache for fresh content	When content may have changed
`engine=jina`	Route through Jina Reader API	Alternative conversion engine

curl -s "https://md.genedai.me/https://example.com/article?raw=true" | head -c 100000

curl -s "https://md.genedai.me/https://example.com?raw=true&selector=article"
curl -s "https://md.genedai.me/https://example.com?raw=true&selector=%23main-content"

curl -s "https://md.genedai.me/https://spa-app.com?raw=true&force_browser=true"

curl -s "https://md.genedai.me/https://example.com?format=json&raw=true"

# WeChat article
curl -s "https://md.genedai.me/https://mp.weixin.qq.com/s/xxxxx?raw=true&force_browser=true"

# Zhihu article
curl -s "https://md.genedai.me/https://zhuanlan.zhihu.com/p/123456?raw=true"

# Twitter/X
curl -s "https://md.genedai.me/https://x.com/user/status/123456?raw=true&force_browser=true"

# Reddit
curl -s "https://md.genedai.me/https://reddit.com/r/programming/comments/xxx?raw=true"

API	Endpoint	Purpose
Batch	`POST /api/batch`	Convert up to 10 URLs in one request
Extract	`POST /api/extract`	Structured field extraction (CSS/XPath/Regex)
Deep Crawl	`POST /api/deepcrawl`	Crawl a site with filtering and scoring
Jobs	`POST /api/jobs`	Async task queue for long-running operations
Health	`GET /api/health`	Service health check (no auth needed)
llms.txt	`GET /llms.txt`	API description for AI discoverability

curl -s -X POST "https://md.genedai.me/api/batch" \
  -H "Authorization: Bearer <API_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com/a", "https://example.com/b"]}'

curl -s -X POST "https://md.genedai.me/api/extract" \
  -H "Authorization: Bearer <API_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"strategy": "css", "url": "https://example.com", "schema": {"fields": [{"name": "title", "selector": "h1", "type": "text"}]}}'

curl -s -X POST "https://md.genedai.me/api/deepcrawl" \
  -H "Authorization: Bearer <API_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"seed": "https://docs.example.com", "max_depth": 2, "max_pages": 20, "strategy": "bfs"}'

Category	Platforms
Chinese	WeChat, Zhihu, Feishu/Lark, Yuque, Juejin, CSDN, 36Kr, Toutiao, Weibo, NetEase
Social	Twitter/X, Reddit, Telegram
International	GitHub, Substack, Medium
Academic/Productivity	arxiv, Wikipedia, YouTube, Google Docs, Notion

Header	Meaning
`X-Markdown-Method`	`native`, `readability+turndown`, or `browser+readability+turndown`
`X-Cache-Status`	`HIT` or `MISS`
`X-Browser-Rendered`	`true` if headless Chrome was used
`X-Paywall-Detected`	`true` if paywall was triggered
`X-Markdown-Fallbacks`	Comma-separated fallback strategies applied

Symptom	Solution
Empty content returned	Add `force_browser=true`
Timeout error	Use `selector` to narrow scope, or the page is too large
429 Too Many Requests	Wait a few seconds and retry
503 Service Unavailable	For protected endpoints: check API_TOKEN; for rate limit: wait and retry
Garbled/incomplete content	Try `engine=jina` as alternative
Platform-specific failure	Verify URL format matches adapter patterns

Document	Content	Lines
`references/advanced-apis.md`	Batch, Extract, DeepCrawl, Jobs API — full parameters, request/response formats	~350
`references/platform-adapters.md`	21 platform adapters — URL patterns, special handling, troubleshooting	~300
`assets/recipes.md`	Copy-paste curl recipes — basic, platform, advanced, diagnostic	~180

Route	Auth Required?
`/<url>?raw=true`	No (public)
`/api/batch`	Yes (`API_TOKEN`)
`/api/extract`	Yes (`API_TOKEN`)
`/api/deepcrawl`	Yes (`API_TOKEN`)
`/api/jobs/*`	Yes (`API_TOKEN`)
`/api/health`	No

website2markdown

설치 방법

소스 파일 검토