Run any Skill in Manus with one click

web-scraper

Stars2

Forks1

UpdatedFebruary 6, 2026 at 22:15

Fetch any URL and extract clean readable content as text or markdown. Use this skill when the user asks to scrape a webpage, extract text from a URL, fetch website content, read an article from a link, or download webpage content.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

dalehurley

dalehurley/phpbot

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Software DevelopersComputer and Mathematical Occupations·SOC 15-1252

File Explorer

2 files

SKILL.md

readonly

name	web-scraper
description	Fetch any URL and extract clean readable content as text or markdown. Use this skill when the user asks to scrape a webpage, extract text from a URL, fetch website content, read an article from a link, or download webpage content.
tags	["web","scraper","url","fetch","extract","article","html","utility"]
version	0.1.0

Skill: web-scraper

When to Use

Use this skill when the user asks to:

Scrape or fetch content from a URL
Extract text from a webpage
Read an article from a link
Get the content of a website
Download and convert a webpage to text/markdown
Summarize a web page

Input Parameters

Parameter	Required	Description	Example
`url`	Yes	URL to scrape	https://example.com/article
`format`	No	Output format: `markdown` (default), `text`, `html`	markdown
`output`	No	Save output to file path	article.md

Procedure

Get the URL from the user's request

Run the bundled script:

# Fetch and extract as markdown (default)
python3 skills/web-scraper/scripts/scrape.py "https://example.com/article"

# Extract as plain text
python3 skills/web-scraper/scripts/scrape.py "https://example.com" --format text

# Save to file
python3 skills/web-scraper/scripts/scrape.py "https://example.com" --output article.md

The script auto-installs trafilatura and requests if needed
Present the extracted content to the user

Bundled Scripts

Script	Type	Description
`scripts/scrape.py`	Python	Fetch URL and extract clean content

Script Usage

# Extract article content as markdown
python3 scripts/scrape.py "https://example.com/blog-post"

# Extract as plain text
python3 scripts/scrape.py "https://example.com" --format text

# Keep raw HTML
python3 scripts/scrape.py "https://example.com" --format html

# Save to file
python3 scripts/scrape.py "https://example.com" --output page.md

# Include metadata (title, author, date)
python3 scripts/scrape.py "https://example.com/article" --metadata

Example

scrape this URL: https://example.com/article
extract the text from this webpage
fetch the content of this link and save it
read this article: https://blog.example.com/post-1
get the content from this website

More from this repository

same repository

append-text-to-file

dalehurley/phpbot

Append or add new text content to an existing file. Use this skill when the user asks to update a file, add content, append text, add a verse, add a section, or extend a document with additional material. Supports any text-based file format including .txt, .md, .py, .json, and more.

2026-02-192

create-text-file

dalehurley/phpbot

Create and write text content to a file in a specified directory. Use this skill when the user asks to create a file, write content to a file, save text, generate a document, or write a poem, story, script, or any text-based content to disk. Supports any text format and custom file paths.

2026-02-192

update-text-file

dalehurley/phpbot

Update or modify text files (poems, documents, notes, etc.) in any directory. Use this skill when the user asks to edit, update, modify, revise, or rewrite a text file, poem, document, or similar content. Supports adding new content, expanding existing text, or replacing file contents while preserving the original location.

2026-02-192

analyze-codebase-for-improvements

dalehurley/phpbot

Analyze a codebase or project to identify actionable improvements, architectural enhancements, and optimization opportunities. Use this skill when the user asks to review code quality, suggest improvements, identify bottlenecks, audit a system, or get recommendations for enhancing a project. Generates a prioritized list of improvements with problem statements, solutions, and impact analysis.

2026-02-192

analyze-document

dalehurley/phpbot

Analyze and provide detailed feedback on documents including PDFs, text files, and other formats. Use this skill when the user asks for your thoughts on a document, wants a review, analysis, summary, critique, or evaluation of any written material. Supports document upload, file path references, or direct text content.

2026-02-192

answer-factual-question

dalehurley/phpbot

Answer factual questions about events, people, places, and general knowledge topics. Use this skill when the user asks 'who won', 'what happened', 'when did', 'where is', or other questions seeking factual information about sports, history, geography, current events, or other verifiable facts.

2026-02-192

name	web-scraper
description	Fetch any URL and extract clean readable content as text or markdown. Use this skill when the user asks to scrape a webpage, extract text from a URL, fetch website content, read an article from a link, or download webpage content.
tags	["web","scraper","url","fetch","extract","article","html","utility"]
version	0.1.0

Skill: web-scraper

When to Use

Use this skill when the user asks to:

Scrape or fetch content from a URL
Extract text from a webpage
Read an article from a link
Get the content of a website
Download and convert a webpage to text/markdown
Summarize a web page

Input Parameters

Parameter	Required	Description	Example
`url`	Yes	URL to scrape	https://example.com/article
`format`	No	Output format: `markdown` (default), `text`, `html`	markdown
`output`	No	Save output to file path	article.md

Procedure

Get the URL from the user's request

Run the bundled script:

# Fetch and extract as markdown (default)
python3 skills/web-scraper/scripts/scrape.py "https://example.com/article"

# Extract as plain text
python3 skills/web-scraper/scripts/scrape.py "https://example.com" --format text

# Save to file
python3 skills/web-scraper/scripts/scrape.py "https://example.com" --output article.md

The script auto-installs trafilatura and requests if needed
Present the extracted content to the user

Bundled Scripts

Script	Type	Description
`scripts/scrape.py`	Python	Fetch URL and extract clean content

Script Usage

# Extract article content as markdown
python3 scripts/scrape.py "https://example.com/blog-post"

# Extract as plain text
python3 scripts/scrape.py "https://example.com" --format text

# Keep raw HTML
python3 scripts/scrape.py "https://example.com" --format html

# Save to file
python3 scripts/scrape.py "https://example.com" --output page.md

# Include metadata (title, author, date)
python3 scripts/scrape.py "https://example.com/article" --metadata

Example

scrape this URL: https://example.com/article
extract the text from this webpage
fetch the content of this link and save it
read this article: https://blog.example.com/post-1
get the content from this website