Menu
Web scraping workflows for energy data collection from BSEE and BOEM using Scrapy
The review-first prompt is selected by default. You can switch to a direct command or download a local copy.
Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
npx skills add https://github.com/vamseeachanta/workspace-hub --skill web-scraper-energyThe command stays on one line. Scroll horizontally to inspect it before copying.
Prefer a local copy? Download the files currently available to SkillsMP.
Write outbound email and external messages in Vamsee Achanta's voice — a subtle offer to help, never bold or rash claims. Load before drafting ANY email, LinkedIn/Collide reply, proposal note, or outreach sent under his name.
Save/publish analysis or computation results from ANY ecosystem repo to Hugging Face as a queryable, viewer-renderable dataset. Use when the user wants to "save results to hugging face", "publish dataset to HF", "hugging face data saving", "save analysis results", "hf dataset", "make results queryable", or "render via datasets-server API". Reshapes nested results into flat parquet tables, writes a dataset card with a viewer `configs:` block and provenance, applies license/public-vs-private routing, enforces a domain data-quality gate (faithful-to-source != correct), publishes to `aceengineer/<repo>-<projection>`, and verifies via the datasets-server API.
Clone, create, fork, configure, and manage GitHub repositories. Manage remotes, secrets, releases, and workflows. Works with gh CLI or falls back to git + GitHub REST API via curl.
Showing SKILL.md
Based on SOC occupation classification
| name | web-scraper-energy |
| description | Web scraping workflows for energy data collection from BSEE and BOEM using Scrapy |
| type | reference |
| capabilities | [] |
| requires | [] |
| tags | [] |
| category | data |
| version | 1.0.0 |
Use this skill when you need to:
"""
ABOUTME: Web scraping utilities for energy data collection
ABOUTME: Supports Scrapy spiders and BeautifulSoup parsing
"""
from dataclasses import dataclass
from typing import List, Dict, Optional
from bs4 import BeautifulSoup
import requests
import time
@dataclass
class ScrapingConfig:
"""Configuration for web scraping."""
base_url: str
rate_limit_sec: float = 1.0
max_retries: int = 3
timeout_sec: int = 30
user_agent: str = "WorldEnergyData/1.0"
cache_enabled: bool = True
class BOEMScraper:
*See sub-skills for full details.*
## YAML Configuration Template
```yaml
# config/input/scraping-config.yaml
metadata:
feature_name: "energy-scraping"
created: "2025-01-15"
scraping:
rate_limit_sec: 1.5
max_retries: 3
timeout_sec: 30
cache_enabled: true
cache_ttl_hours: 24
targets:
- name: "lease_sales"
source: "boem"
sale_numbers: [257, 258, 259]
output: "data/lease_sales/"
- name: "platforms"
source: "boem"
areas: [, , ]
output:
*See sub-skills full details.*
```bash
python -m worldenergydata.scraper \
--source boem \
-- lease-sale \
--sale \
--output data/lease_sale_259.csv
python -m worldenergydata.scraper \
--source boem \
-- platforms \
--area GC \
--output data/gc_platforms.csv
No external MCP or paid service needed for energy data scraping.
This skill's requests + BeautifulSoup pattern is sufficient for BSEE/BOEM/EIA targets.
For async fetching at scale (WRK-1202 Tier 3), upgrade to httpx (async) + beautifulsoup4.
For JS-rendered pages, use claude-in-chrome browser automation (already available).
See doc-research-download skill for the full assessment.