Menu
Platform rules, Taiwan title filter, date extraction, and lookback window for the IDE-JETRO scraper
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Planning principles, model selection, and scope rules for the Architect agent
Implementation rules for database migrations, Python scrapers, and Next.js web for the Engineer agent
BaseScraper contract, field rules, and Peatix-specific conventions for the Scraper Expert agent
Scraper test execution rules, output validation criteria, and report format for the Tester agent
BaseScraper contract, field rules, and Peatix-specific conventions for the Scraper Expert agent
Platform rules and field mappings for the 誠品生活日本橋 (eslite spectrum Nihonbashi) scraper
| name | ide_jetro |
| description | Platform rules, Taiwan title filter, date extraction, and lookback window for the IDE-JETRO scraper |
| applyTo | scraper/sources/ide_jetro.py |
| Field | Value |
|---|---|
| Site URL | https://www.ide.go.jp |
| Listing URL | https://www.ide.go.jp/Japanese/Event/Seminar.html |
| Rendering | Static HTML — uses requests + BeautifulSoup (no Playwright) |
| Auth required | No |
| Rate limit | REQUEST_DELAY = 1.0 s between detail-page fetches |
| Source name | ide_jetro |
| Source ID format | ide-jetro_{YYMMDD} from URL path (e.g. /Japanese/Event/Seminar/250926.html → ide-jetro_250926) |
| Event Field | Source |
|---|---|
source_id | ide-jetro_{YYMMDD} from <a href> path |
raw_title | og:title meta tag on detail page, or <h1> |
start_date | First YYYY.MM.DD occurrence in listing <span class="date"> |
end_date | Second date in range notation YYYY.MM.DD〜YYYY.MM.DD; None if single-day |
location_name | IDE venue (usually 幕張メッセ or 研究所内) from body |
raw_description | Detail page body text |
For seminar pages, scraper must parse heading-based sections in detail HTML and map them explicitly:
| Detail section | Target field | Rule |
|---|---|---|
開催日程 | business_hours | extract time range (HH:MM〜HH:MM or HH時MM分〜HH時MM分) |
会場 | location_name | keep venue text; online webinars should stay オンライン |
主催 | organizer | use exact text from section block |
講師・プログラム table | performers (via annotator) | collect speaker names from speaker column and inject into raw_description |
Do not rely on one generic paragraph-only description. If these sections are not parsed, front-end detail pages lose organizer/speaker/hours.
IDE pages are rendered with nested blocks (pbNested, paragraph, table-basic) after <h4> headings. Traditional sibling-only scraping can return empty strings even when content exists.
Required pattern:
開催日程, 会場, 主催, 講師・プログラム).div with class containing pbNested|paragraph|table-basic).When this guard is violated, typical symptoms are:
business_hours = nullorganizer = nullperformer(s) = nullfor events that clearly have these fields on the source page.
Always prepend structured lines before free-text summary when values exist:
会場: ...主催: ...時間: ...講師: ...This ensures annotator can deterministically recover organizer and performer fields.
The listing contains hundreds of seminars; only keep items whose title contains 台湾:
if "台湾" not in item_title:
continue
This is the primary relevance gate. Do not widen it — IDE-JETRO events are academic/business and the false-positive rate for non-Taiwan content is high.
LOOKBACK_DAYS = 180 — only ingest events whose start_date is within the past 180 days or any future date. The listing page accumulates past years; without this cutoff the scraper re-ingests stale records on every run.
Listing page dates are YYYY.MM.DD (dotted, ASCII digits):
Single : 2025.09.26 (金曜)
Range : 2025.11.25 (火曜)〜2026.03.13 (金曜)
_parse_date_dotted(s) extracts the first YYYY.MM.DD occurrence.
The /Sympo/ page mirrors the /Seminar/ listing — 100% overlap. Only scrape Seminar.html; do not add the Symposia URL to avoid duplicate ingestion.