Investigate a failing crawler and propose a fix, starting from a dataset name or an issues.json artifact URL. Covers pulling the diagnostic report, inspecting source data via Zyte, and common failure patterns including sources that are blocked, geo-blocked, 403/429-throttled, or behind a JavaScript challenge or anti-bot protection.
Complete the name framework migration in a crawler (Step 3) by removing all custom name cleaning/splitting logic and the Step 1 review scaffolding, replacing it with a single h.apply_reviewed_name_string or h.apply_reviewed_names call. Use only after Step 1 has been deployed and run in production and the dataset's name reviews have been completed (Step 2).
Rewrite messy or AI-generated crawler code into clean, production-ready style that follows the zavod best practices. Use when the user asks to clean up, refactor, tidy, or "make production-ready" a crawler, or to bring code in line with best practices.
Migrate ad-hoc name cleaning in a crawler to h.review_names (Step 1 of the name framework migration). Use when a crawler.py contains delimiter splits, regex substitutions, bracket stripping, or conditional logic applied to name strings before the name is added or applied.
Release one or more datasets by adding them to a topical collection, bumping coverage.start, and verifying. Use when the user wants to release/publish specific crawled datasets into the OpenSanctions product.
Fix mypy --strict type errors in crawler files. Use when the user asks to make the typechecker happy, fix types, or add type annotations to a crawler.
Scaffold a new sanctions list crawler from a source URL or GitHub issue
Scaffold a new PEP (Politically Exposed Persons) crawler from a source URL or GitHub issue