swfl-arrest-scrapers

Use this skill when expanding the scraper network to new Florida counties. Contains the 67-county expansion roadmap, prioritization framework, site reconnaissance procedure, and lead analysis strategy. The goal is total Florida coverage for maximum bail bond lead generation.

Use this skill when creating a new county arrest scraper, modifying an existing solver, or adding a new county to the scraping pipeline. Contains the canonical architecture, naming conventions, and integration checklist.

Non-negotiable rules, safe refactoring patterns, shared code modification guidelines, secrets management, config hierarchy, and schema change process for the swfl-arrest-scrapers repo. Read this before modifying any core infrastructure.

Testing patterns, fixture guidelines, parser test templates, smoke tests, and red-green TDD methodology for the scraper pipeline. Use when writing tests for new counties, verifying core/ changes, or debugging test failures.

Google Sheets: Read and write spreadsheets via the Sheets API v4.

Systematically strengthen scraper pipelines against network failures, anti-bot measures, data edge cases, and real-world brittleness that breaks idealized scraper designs.

Performance optimization patterns for Python scraper pipelines — connection pooling, memory efficiency, async I/O, and batch operations for high-throughput data ingestion.

Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always

Use when writing, customizing, or troubleshooting GitHub Actions workflows for scraper scheduling, Docker builds, or CI/CD pipeline issues.

Use when working with Google Sheets data output — writing arrest records, managing tabs, handling schema, debugging API errors.

Use when building scrapers that need browser automation — Playwright or DrissionPage. Covers anti-bot evasion, Cloudflare bypass, headless Chrome, and browser-based data extraction.

Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes

Automatically triggered after completing ANY task in this repo — fixing a bug, adding a county, debugging a failure, or modifying infrastructure. Implements a multi-memory learning loop that extracts patterns from experiences and patches skill files to prevent repeating mistakes. Based on the charon-fan/agent-playbook self-improving-agent pattern.

Use this skill when a scraper workflow is failing in GitHub Actions, when a solver is returning 0 records, or when debugging any runtime error in the scraping pipeline. Contains diagnostic procedures, common failure modes, and fix patterns.