菜单
Use this skill when expanding the scraper network to new Florida counties. Contains the 67-county expansion roadmap, prioritization framework, site reconnaissance procedure, and lead analysis strategy. The goal is total Florida coverage for maximum bail bond lead generation.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | county-expansion |
| description | Use this skill when expanding the scraper network to new Florida counties. Contains the 67-county expansion roadmap, prioritization framework, site reconnaissance procedure, and lead analysis strategy. The goal is total Florida coverage for maximum bail bond lead generation. |
Expand from 18 active counties to all 67 Florida counties. Every new county directly increases Shamrock's geographic reach, lead volume, and revenue potential. This isn't just scraping — it's market domination.
Brevard, Charlotte, Collier, DeSoto, Hendry, Highlands, Hillsborough, Indian River, Lake, Lee*, Manatee, Martin, Orange, Osceola, Palm Beach, Pinellas, Polk, Sarasota, Seminole
*Lee is scraped by GAS internal scraper (shamrock-bail-portal-site), not this repo.
| County | Pop | Jail System | URL Pattern | Notes |
|---|---|---|---|---|
| Miami-Dade | 2.7M | Miami-Dade Corrections | corrections.miamidade.gov | Largest county, complex system |
| Broward | 1.9M | BSO | sheriff.broward.org | 2nd largest, high volume |
| Duval | 1.0M | JSO | inmatesearch.jaxsheriff.org | Jacksonville metro |
| Pasco | 560K | PCSO | pascosheriff.com | Tampa metro spillover |
| Volusia | 550K | VCSO | volusia.org | Daytona Beach area |
| St. Lucie | 340K | SLCSO | stluciesheriff.com | Treasure Coast expansion |
Escambia, Leon, Alachua, Marion, Okaloosa, Bay, St. Johns, Flagler, Hernando, Citrus, Sumter, Putnam, Columbia, Nassau, Clay, Santa Rosa, Walton, Osceola (done), Okeechobee, Glades
Baker, Bradford, Calhoun, Dixie, Franklin, Gadsden, Gilchrist, Gulf, Hamilton, Hardee, Holmes, Jackson, Jefferson, Lafayette, Levy, Liberty, Madison, Monroe, Suwannee, Taylor, Union, Wakulla, Washington
Before writing a single line of code, follow this recon checklist:
# Common URL patterns to try:
https://www.{county}sheriff.org/inmate-search
https://www.{county}so.org/inmates
https://jailviewer.{county}clerk.com
https://www.{county}jail.com
https://apps.{county}.org/inmatesearch
Open DevTools (Network tab) and classify:
| What You See | Stack | Tool to Use |
|---|---|---|
| Full HTML in page source | Static | requests + BeautifulSoup |
| AJAX/XHR calls returning JSON | API-first | requests (direct API) |
| Blank HTML, JS loads data | React/Vue SPA | DrissionPage |
| ASP.NET ViewState, __EVENTVALIDATION | .NET PostBack | curl_cffi + state management |
| Cloudflare challenge page | CF Protected | DrissionPage (headless Chrome) |
| CAPTCHA on search | CAPTCHA | May need solver service — evaluate ROI |
Many jail systems use commercial software (JailTracker, Omnixx, Tyler Technologies) that expose JSON APIs under the hood. Check:
/api/inmates, /api/search, /api/bookingsAccept: application/json is sentcurl -s -o /dev/null -w "%{http_code}" -H "User-Agent: Mozilla/5.0" "URL"
# 200 = open access
# 403 = bot detection (need browser or curl_cffi impersonation)
# 503 = Cloudflare challenge (need DrissionPage)
For a site with no anti-bot and standard HTML:
# 1. Create county directory
COUNTY=duval
mkdir -p counties/$COUNTY
touch counties/$COUNTY/__init__.py
cp counties/_template/solver.py counties/$COUNTY/solver.py
cp counties/_template/runner.py counties/$COUNTY/runner.py # if not already identical
# 2. Edit solver.py — update these 4 things:
# - BASE_URL
# - scrape_{county}() function name
# - Page structure parsing
# - Record field mapping
# 3. Create config
cp config/counties/_defaults.yaml config/counties/$COUNTY.yaml
# Edit: name, url, sheet_tab
# 4. Create workflow
cp .github/workflows/scrape_brevard.yml .github/workflows/scrape_$COUNTY.yml
# Edit: name, cron schedule (stagger!), county name
# 5. Test locally
python counties/$COUNTY/solver.py --days-back 3
# 6. Push and verify
git add -A && git commit -m "feat: add $COUNTY county scraper"
git push origin main
Workflows must be staggered to avoid:
Pattern: Assign each county a unique (minute, hour-set) pair:
# Example: Every 6 hours, staggered by 5 minutes
County A: '0 */6 * * *'
County B: '5 */6 * * *'
County C: '10 */6 * * *'
...
Not all arrests are bail-bondable. After scraping, the pipeline should:
The core record dict maps to Google Sheets columns via core/writers/sheets_writer.py:
Booking_Number, County, State, Facility, Full_Name, First_Name, Last_Name,
Middle_Name, DOB, Sex, Race, Height, Weight, Hair_Color, Eye_Color,
Address, City, Zipcode, Booking_Date, Release_Date, Status,
Charges, Bond_Amount, Bond_Type, Bond_Status, Agency, Case_Number,
Court_Date, Court_Type, Court_Location, Mugshot_URL, Detail_URL,
Scrape_Timestamp, Person_ID, Notes, Phone, Email, Emergency_Contact, Occupation
Many Florida counties use the same commercial software. If you crack one, you crack many:
| Software | Counties Using It | API Pattern |
|---|---|---|
| JailTracker (Lexis Nexis) | 15+ FL counties | REST API at /inmate_details.aspx?id= |
| Omnixx/Tyler Odyssey | 10+ FL counties | .aspx postback forms with ViewState |
| Benchmark Analytics | Several | JSON API at /api/search |
| In-house | Varies | Custom — must reverse-engineer per site |
[2026-04-14] DrissionPage requires Chromium — always wrap import in try/except[2026-04-14] SPA/React sites render data via JS — requests gets empty shells[2026-04-14] Some sites block headless Chrome UA — use realistic User-Agent strings[2026-04-14] Rate limiting: Add 0.5-1s delays between detail page fetches[2026-04-14] Always set needs_xvfb: true in workflow for browser-based scrapersUse this skill when creating a new county arrest scraper, modifying an existing solver, or adding a new county to the scraping pipeline. Contains the canonical architecture, naming conventions, and integration checklist.
Non-negotiable rules, safe refactoring patterns, shared code modification guidelines, secrets management, config hierarchy, and schema change process for the swfl-arrest-scrapers repo. Read this before modifying any core infrastructure.
Testing patterns, fixture guidelines, parser test templates, smoke tests, and red-green TDD methodology for the scraper pipeline. Use when writing tests for new counties, verifying core/ changes, or debugging test failures.
Google Sheets: Read and write spreadsheets via the Sheets API v4.
Systematically strengthen scraper pipelines against network failures, anti-bot measures, data edge cases, and real-world brittleness that breaks idealized scraper designs.
Performance optimization patterns for Python scraper pipelines — connection pooling, memory efficiency, async I/O, and batch operations for high-throughput data ingestion.