linkedin_webscraping

Business rules and architecture for this LinkedIn job scraper — how it scrapes last-24h posts, scores them against the candidate profile, writes a daily digest of the few best matches, and pushes real-time ntfy alerts for fresh strong matches. Use when working on scraping, scoring, the daily summary, the alert poller, session/login, or persistence in this repo.

Weekly analysis of WHEN good-matching job posts appear — day-of-week and hour-of-day distributions of high-scoring posts in the scraped corpus — so you can track shifts in posting patterns and tune when to check the digest / run the alert poller. Use when asked about best time/day to look for jobs, to refresh the timing recommendation, or to run the recurring trend report.

The scoring business logic and validation rules for the LinkedIn job digest — how a post's score is computed, what counts as a fit vs a dealbreaker, the keyword-matching gotchas, and the tuning rationale derived from analyzing the real scraped corpus. Use when reviewing or changing buckets.json weights/keywords, debugging why a post scored the way it did, or auditing match quality.

Step-by-step method for auditing the LinkedIn job heuristic's false positives (bad posts that surface in the digest) and false negatives (good posts wrongly excluded), then turning findings into safe buckets.json changes. Use when asked to "evaluate/analyze false positives and false negatives", audit match quality, find why good posts are missing or junk is surfacing, or before/after-check a scoring change. Ships audit_matches.py to automate it.

Generate the job-matching keyword config (app/scrapper/config/buckets.json) from a candidate profile markdown (e.g. eu.md / a resume). Use when onboarding a new profile, retuning match quality, or when the user provides an updated profile and wants the scoring buckets regenerated.