source-himalayas

name	source-himalayas
description	Himalayas source node for GARY — paginate ALL match pages and scrape each job's full JD via CDP (Cloudflare auto-clears ~4s → poll until clear), validate fit by description not slug. Use when sourcing Himalayas. A GARY skill by Julián Nicholls (@jnichollsc).
metadata	{"author":"Julián Nicholls (@jnichollsc)"}

Himalayas source (by @jnichollsc)

Candidate-agnostic. Canonical rules: docs/operating-rules.md; per-board runbook: docs/career-ops-map.md §4.

Engine: node engines/himalayas-matches.mjs over the logged-in automation browser (CDP debug Chrome — port/profile are DATA in config/profile.yml → job_search.automation_browser; never the user's personal browser). Paginate /jobs/matches?page=1..N, map every match, then navigate each job and scrape the FULL JD. Himalayas is an aggregator: apply links OUT to the company ATS (auto-fillable like any external ATS). Validate fit by description, not slug. Candidate facts come from config/profile.yml + the NotebookLM RAG. Never click Submit. Flags: --max-pages N, --no-desc, --fresh.

1. BLOCKS / anti-bot

• Cloudflare "Just a moment…" guards every job DETAIL page (the matches LIST loads fine). It auto-clears ~4s in the real logged-in browser, so the engine POLLS the page (re-check title/content every ~2s, up to ~16s) and extracts ONLY once the challenge clears. A short fixed wait captures the challenge page — that was the original bug (208 "Just a moment" captures).
• 429 / rate-limit → anti-block cadence: MAX 2 concurrent JD fetches + ~2s gap between batches. Runs serial-ish on the CDP browser.
• The public /jobs/api is NOT a workaround — it returns only the latest ~20 GLOBAL jobs (offset-paginated), ignores limit/company, has ZERO overlap with personalized matches, and per-job .json is itself Cloudflare-blocked. CDP + poll-until-clear is the only path to personalized-match JDs.
• Closed/expired → a page that never clears past the challenge or renders no JD is left uncaptured (resumable retry), never scored on the title.

2. AUTOMATION

• Core CDP loop (agent-browser): open <url> → snapshot -i (@eN refs) → click/fill @eN → re-snapshot -i (refs go stale). Never close/close --all; to move on, open the next URL.
• Resumable: output/gmail-harvest/himalayas-jobs.json is rewritten after every batch; a re-run skips jobs already captured with a real (non-blocked) description, so a timeout never loses progress. Runs in the background (~8–10 min for ~200 jobs).
• Apply = external ATS → auto-fillable: node engines/apply-from-linkedin.mjs "<job-or-apply-url>" "<cvPath>" follows the external tab (Greenhouse/Ashby/Workable/JazzHR/etc.), fills safe fields + EEO + standard answers, and advances to Submit and STOPS. Captcha + final Submit stay the user's.

3. PAGINATION

• Traverse the WHOLE set: open /jobs/matches?page=1, read the MAX page number from the pagination control, then loop ?page=1..N mapping every match {url,title,company} (dedup by url across pages). The matches list is multi-page (e.g. ~11 pages) — title-only filtering misses offers.
• ≤1-week early-stop: Himalayas has no built-in date filter → apply the recency window yourself at the triage step from the card/JD date.
• Card-pattern pre-filter: map all matches first (cheap), then spend the expensive poll-until-clear JD read only on the deduped set.

4. WEBFETCH FILTERING

• Primary validation is via logged-in CDP (poll-until-clear), NOT WebFetch — Cloudflare blocks WebFetch/.json for detail pages.
• WebFetch is only the fallback when CDP is unavailable: fetch the JD page directly at the same cadence (≤2 concurrent / ~2s gap).
• Rule: read the real JD, never the slug/title alone. Fit is decided on each job's captured description in himalayas-jobs.json against the two-way-fit gate (eligibility / comp / YOE / no-AWS-hardreq / company-dedup / posted ≤1 week), then merged into the master doc.