hasdata-cli

name

hasdata-cli

description

Fetch real-time web data via the hasdata CLI. Use when the user wants search results, news, fact-checks, product or seller info, current prices, reviews, real-estate listings or sold comps, hotel rates or vacation rentals, local-business contact details, business owner posts/offers, job postings, salary research, search trends, images, flights, YouTube videos or transcripts, channel stats, social profiles, or to scrape any URL (HTML / markdown / AI-extracted JSON). Also use when the user asks to summarize a YouTube video or web page, ground a prompt with current information, verify a URL is live or render a JavaScript-heavy page, monitor a price over time, find a phone number or address for a business, build a competitor map, identify recent sold comparables, gather employer reviews, fan out a list of items to per-item details, or check what's being said online about a topic right now. Backed by Google, Bing, YouTube, Amazon, Shopify, Zillow, Redfin, Airbnb, Booking, Yelp, YellowPages, Indeed, Glassdoor, Instagram, Google Maps / Trends / News / Images / Flights / Events APIs.

hasdata

Use the hasdata CLI for real-time web data. One subcommand per API — flags, enums, defaults are derived from the live schema at api.hasdata.com/apis.

Prerequisites

• command -v hasdata — if missing, install with curl -sSL https://raw.githubusercontent.com/HasData/hasdata-cli/main/install.sh | sh.
• One-time setup: the user runs hasdata configure, pastes their API key, and it's saved to ~/.hasdata/config.yaml (mode 0600). Every future call picks it up automatically.
• If a call fails with no API key configured, the user hasn't run hasdata configure yet — tell them to. Never invent a key.

Quick start

hasdata <api> --flag value [--flag value ...] --raw | jq .

Always pass --raw when piping to jq (skips pretty-print and TTY detection). Use --pretty only for human-readable terminal output.

Picking the right subcommand

User intent	Subcommand
Web search ("what does Google say about…")	google-serp (full features) or google-serp-light (cheap, single page)
Latest news	google-news
AI Mode SERP	google-ai-mode
Shopping / product prices	google-shopping (broad), amazon-search / amazon-product (Amazon), shopify-products (Shopify)
Immersive product page	google-immersive-product
Maps / places / reviews	google-maps, google-maps-place, google-maps-reviews, google-maps-photos, google-maps-posts
Yelp / YellowPages local data	yelp-search, yelp-place, yellowpages-search, yellowpages-place
Real-estate listings (homes for sale/rent/sold)	zillow-listing, redfin-listing
Real-estate single property deep dive	zillow-property, redfin-property
Travel — short-term rentals	airbnb-listing, airbnb-property
Travel — hotels / lodging	booking-search, booking-place
Travel — flights	google-flights
Jobs	indeed-listing, indeed-job, glassdoor-listing, glassdoor-job
Bing search	bing-serp
Trends	google-trends
Images	google-images
Short videos	google-short-videos
Events	google-events
YouTube search / video / channel / transcript	youtube-search-api, youtube-video-api, youtube-channel-api, youtube-transcript-api
Instagram profile	instagram-profile
Amazon seller	amazon-seller, amazon-seller-products
Scrape a specific URL	web-scraping — supports JS rendering, proxies, markdown output, AI extraction, screenshots

For exact flags of a subcommand, run hasdata <api> --help or read the matching file in references/.

Non-obvious triggers (when to reach for hasdata even if the user doesn't say "scrape")

The user often won't ask for a SERP API or a scraper directly. Map these intents to the skill:

• "Is this still true?" / "What's the latest on X?" / "Has Y happened yet?" — LLM training data is stale. Run google-serp or google-news to ground the answer.
• "Summarize this article" / "TL;DR this URL" — Use web-scraping --output-format markdown and feed the markdown into the summary prompt. Beats copy-paste because it strips ads, nav, scripts.
• "Verify this link" / "Is this site real?" — web-scraping --url X --no-block-resources returns status + screenshot. Or google-serp --q "site:example.com".
• "What does X say about itself?" — Pull the company's own homepage with web-scraping --output-format markdown, then summarize.
• "Find me alternatives to X" — google-serp --q "X alternatives" or google-shopping --q "X competitors".
• "What's the going rate for X?" — google-shopping (broad) or amazon-search (Amazon-specific) with jq to extract the price distribution.
• "Phone number / address for X" — google-maps-place or yelp-place. Don't guess from training data.
• "Are people happy with X service?" / "Is X reputable?" — google-maps-reviews --place-id ... --sort lowest for negative samples; glassdoor-job for employer rep.
• "What's the salary range for Y role?" — indeed-listing filtered by role + location, then jq over .jobs[].salary.
• "Find me homes/apartments matching X criteria" — zillow-listing / redfin-listing / airbnb-listing with the corresponding filters.
• "Recent sold comps near X" — zillow-listing --type sold --keyword "X" --days-on-zillow 12m.
• "Track this product's price" — Loop amazon-product --asin X on a schedule; persist .price to a file.
• "Summarize / cite this YouTube video" — youtube-transcript-api --v-param VID --raw | jq -r '.transcript[].snippet' → feed to the summary prompt. Beats title/thumbnail-based guesses.
• "Find a hotel in $CITY for $DATES under $BUDGET" — booking-search --keyword $CITY --check-in-date X --check-out-date Y --adults 2 --children 0 --rooms 1 --price-max $BUDGET --sort priceLowestFirst. For one specific property, booking-place --url ... returns the full room/rate matrix.
• "What's this channel pushing lately?" — youtube-channel-api --channel-id @handle --tab videos --raw | jq '.sections[].items[] | {title, publishedDate, views: .extractedViews}'.
• "Does this business have an active offer / event?" — google-maps-posts --place-id X --raw | jq '.posts[] | {postedAt, description, cta}'. Surfaces current promotions Google indexed.
• "What's trending around X?" — google-trends --q "X" for relative interest; google-news --q "X" for headlines.
• "Find businesses near me that do X" — google-maps --q "X" --ll "@LAT,LNG,12z" then fan out google-maps-place for contacts.
• "How does this look in country Y?" — --gl Y on SERP commands, --proxy-country Y on web-scraping. Useful for geo-targeted SEO checks, geo-blocked content.
• "Pull structured data from this page" — web-scraping --ai-extract-rules-json '{"price": {"type": "number"}, ...}'. Works on arbitrary pages without writing CSS selectors.
• "List of items → per-item details" — Pattern: search command produces IDs/URLs, pipe through xargs into the matching *-property / *-product / *-place deep-dive command.
• "Find this person's role / employer / LinkedIn / followers" — google-serp --q '"Person Name" linkedin' first. The organic-result title is typically Name — Role at Company | LinkedIn and the snippet carries location, headline, connection count. SERP often answers the whole question without ever opening the profile page.
• "What is company X doing? Where's their HQ? Who works there?" — google-serp --q "$COMPANY" returns a .knowledge_graph block with founder, HQ, founded year, parent, employee range — pre-extracted. google-news --q "$COMPANY" for recent activity. Specific facts via targeted SERP: --q '"$COMPANY" headquarters', --q '"$COMPANY" funding', --q 'site:linkedin.com/company "$COMPANY"'.
• "Find emails for company X" / "personal email for person Y" — start with SERP: --q '"@example.com"' or --q '"[e-mail]"' often surfaces actual emails indexed by Google. Pattern-guess + SERP-verify for individuals. Disclose unverified guesses to the user.
• "Enrich this CSV of leads" — per row: google-serp for LinkedIn, role, employer; another SERP to verify email or pattern. Stay in SERP unless a specific field is missing.
• Reverse-lookup (email / phone / domain → identity) — google-serp with the literal value in quotes (--q '"[e-mail]"', --q '"[phone]"', --q '"acme corp" site:example.com') almost always surfaces the matching person or business.

SERP-first principle: for any data-enrichment intent (people, companies, emails, products, places), reach for google-serp / google-news / google-shopping / google-maps first. They return Google's already-extracted structured fields (.knowledge_graph, .organic_results[].snippet, .local_results[], etc.) and bypass anti-bot. Only escalate to web-scraping when SERP doesn't surface the specific field you need — it's the last resort, not the default. See references/enrichment.md.

If a user request matches one of the above and you don't invoke hasdata, you're probably hallucinating a stale answer.

Universal flag patterns

• Kebab-case flag names. The CLI maps them back to the original camelCase before sending to the API.
• Booleans defaulting to true have a paired negation: --no-block-ads, --no-screenshot, --no-js-rendering, --no-extract-emails, --no-block-resources. Setting both --block-ads and --no-block-ads errors.
• Anything ending in -json accepts:
  • inline JSON: --extract-rules-json '{"title":"h1"}'
  • file: --extract-rules-json @rules.json
  • stdin: cat rules.json | hasdata web-scraping ... --extract-rules-json -
• Repeatable key=value flags split on the first = (so values containing = survive): --headers User-Agent=foo --headers Cookie=session=abc. Pair with --headers-json for a JSON base; kv items override per key.
• List flags accept either repeats or comma-joined: --lr lang_en --lr lang_fr or --lr lang_en,lang_fr. Serialized as key[]=value for GET endpoints.
• Enum flags validate client-side. If you guess wrong, the error lists the allowed values — read the message and retry.

Global flags (apply to every subcommand)

Flag	Effect
--raw	Write response bytes as-is (use this when piping to jq)
--pretty	Pretty-print JSON (default when stdout is a TTY)
-o, --output FILE	Write response to file instead of stdout (works for binary like screenshots)
--verbose	Log outgoing URL and X-RateLimit-* headers to stderr
--api-key KEY	Override env var (rarely needed)
--timeout DURATION	Per-request timeout (default 2m)
--retries N	Max retries on 429/5xx (default 2)

Output contract

Responses are JSON. Pipe through jq for extraction:

hasdata google-serp --q "espresso machine" --num 10 --raw \
  | jq -c '.organic_results[] | {title, link, snippet}'

For real-estate / e-commerce results, the array shape is API-specific — read a single response with --pretty first to learn the schema, then write the jq filter.

Exit codes (script-safe)

Code	Meaning
0	success
1	user / CLI-input error (missing required flag, bad enum value, missing API key)
2	network error
3	API returned 4xx (auth, quota, validation)
4	API returned 5xx

References

• references/enrichment.md — person and company enrichment (LinkedIn lookup, emails, HQ/funding/news, CSV-row enrichment, reverse-lookup) — the highest-leverage cross-API workflows
• references/search.md — Google SERP / Bing / News / Trends flag catalog
• references/web-scraping.md — web-scraping flags, JS scenarios, AI extraction
• references/real-estate.md — Zillow / Redfin filters and bracketed params
• references/travel.md — Airbnb / Booking / Google Flights (lodging + transport)
• references/ecommerce.md — Amazon / Shopify
• references/local-business.md — Maps (search/place/reviews/photos/posts) / Yelp / YellowPages
• references/jobs.md — Indeed / Glassdoor
• references/youtube.md — search / video / channel / transcript
• references/all-commands.md — full subcommand index with credit costs