// Search LinkedIn posts by keywords using Crustdata API directly, deduplicate and sort by engagement. Outputs to CSV or JSON. Use when researching LinkedIn content around specific topics.
Crawl a website's sitemap and blog index to build a complete content inventory. Lists every page with URL, title, publish date, content type, and topic cluster. Groups content by category and topic. Optionally deep-reads top N pages for quality analysis and funnel stage tagging. Use before SEO audits, content gap analysis, or brand voice extraction.
Mine the highest-converting ad angles from customer reviews, Reddit complaints, support tickets, and competitor ads. Extracts actual pain language, competitor weaknesses, and outcome phrases that real buyers use. Outputs a ranked angle bank with proof quotes and recommended ad formats per angle.
Scrape competitor ads from Meta and Google ad libraries, cluster by hook/angle/format, and surface new creative directions your team hasn't tested. Chains meta-ad-scraper and google-ad-scraper. Use when a marketing team wants to understand the competitive ad landscape before launching a campaign, or wants fresh creative inspiration from what's already working in the market.
Analyze the message match between your ads and landing pages. Checks if the promise in the ad copy carries through to the landing page headline, body, and CTA. Flags disconnects that kill conversion rates. Works with Google, Meta, and LinkedIn ads.
Research a specific competitor across their website, reviews, ads, social presence, and pricing — then produce a structured sales battlecard with positioning traps, objection handlers, landmine questions, and win/loss themes. Chains web research, review mining, and ad intelligence. Use when sales needs competitive ammo or when entering a new market with established incumbents.
End-to-end champion/buyer/user job change signal composite. Takes a set of known people (past buyers, champions, power users), detects when they move to a new company, researches the new company for ICP fit, and drafts personalized outreach leveraging the existing relationship. Tool-agnostic — works with any people source, detection method, and outreach platform.
| name | linkedin-post-research |
| description | Search LinkedIn posts by keywords using Crustdata API directly, deduplicate and sort by engagement. Outputs to CSV or JSON. Use when researching LinkedIn content around specific topics. |
Search LinkedIn for posts matching keywords via the Crustdata API, deduplicate results, and output sorted by engagement.
Requires GOOSEWORKS_API_KEY environment variable (or CRUSTDATA_API_TOKEN as fallback). No external dependencies — uses Python stdlib only.
# Single keyword search
python3 skills/linkedin-post-research/scripts/search_posts.py \
--keyword "AI sourcing" \
--time-frame past-week
# Multiple keywords, output CSV
python3 skills/linkedin-post-research/scripts/search_posts.py \
--keyword "talent sourcing tools" \
--keyword "recruiting automation" \
--keyword "AI sourcing" \
--time-frame past-week \
--output csv \
--output-file results.csv
# Keywords from file, multiple pages
python3 skills/linkedin-post-research/scripts/search_posts.py \
--keywords-file keywords.txt \
--time-frame past-week \
--pages 3 \
--output json \
--output-file results.json
# Summary only (prints to stderr)
python3 skills/linkedin-post-research/scripts/search_posts.py \
--keyword "recruiting stack" \
--output summary
--keyword flags or --keywords-file (one per line)past-day, past-week, past-month, past-quarter, past-year, all-time (default: past-month)relevance or date (default: relevance)| Flag | Default | Description |
|---|---|---|
--keyword, -k | required | Keyword to search (repeatable) |
--keywords-file, -f | — | File with one keyword per line (lines starting with # are ignored) |
--time-frame, -t | past-month | Time filter |
--sort-by, -s | relevance | Sort order |
--pages, -p | 1 | Pages per keyword (~5 posts per page) |
--limit, -l | — | Exact number of posts per API call (1-100) |
--output, -o | json | Output format: json, csv, summary |
--output-file | stdout | Write output to file |
--max-workers | 6 | Max parallel API calls |
/screener/linkedin_posts/keyword_search API in parallel for all (keyword × page) combinationsbackend_urn{
"author": "Jane Smith",
"keyword": "AI sourcing",
"reactions": 142,
"comments": 28,
"date": "2026-02-20",
"post_preview": "First 200 chars of the post text...",
"url": "https://www.linkedin.com/posts/...",
"backend_urn": "urn:li:activity:123456789",
"num_shares": 12,
"reactions_by_type": "{\"LIKE\": 100, \"EMPATHY\": 30, \"PRAISE\": 12}",
"is_repost": false
}
| Column | Description |
|---|---|
| author | LinkedIn post author name |
| keyword | Which search keyword matched this post |
| reactions | Total reaction count |
| comments | Total comment count |
| date | Post date (YYYY-MM-DD) |
| post_preview | First ~200 characters of the post |
| url | Direct link to the LinkedIn post |
| backend_urn | Unique post identifier |
| num_shares | Number of shares |
Endpoint: GET https://api.crustdata.com/screener/linkedin_posts/keyword_search
Parameters:
keyword — Search termpage — Page number (1-based, ~5 posts per page)sort_by — relevance or datedate_posted — Time filter (past-day, past-week, etc.)limit — Exact number of posts to return (1-100)Auth: Authorization: Bearer <GOOSEWORKS_API_KEY> (via GooseWorks proxy) or Authorization: Token <CRUSTDATA_API_TOKEN> (direct)
Credit usage: 1 credit per post returned.
Rate limits: Searches run in parallel (default 6 workers). The script handles 429 responses with automatic retry.
| Variable | Required | Description |
|---|---|---|
GOOSEWORKS_API_KEY | Yes* | GooseWorks API key (proxies to Crustdata) |
CRUSTDATA_API_TOKEN | Fallback | Direct Crustdata API token (used if GOOSEWORKS_API_KEY is not set) |
~1 credit per post returned. Searching 10 keywords × 1 page = ~50 posts = ~50 credits.
After getting the post list, common next steps:
linkedin-commenter-extractor — pass post URLs to find warm leadslead-qualification — score extracted people against ICPExample pipeline:
# Step 1: Search posts
python3 skills/linkedin-post-research/scripts/search_posts.py \
--keyword "AI sourcing" --keyword "recruiting automation" \
--time-frame past-week --output json --output-file posts.json
# Step 2: Extract post URLs for commenter extraction
cat posts.json | python3 -c "
import json, sys
posts = json.load(sys.stdin)
# Filter: keep posts with 5+ comments
for p in posts:
if p['comments'] >= 5:
print(p['url'])
" > post_urls.txt
# Step 3: Extract commenters from those posts
while read url; do
python3 skills/linkedin-commenter-extractor/scripts/extract_commenters.py --post-url "\$url" --output json
done < post_urls.txt