find-twitter-influencers

# Find Twitter Influencers ## Setup Read your credentials from ~/.gooseworks/credentials.json: ```bash export GOOSEWORKS_API_KEY=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json'))['api_key'])") export GOOSEWORKS_API_BASE=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json')).get('api_base','https://api.gooseworks.ai'))") ``` If ~/.gooseworks/credentials.json does not exist, tell the user to run: `npx gooseworks login` All endpoints use Bearer auth: `-H "Authorization: Bearer $GOOSEWORKS_API_KEY"` Discover, score, and enrich Twitter/X influencers relevant to a company, product, or niche. Returns a ranked list with engagement metrics, relevance reasoning, and contact info. ## Workflow ### 1. Parse the Request Extract from the user's query: - **Company name or domain** (required) — the brand seeking influencers - **Niche/vertical** (optional) — e.g., "fintech Twitter", "AI/ML creators", "DTC beauty" - **Size preference** (optional) — mid-tier (10K-100K), macro (100K+), or mixed (default: 10K+ minimum) - **Max results** (optional, default 20 — scale up or down if the user asks) ### 2. Resolve the Company Use Brand.dev to get domain, industry, description, target audience, and keywords. This context drives all subsequent searches. ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"brand-dev","path":"/v1/brand/retrieve-by-name","query":{"name":"Acme","Corp":""}}' ``` If a domain is provided directly: ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"brand-dev","path":"/v1/brand/retrieve","query":{"domain":"acme.com"}}' ``` From the result, build a **company context string**: company name, domain, industry, description, and target audience keywords. Example: `"Acme Corp acme.com developer tools API platform engineering teams"`. Use this in all search queries. ### 3. Discover Candidates Run three strategies **in parallel** to maximize coverage: **Strategy A — Exa search for curated influencer lists** (primary, highest signal): IMPORTANT: x.com and twitter.com profiles are NOT in Exa's search index, so `includeDomains: ["x.com"]` will return zero Twitter results. Instead, search for curated list pages, blog posts, and articles about influencers in the niche. Use `contents.text` to get the page content so you can extract Twitter handles from it. Run 5+ query variations to maximize coverage. Include the **core niche** AND **adjacent niches** — many influencers span related topics. Each query returns 10 results with text content: ```bash # Query 1: Core niche — curated lists curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/search"}' "query": "best {niche} Twitter accounts to follow", "numResults": 10, "contents": {"text": {"maxCharacters": 5000}} }' # Query 2: Core niche — different phrasing curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/search"}' "query": "top {industry} influencers on X Twitter must follow", "numResults": 10, "contents": {"text": {"maxCharacters": 5000}} }' # Query 3: Core niche — thought leaders curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/search"}' "query": "{niche} thought leaders creators Twitter handles", "numResults": 10, "contents": {"text": {"maxCharacters": 5000}} }' # Query 4: Adjacent niche 1 (e.g., if niche is "identity verification", try "fraud prevention") curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/search"}' "query": "top {adjacent_niche_1} influencers Twitter accounts to follow", "numResults": 10, "contents": {"text": {"maxCharacters": 5000}} }' # Query 5: Adjacent niche 2 (e.g., "cybersecurity", "regtech", "biometrics") curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/search"}' "query": "best {adjacent_niche_2} experts creators on X Twitter", "numResults": 10, "contents": {"text": {"maxCharacters": 5000}} }' ``` **Choosing adjacent niches**: From the company context (Step 2), identify 2-3 related verticals. Examples: - Identity verification → fraud prevention, cybersecurity, regtech, biometrics - Fintech → banking, payments, DeFi, financial regulation - AI/ML → data science, MLOps, developer tools - DTC beauty → skincare, wellness, lifestyle, clean beauty These queries return listicle pages (e.g., "Top 60 Fintech Influencers", "FinTwit Accounts to Follow") whose text content contains Twitter handles, bios, and follower counts. Parse these in Step 4. **Strategy B — Exa findSimilar** (expand from strong listicle finds): After Strategy A returns results, pick 1-2 of the best curated list URLs and find similar pages: ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/findSimilar"}' "url": "https://example.com/top-fintech-twitter-influencers", "numResults": 5, "contents": {"text": {"maxCharacters": 5000}} }' ``` This surfaces additional curated lists that keyword search may miss. Do NOT use Twitter/X profile URLs for findSimilar — they are not in Exa's index and will return empty results. **Strategy C — Fiber natural-language-search** (catch LinkedIn-heavy professionals): Some influencers are better indexed on LinkedIn but have active Twitter accounts. Fiber can surface these. Run 2-3 queries covering the core niche and adjacent topics: ```bash # Core niche curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"fiber","path":"/v1/natural-language-search/profiles"}' "query": "{niche} thought leader content creator with Twitter presence at {industry} companies", "pageSize": 20 }' # Adjacent niche — broader coverage curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"fiber","path":"/v1/natural-language-search/profiles"}' "query": "{adjacent_niche} influencer expert with large social media following", "pageSize": 20 }' ``` Cross-reference Fiber results with Twitter in Step 5 — only keep people with active Twitter accounts. Also extract LinkedIn URLs from Fiber results — these are critical for contact enrichment in Step 7. ### 4. Extract & Deduplicate Usernames From the **text content** of Exa listicle pages, parse Twitter handles using multiple patterns: - `@username` mentions in the article text - URLs matching `x.com/username` or `twitter.com/username` - Strip trailing URL paths (e.g., `/status/123`, `/followers`) — only keep base profile usernames - Discard non-profile patterns (search pages, hashtag pages, `x.com/home`, `x.com/search`) - Deduplicate by username (case-insensitive) - Flag and remove obvious brand/company accounts (e.g., `@stripe`, `@shopify`) — focus on individual creators From Fiber results, extract any Twitter/X URLs from social profiles. Add new handles to the candidate pool. Also extract LinkedIn URLs mentioned alongside Twitter handles in listicle pages — save these for contact enrichment in Step 7. Target: ~100-150 unique handles after dedup. Curated list pages typically mention 20-50 handles each, so 5+ good listicle results across core and adjacent niches yield a large candidate pool. ### 5. Get Twitter Profiles + Engagement Use Scrape Creators to fetch structured Twitter data. This is a two-step process: fetch profiles for ALL candidates (cheap, 1 credit each), then fetch tweets for the top ~2x the target result count after profile filtering (e.g., ~40 if targeting 20 results). **Step 1 — Fetch profiles for all candidates:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"scrapecreators","path":"/v1/twitter/profile","query":{"handle":"examplehandle"}}' ``` Returns nested JSON. Key fields are inside `core` and `legacy` objects: - `core.screen_name`, `core.name` — handle and display name - `legacy.description` — bio text - `legacy.followers_count`, `legacy.friends_count`, `legacy.statuses_count` — counts - `legacy.location`, `legacy.created_at` — location and account age - `is_blue_verified` — verification status The profile URL is `https://x.com/{screen_name}`. **IMPORTANT — Parallelize profile fetches:** Do NOT fetch profiles one-by-one in a sequential loop. Instead, launch ALL profile fetch calls in parallel (multiple tool calls in a single message). This dramatically reduces total wait time from minutes to seconds. Group into batches of 10-15 parallel calls if needed. Apply **hard filters** to narrow the pool: - Fewer than 10,000 followers (default minimum — adjustable if user specifies a different threshold) - Empty bio or bio completely unrelated to the niche - Suspended or not-found accounts (API returns error) - Bio indicates they've left X (e.g., "find me on Mastodon/Bluesky", "abandoned this site") - **Too broad/generic accounts** — Remove accounts whose content is not specifically relevant to the target niche. For example, a general "tech evangelist" account with 500K+ followers that rarely tweets about the niche is less valuable than a 20K-follower specialist. If an account's bio and recent tweets show no specific connection to the target niche or adjacent niches, exclude it regardless of follower count **Step 2 — Fetch tweets for top candidates** (after profile filtering — fetch ~2x the target result count to allow for filtering, e.g., ~40 if targeting 20 results): ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"scrapecreators","path":"/v1/twitter/user-tweets","query":{"handle":"examplehandle"}}' ``` Returns an array of tweet objects. **IMPORTANT — The data is nested inside each tweet object:** - `legacy.full_text` — tweet text - `legacy.favorite_count` — likes (integer) - `legacy.retweet_count` — retweets (integer) - `legacy.reply_count` — replies (integer) - `legacy.created_at` — timestamp - `views.count` — view count (may not always be present) - `url` — direct link to the tweet (top-level field) Note: `favorite_count`, `retweet_count`, `reply_count` are inside the `legacy` object, NOT at the top level of each tweet. **IMPORTANT — Parallelize tweet fetches:** Just like profile fetches, launch ALL tweet fetch calls in parallel (multiple tool calls in a single message). Do NOT use sequential for-loops. This is the single biggest speed optimization. For each candidate, identify the **top 3 tweets by engagement** (likes + retweets + replies). Save the best one with its URL for inclusion in the final results table. From the tweet data, calculate: - **Average likes** per tweet - **Average retweets** per tweet - **Average replies** per tweet - **Engagement rate**: (avg likes + avg retweets + avg replies) / followers * 100 - **Post frequency**: inferred from `created_at` dates - **Content themes**: what topics they tweet about most (from `full_text`) **Additional hard filters** (applied after tweet fetch): - No tweets in the last 30 days (inactive) - Protected/private accounts Skip reply-only accounts (>80% of tweets are replies to others with minimal engagement). ### 6. Score & Rank Apply a composite scoring model: | Factor | Weight | Signal | |--------|--------|--------| | **Relevance** | 40% | Bio keywords, content themes, audience overlap with target company | | **Engagement rate** | 25% | Higher is better; micro-influencers often outperform macro here | | **Follower count** | 15% | Log-scaled — diminishing returns above 100K | | **Content quality** | 10% | Original content vs retweets, thread depth, media usage | | **Audience alignment** | 10% | Do their followers match the company's target audience? Inferred from bio + content themes | **Scoring guidelines:** - Relevance: Compare bio and recent tweet topics against the company context string from Step 2. Exact niche match = high score. Adjacent niche = medium. Generic/unrelated = low. - Engagement rate benchmarks: >3% excellent, 1-3% good, <1% below average (varies by follower tier) - Content quality: Penalize accounts that are >50% retweets. Reward original threads, insights, and media-rich posts. Rank all candidates by composite score. Select the top N (default 20) for the final list. 20 is a good default — enough to give the user real options without overwhelming them. Scale up if the user asks for more, but always include at least the top 20 if that many qualify. ### 7. Enrich Contacts For the final list, find email addresses and LinkedIn profiles. The key insight: **discover LinkedIn URLs first** — they dramatically improve match rates for all enrichment APIs. **Step 1 — Collect what you already have**: From previous steps, gather: