| name | apify-research-connector |
| description | Use when a project needs Apify-powered web, social, SERP, marketplace, or competitor research with credential, compliance, and dataset controls. |
Apify Research Connector
Use Apify as a credential-gated data adapter for research, not as a bypass for
platform rules. Keep credentials in ignored env stores and keep outputs
provenance-rich.
Context Gate
Operate only from the current project folder. Read local docs, .env.example,
package scripts, existing Apify modules, and Brain-1 wiki pages. If project
identity or credential ownership is unclear, stop with a blocked report.
Credential Gate
Check presence without printing values:
git check-ignore -v .env.local .env 2>/dev/null || true
awk -F= '/^APIFY_(API_TOKEN|API_KEY)=/{print $1"=present"}' .env.local 2>/dev/null || true
vercel env ls --scope unite-group 2>/dev/null | awk '/APIFY|apify/{print $1, $2, $3, $4, $5}'
Preferred variable is APIFY_API_TOKEN. Treat APIFY_API_KEY as legacy or
project-specific until verified. Never echo, log, screenshot, commit, or paste
token values.
Official Docs Gate
Refresh current Apify docs before adding or changing integration behavior:
- API overview:
https://docs.apify.com/api
- Run Actor and retrieve data:
https://docs.apify.com/academy/api/run-actor-and-retrieve-data-via-api
- JavaScript client:
https://docs.apify.com/api/client/js
- Dataset API:
https://docs.apify.com/api/v2/default-dataset
- Apify API page:
https://apify.com/api
Operating Pattern
- Define the research question, platform, and business use.
- Select
mode: official_actor, custom_actor, licensed_vendor,
manual_review, or blocked.
- Check actor page/docs, input schema, pricing, usage limits, and terms.
- Prefer existing project service modules and scripts before adding code.
- Run the smallest verification that proves auth and actor compatibility.
- For short runs, use Apify client
.call() or sync endpoints; for longer
runs, use async run plus polling or webhooks.
- Store structured outputs in datasets; use key-value stores for single files,
images, PDFs, or
OUTPUT.
- Capture provenance: actor ID, input, run ID, dataset ID, access date,
source URLs, platform terms checked, and limitations.
- Feed findings into wiki/content/SEO/YouTube skills with clear source labels.
Safety Rules
- No CAPTCHA bypass, logged-in scraping, paywall bypass, forbidden platform
scraping, personal-data enrichment, or terms evasion.
- Do not use Apify to scrape Google Search directly when Search Console,
official APIs, licensed vendors, or manual review are the correct route.
- Use official platform APIs when a platform requires them.
- Treat Store actors as adapters with their own docs and commercial terms.
- Emit a blocked JSON/report when credentials, rights, consent, rate limits, or
actor schemas are missing.
Verification
Low-cost auth check:
node -e "const fs=require('fs'); const key='APIFY_'+'API_TOKEN'; const s=fs.readFileSync('.env.local','utf8'); const m=s.match(new RegExp('^'+key+'=(.*)$','m')); process.exit(m?0:2)"
Live API identity check without printing token:
TOKEN=$(node -e "const fs=require('fs'); const key='APIFY_'+'API_TOKEN'; const s=fs.readFileSync('.env.local','utf8'); const m=s.match(new RegExp('^'+key+'=(.*)$','m')); if(!m) process.exit(2); let v=m[1].trim().replace(/^['\\\"]|['\\\"]$/g,''); process.stdout.write(v)")
curl -fsS -H "Authorization: Bearer $TOKEN" https://api.apify.com/v2/users/me \
| jq '{username: .data.username, idPresent: (.data.id != null)}'
Do not run paid actors until the expected cost and approval gate are explicit.
Output Contract
Return:
- project and credential status, masked
- Synthex/Vercel env coverage when relevant
- mode and actor/task selected
- docs and terms checked
- run/dataset IDs, if created
- output location
- compliance blockers
- wiki/catalog updates
