Bright Data Deploy Integration
Overview
Deploy Bright Data scraping applications to cloud platforms with proper secrets management. Key consideration: Bright Data proxy connections require outbound TCP to brd.superproxy.io on ports 33335 (proxy) and 9222 (Scraping Browser).
Prerequisites
- Bright Data production zone credentials
- Platform CLI installed (vercel, fly, or gcloud)
- Application tested in staging
Instructions
Step 1: Vercel Deployment (Serverless)
vercel env add BRIGHTDATA_CUSTOMER_ID production
vercel env add BRIGHTDATA_ZONE production
vercel env add BRIGHTDATA_ZONE_PASSWORD production
vercel env add BRIGHTDATA_API_TOKEN production
{
"functions": {
"api/scrape.ts": {
"maxDuration": 60
}
}
}
import type { VercelRequest, VercelResponse } from '@vercel/node';
import axios from 'axios';
import https from 'https';
export default async function handler(req: VercelRequest, res: VercelResponse) {
const { url } = req.body;
if (!url) return res.status(400).json({ error: 'url required' });
const proxy = {
host: 'brd.superproxy.io',
port: 33335,
auth: {
username: `brd-customer-${process.env.BRIGHTDATA_CUSTOMER_ID}-zone-${process.env.BRIGHTDATA_ZONE}`,
password: process.env.BRIGHTDATA_ZONE_PASSWORD!,
},
};
try {
const response = await axios.get(url, {
proxy,
httpsAgent: new https.Agent({ rejectUnauthorized: false }),
timeout: 55000,
});
res.json({ status: response.status, length: response.data.length });
} catch (error: any) {
res.status(502).json({ error: error.message });
}
}
Step 2: Fly.io Deployment (Long-Running)
app = "my-scraper"
primary_region = "iad"
[env]
NODE_ENV = "production"
[http_service]
internal_port = 3000
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
fly secrets set BRIGHTDATA_CUSTOMER_ID=c_abc123
fly secrets set BRIGHTDATA_ZONE=web_unlocker_prod
fly secrets set BRIGHTDATA_ZONE_PASSWORD=z_prod_pass
fly secrets set BRIGHTDATA_API_TOKEN=prod_token
fly deploy
Step 3: Google Cloud Run
echo -n "c_abc123" | gcloud secrets create brightdata-customer-id --data-file=-
echo -n "web_unlocker_prod" | gcloud secrets create brightdata-zone --data-file=-
echo -n "z_prod_pass" | gcloud secrets create brightdata-zone-password --data-file=-
gcloud run deploy scraper \
--image gcr.io/$PROJECT_ID/scraper \
--region us-central1 \
--set-secrets=BRIGHTDATA_CUSTOMER_ID=brightdata-customer-id:latest \
--set-secrets=BRIGHTDATA_ZONE=brightdata-zone:latest \
--set-secrets=BRIGHTDATA_ZONE_PASSWORD=brightdata-zone-password:latest \
--timeout=120 \
--memory=512Mi
Step 4: Platform Comparison
| Feature | Vercel | Fly.io | Cloud Run |
|---|
| Max timeout | 60s (Pro: 300s) | No limit | 3600s |
| WebSocket (Scraping Browser) | No | Yes | No |
| Outbound TCP | Yes | Yes | Yes |
| Best for | Web Unlocker API | Scraping Browser | Batch scraping |
| Cold start | Fast | Configurable | Medium |
Step 5: Health Check Endpoint
export async function GET() {
try {
const proxy = { host: 'brd.superproxy.io', port: 33335, auth: { username: `brd-customer-${process.env.BRIGHTDATA_CUSTOMER_ID}-zone-${process.env.BRIGHTDATA_ZONE}`, password: process.env.BRIGHTDATA_ZONE_PASSWORD! } };
const start = Date.now();
const res = await axios.get('https://lumtest.com/myip.json', { proxy, httpsAgent: new (await import('https')).Agent({ rejectUnauthorized: false }), timeout: 15000 });
return Response.json({ status: 'healthy', proxy_ip: res.data.ip, latency_ms: Date.now() - start });
} catch {
return Response.json({ : }, { : });
}
}
Output
- Platform-specific deployment with secrets management
- Health check endpoint verifying proxy connectivity
- Timeout and memory configured for scraping workloads
Error Handling
| Issue | Cause | Solution |
|---|
| Vercel timeout | 60s limit too short | Upgrade to Pro (300s) or use Fly.io |
| WebSocket refused | Platform blocks WS | Use Fly.io for Scraping Browser |
| Cold start timeout | Proxy handshake slow | Configure min instances |
| Secrets not found | Wrong env name | Verify with platform CLI |
Resources
Next Steps
For webhook handling, see brightdata-webhooks-events.
