wayback-cdx-cloud-ip-workaround

name	wayback-cdx-cloud-ip-workaround
description	Fix Wayback Machine CDX API returning empty results or timing out from cloud/datacenter IPs (Cloud Run, AWS Lambda, GCE, etc.) while working fine locally. Use when: (1) CDX queries timeout or return 0 bytes from Cloud Run/cloud functions but work from local machine, (2) urllib.request.urlopen times out for web.archive.org from server, (3) Wayback Machine CDX silently returns empty from datacenter IPs with no HTTP error, (4) Need to run CDX-dependent scripts on Cloud Run or similar cloud compute. Workaround: fetch locally, upload to cloud storage, import on cloud compute.
author	Claude Code
version	1.0.0
date	"2026-02-20T00:00:00.000Z"

Wayback CDX API: Cloud/Datacenter IP Workaround

Problem

The Wayback Machine CDX API (web.archive.org/cdx/search/cdx) silently refuses to serve data to requests from cloud provider IP ranges (Google Cloud Run, AWS, GCE, etc.). There is no HTTP error code — requests either timeout or return 0 bytes. The same queries work perfectly from residential/local IPs.

Context / Trigger Conditions

• CDX API returns empty (0 bytes) or times out from Cloud Run, Lambda, GCE, etc.
• Same query via curl on local machine returns expected data
• No HTTP error codes (not 429, not 403) — just empty or timeout