| name | wayback |
| description | Query and explore the Wayback Archiver personal web archiving system |
Wayback Archiver
Wayback Archiver is a self-hosted personal web archiving system that captures and preserves web pages with full fidelity — HTML, CSS, JavaScript, images, fonts, and all resources.
Repository: https://github.com/icodeface/wayback-archiver
When to use
Use this skill when the user wants to:
- Search archived web pages
- View snapshots of a specific URL
- List recent archives or filter by date range
- Get details about a specific archived page
- Read page content in Markdown format (for AI/LLM consumption)
- Explore the archiving system's data
Prerequisites
Before using this skill, ensure the Wayback Archiver server is running:
- Chrome or Firefox with Tampermonkey extension (v5.3+)
- Database (optional): SQLite (default, zero-config) or PostgreSQL 14+ (for remote deployment)
Quick Start
1. Download Pre-built Binaries
Download from Releases:
- Server binary:
wayback-server-<os>-<arch>.tar.gz (or .zip for Windows)
- Userscript:
wayback-userscript.js
Extract the server binary:
tar -xzf wayback-server-*.tar.gz
Building from source? See docs/BUILD.md.
2. Setup Database
By default, Wayback Archiver uses SQLite (zero-config, single-file database). For remote deployment with multiple users, PostgreSQL is recommended.
Option 1: SQLite (Default)
No setup required! The database file will be created automatically at ./data/wayback.db on first run.
Option 2: PostgreSQL
echo "DB_TYPE=postgres" >> .env
echo "DB_NAME=wayback" >> .env
Server startup automatically creates the configured PostgreSQL database and tables. If the database does not exist yet, the configured user must have permission to create databases.
Upgrade note for existing installs: Any installation upgrading from < 1.1.0 to 1.1.0 or later needs the pages.snapshot_state column. The server applies this automatically during startup when the database user has sufficient privileges. Otherwise, run the following idempotent SQL manually first:
ALTER TABLE pages ADD COLUMN IF NOT EXISTS snapshot_state VARCHAR(16);
UPDATE pages SET snapshot_state = 'ready' WHERE snapshot_state IS NULL OR snapshot_state = '';
ALTER TABLE pages ALTER COLUMN snapshot_state SET DEFAULT 'pending';
ALTER TABLE pages ALTER COLUMN snapshot_state SET NOT NULL;
3. Start Server
./wayback-server
Server runs at http://localhost:8080 by default.
4. Install Browser Script
- Download
wayback-userscript.js from Releases
- Open Tampermonkey dashboard
- Create new script and paste the content
- Save and enable
Chrome users: Enable "Allow user scripts" in Tampermonkey's extension settings (right-click icon → Manage extension). Firefox does not require this.
API Usage
Base URL: http://localhost:8080
Authentication (Optional)
When AUTH_PASSWORD is set, use HTTP Basic Auth:
- Username:
wayback
- Password:
$AUTH_PASSWORD
Endpoints
List All Pages
curl "http://localhost:8080/api/pages?limit=100&offset=0"
Search Pages
curl "http://localhost:8080/api/search?q=$KEYWORD"
Filter by Date Range
curl "http://localhost:8080/api/pages?from=2026-03-01&to=2026-03-12"
Get Page Details
curl "http://localhost:8080/api/pages/$PAGE_ID"
Get Page Content as Markdown
Returns the page body as clean Markdown (strips scripts, nav, footer, etc.). Ideal for AI/LLM consumption.
curl "http://localhost:8080/view/$PAGE_ID/md"
Create Public Share Link
Creates an unauthenticated link for one fixed snapshot. The returned token is only shown once.
The /view/:id snapshot header also has a Share button that creates and copies the public URL, then exposes Revoke for that share.
curl -X POST "http://localhost:8080/api/pages/$PAGE_ID/shares" \
-H "Content-Type: application/json" \
-d '{}'
Get Timeline for URL
curl "http://localhost:8080/api/pages/timeline?url=$ENCODED_URL"
View Archived Page
Open in browser:
http://localhost:8080/view/$PAGE_ID
Data Models
Page
{
id: number;
url: string;
title: string;
captured_at: string;
html_path: string;
content_hash: string;
first_visited: string;
last_visited: string;
created_at: string;
}
Resource
{
id: number;
url: string;
content_hash: string;
resource_type: string;
file_path: string;
file_size: number;
first_seen: string;
last_seen: string;
}
Storage Layout
data/
├── html/ # HTML snapshots, organized by date
│ └── 2026/03/12/
│ └── <timestamp>_<hash>.html
└── resources/ # Deduplicated static resources
└── ab/cd/
└── <sha256>.css
Features
- High-fidelity replay: CSSOM serialization, computed styles inlining
- Full-page capture: HTML, CSS, JS, images, fonts
- Cross-origin resource recovery: Server-side download of CORS-blocked resources
- Content-hash deduplication: SHA-256 based, shared resources stored once
- Version history: Multiple snapshots per URL, distinguished by timestamp
- Smart dedup: Session-level + server-level prevents redundant captures
- Dynamic content support: Captures live DOM state, auto-updates on significant mutations
- SPA-aware: Detects SPA navigation, resets capture state per route
- Anti-refresh protection: Archived pages frozen (timers, WebSockets, navigation APIs neutralized)
- Hardened cross-origin iframe capture: Embedded cross-origin documents are still captured, but bridge requests are signed and frame HTML returns over a private
MessageChannel
- Capture body guardrail:
/api/archive and /api/archive/:id reject decompressed JSON bodies above 32 MiB
- Private page filter: Browser side skips loopback, RFC1918, link-local, ULA/loopback IPv6, and
.local pages
Testing
See docs/BUILD.md for build and test instructions.
Troubleshooting
| Issue | Solution |
|---|
| Server won't start | Check database configuration; for PostgreSQL ensure it's running and database exists |
| Pages not archiving | Verify Tampermonkey script is enabled and server is reachable |
| Missing resources | Check proxy settings if behind corporate firewall |
| Authentication errors | Verify AUTH_PASSWORD env var matches your curl credentials |
Configuration
Create .env file in the project root (or set environment variables directly):
The server automatically loads .env from the working directory if it exists.
DB_TYPE=sqlite
DB_PATH=./data/wayback.db
DB_HOST=localhost
DB_PORT=5432
DB_PASSWORD=
DB_NAME=wayback
DB_SSLMODE=disable
SERVER_HOST=0.0.0.0
SERVER_PORT=8080
DATA_DIR=./data
RESOURCE_METADATA_CACHE_MB=256
AUTH_PASSWORD=
ENABLE_DEBUG_API=false
License
MIT