Data migration and synchronization to SurrealDB from MongoDB, PostgreSQL, MySQL, Neo4j, Kafka, and JSONL. Full and incremental CDC sync. Part of the surreal-skills collection.
Installation
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Data migration and synchronization to SurrealDB from MongoDB, PostgreSQL, MySQL, Neo4j, Kafka, and JSONL. Full and incremental CDC sync. Part of the surreal-skills collection.
Surreal-Sync -- Data Migration and Synchronization
Surreal-Sync is a CLI tool for migrating data from various database sources to
SurrealDB with full and incremental synchronization via Change Data Capture (CDC).
Supported Sources
Source
Full Sync
Incremental CDC
Method
MongoDB
Yes
Yes
Change streams
MySQL
Yes
Yes
Trigger-based CDC + sequence checkpoints
PostgreSQL (triggers)
Yes
Yes
Trigger-based CDC + sequence checkpoints
PostgreSQL (wal2json)
Yes
Yes
Logical replication with wal2json plugin
Neo4j
Yes
Yes
Timestamp-based tracking
JSONL Files
Yes
N/A
Batch import from JSON Lines
Apache Kafka
Yes
Yes
Consumer subscriptions with deduplication
Quick Start
# Install surreal-sync (Rust binary)
cargo install surreal-sync
# Full sync from PostgreSQL (trigger-based)
surreal-sync from postgres trigger-full \
--connection-string "postgresql://user:pass@localhost/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
# Incremental CDC from PostgreSQL (wal2json)
surreal-sync from postgres wal2json \
--connection-string "postgresql://user:pass@localhost/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
# Full sync from MongoDB
surreal-sync from mongo full \
--connection-string "mongodb://localhost:27017/mydb" \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
# Batch import from JSONL
surreal-sync from jsonl import \
--file data.jsonl \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
# Consume from Kafka
surreal-sync from kafka consume \
--bootstrap-servers "localhost:9092" \
--topic my-events \
--surreal-endpoint "http://localhost:8000" \
--surreal-username root \
--surreal-password root \
--to-namespace prod \
--to-database main
