| name | sync-protocols |
| description | Design delta-sync protocols for mobile with cursor-based pagination and durable checkpoints. Use when building server-to-client data sync for offline-capable apps. |
Sync Protocols (Delta Sync)
Instructions
Mobile apps with offline support need a protocol that lets them catch up efficiently after long silences. A good sync protocol is boring: it is a cursor, a delta, and a checkpoint, repeated.
1. Model
Each syncable entity type has:
- A monotonic server-side revision number (
rev), incremented on every write.
- A
deleted_at column (soft deletes survive delta sync).
- A stable
id.
The client stores per-entity-type:
- Its last seen server
rev (the checkpoint).
- A queue of local pending mutations to send.
2. Pull Endpoint
GET /v1/sync/articles?since=4812&limit=500
{
"changes": [
{ "op": "upsert", "rev": 4813, "entity": { "id": "a_01", "title": "...", "updated_at": "..." } },
{ "op": "delete", "rev": 4814, "id": "a_02" }
],
"next_cursor": "4814",
"has_more": false,
"server_time": "2026-04-15T09:00:00Z"
}
Rules:
since is the client's checkpoint. Server returns changes with rev > since, ordered ascending.limit defaults 500, max 2000.has_more: true means the client paginates immediately (same request with new since).- The client only updates its checkpoint to
max(rev) after successfully applying the page. On crash mid-apply, the next sync replays the same page.
Postgres-backed implementation:
const limit = Math.min(Number(req.query.limit ?? 500), 2000);
const since = Number(req.query.since ?? 0);
const rows = await db.query(
`SELECT id, rev, deleted_at, data
FROM articles
WHERE rev > $1
ORDER BY rev ASC
LIMIT $2 + 1`,
[since, limit],
);
const hasMore = rows.length > limit;
const page = rows.slice(0, limit);
res.json({
changes: page.map(r => r.deleted_at
? { op: "delete", rev: r.rev, id: r.id }
: { op: "upsert", rev: r.rev, entity: r.data }),
next_cursor: String(page.at(-1)?.rev ?? since),
has_more: hasMore,
server_time: new Date().toISOString(),
});
3. Push Endpoint
POST /v1/sync/articles
{
"mutations": [
{ "op": "upsert", "client_id": "cli_1", "base_rev": 4812,
"entity": { "id": "a_03", "title": "Draft" } },
{ "op": "delete", "client_id": "cli_2", "base_rev": 4812, "id": "a_04" }
]
}
Each mutation carries:
client_id for idempotency (the client retries safely).base_rev = the revision the client saw before making this change (for conflict detection; see conflict-resolution-backend).
Response includes the assigned server rev per accepted mutation and any conflicts:
{
"results": [
{ "client_id": "cli_1", "status": "ok", "rev": 4815, "id": "a_03" },
{ "client_id": "cli_2", "status": "conflict", "current_rev": 4810, "current_entity": { "...": "..." } }
]
}
4. Monotonic Revision Assignment
The server's rev must be strictly increasing across all writes of a given entity type (not per-entity). Options:
- Postgres
BIGSERIAL / GENERATED ALWAYS AS IDENTITY.
- Redis
INCR per table.
- Hybrid logical clock for multi-region (rare; usually eventual global
rev is good enough).
Never reuse a revision. A rollback writes a new revision that reverses state.
5. Deletes
Soft delete by setting deleted_at and bumping rev. The sync stream emits a delete op. Periodic vacuum removes rows with deleted_at < now() - 90 days, but clients must have checkpointed past that rev first.
6. Full Resync / Reset
If the gap is too large or the client was offline past the vacuum horizon, return:
{ "changes": [], "reset": true, "server_time": "..." }
The client wipes local state for this entity type, sets since = 0, and pulls again.
7. Multi-Entity Sync
Prefer one endpoint per entity type. A "sync everything" endpoint couples unrelated entities and bloats payloads.
For a single-round-trip sync screen, an aggregator (BFF) can call the per-type endpoints in parallel and return a composite.
8. Client Consumption (Kotlin Multiplatform-style pseudocode)
suspend fun syncArticles() {
var since = store.checkpoint("articles")
while (true) {
val page = api.pull("/v1/sync/articles?since=$since&limit=500")
db.transaction {
page.changes.forEach { apply(it) }
store.setCheckpoint("articles", page.nextCursor.toLong())
}
since = page.nextCursor.toLong()
if (!page.hasMore) break
}
}
Apply inside a transaction so checkpoint advancement is atomic with change application.
9. Backpressure and Scheduling
- Sync on app foreground, on network regain, and on silent-push hints.
- Respect
Retry-After if the server signals overload.
- Throttle background sync to once every 5–10 minutes unless prompted.
10. Observability
Track, per entity type: bytes/sync, changes/sync, has_more ratio (too high means limit too low), and reset rate (too high means retention too short).
Checklist
