Collaboration & Sync

Offline-First Sync Engine

Local edits that apply instantly and reconcile when online: every device a leader, causality tracked with vector clocks, conflicts merged deterministically by CRDTs, and convergence guaranteed by eventual consistency.

~35 min · advanced

Problem & Requirements

Let a client read and write its data while offline, then reconcile with other clients once a connection returns. Each device holds a full local replica and applies edits to it immediately, with no round trip — the app stays responsive on a plane or a subway. When the network comes back, replicas exchange what changed and end up agreeing. Two devices may have edited the same field while both were offline, so the engine has to merge divergent histories rather than assume one canonical sequence of writes.

That last fact is the whole design. There is no primary to serialize writes, because writes happen offline against whichever replica is at hand, so every replica is a leader — this is multi-leader replication. With no central order, convergence cannot come from a lock or a single authority; it has to come from the data itself. Causality between edits is tracked with vector clocks so the engine can tell a sequential edit from a concurrent one, the values are CRDTs so any two replicas that have seen the same edits compute the same merged state regardless of arrival order, and the system targets eventual consistency: available and writable while partitioned, converging once edits are delivered. The spine throughout is that merge is a pure function of state, not a decision made by a coordinator.

Functional

set(key, value) / get(key) apply to the local replica immediately, offline.
Edits are captured as operations that can be shipped to and replayed on other replicas.
sync(peer) exchanges changes bidirectionally; after delivery, all replicas hold equal state.

Non-functional (back-of-envelope)

Quantity	Target	What it forces
Local write latency	< 1 ms	Writes hit the local replica; sync is asynchronous and off the write path
Offline duration	hours to weeks	Edits buffer locally; merge must handle long divergence, not just brief gaps
Replicas per document	small (2–50)	Vector-clock size scales with replica count, so the count cannot be unbounded
Sync payload	O(changes), not O(state)	A since-cursor delta exchange, never a full-state push
Conflict rate	low but nonzero	Every concurrent-edit case must converge, not only the common ones
Convergence	all replicas equal once delivered	Order-independent, deterministic merge — a CRDT, not ad-hoc resolution
Metadata overhead	bounded over time	Op log and tombstones compacted once edits are causally stable

The dangerous row is the conflict rate. Edit volume is easy — most edits touch different keys and merge trivially. The system lives or dies on the concurrent edit to the same field, because that is where two replicas can silently end up holding different state while both believe they are synced. Making that case converge deterministically is build step four, and it is the difference between a sync engine and a data-corruption bug that only appears after a flight.

Design

Six components, each tied to the principle it applies:

Local replica with optimistic writes — every device holds the full dataset and accepts writes locally with no coordination, which is multi-leader replication taken to its limit: the replica count equals the device count and any of them can write.
Operation log — edits are recorded as operations with metadata, not just overwritten state, so they can be shipped, replayed, and merged rather than clobbered.
Vector clocks — each replica carries a vector clock and stamps every operation, so the engine distinguishes "B causally followed A" from "A and B were concurrent." This is vector clocks.
CRDT values — registers, sets, and counters whose merge is commutative, associative, and idempotent, so two replicas that have seen the same operations converge regardless of order. This is CRDTs.
Delta sync protocol — a bidirectional push/pull keyed on the peer's vector clock, exchanging only operations the peer has not seen. This is multi-leader replication on the wire, driving toward eventual consistency.
Compaction and convergence — once an operation is causally stable across all replicas, the log and its tombstones are collected, which bounds metadata while preserving the eventual consistency guarantee.

The named systems split into two families, and blurring them hides the central tradeoff. The build here follows the peer-convergence path — the same model as Automerge and Yjs — where replicas merge each other's operations with no authoritative server in the loop. Couchbase Lite is a multi-master variant of this: an embedded database that replicates with revision trees and resolves a conflict deterministically by revision id (its CouchDB lineage), surfacing the loser for optional app resolution. Replicache and Linear take the other path, server-authoritative reconciliation: a client applies a mutation optimistically against its local store, the server replays mutations in a canonical order and becomes the source of truth, and the client rebases its optimistic state onto that order — the technique borrowed from game netcode. Replicache pulls incremental patches against a server-issued cursor; Linear keeps a local normalized object graph synced through a transaction log with last-write-wins at property granularity. The peer-CRDT path needs no server to converge but bakes merge semantics into the data types; the server-reconciliation path gets simpler merges and a single order at the cost of requiring the server to be reachable to finalize truth.

Build it

Start with a store that writes locally and syncs by copying state. set mutates an in-memory map immediately, and sync ships the whole map both directions. It satisfies the offline-write requirement and is correct for a single device. Its failure appears the moment two devices edit while disconnected: copying state means the later copy overwrites the earlier one wholesale, so one device's edits vanish, and there is no way to tell a real sequence from two concurrent edits. The next step captures edits as operations instead of overwriting state.

Record each edit as an operation (key, value, timestamp), append it to a log, and sync by replaying the peer's operations rather than copying state. Applying by timestamp lets edits to different keys both survive, and it makes every replica a writer whose changes propagate as a stream — multi-leader replication in its plainest form. The failure is in the timestamp: wall clocks drift and disagree across devices, so "later" is a lie, and a tie or a genuinely concurrent edit is indistinguishable from a sequential one. The next step replaces wall-clock ordering with real causality.

Stamp every operation with a vector clock — a per-replica counter map — so the engine can compare two edits and decide whether one causally preceded the other or they happened concurrently. Incrementing the replica's own entry on each write and merging clocks on sync gives an ordering that survives clock skew, because it counts events rather than reading a clock. This is vector clocks. The failure that remains is subtle: detecting that two edits are concurrent is not the same as resolving them, and an ad-hoc choice made independently on each replica can leave them holding different values. The next step makes resolution deterministic.

Make the value a CRDT so its merge is a pure function of the two states — commutative, associative, and idempotent — and concurrent edits resolve the same way on every replica. A last-write-wins register breaks ties between concurrent writes by a total order on (causal position, replica id), which is arbitrary but identical everywhere, so no two replicas can disagree. This is CRDTs, and it is the fix for the silent divergence the requirements flagged: convergence now follows from the type, not from a coordinator. Collections use an OR-Set (add and remove tagged by unique ids, with tombstones) for the same reason. The failure left is cost — every sync still ships the entire log and all state.

Shipping every operation on every sync is O(state) and collapses as the log grows. Use the peer's vector clock as a cursor: send only operations the peer has not already seen, in both directions, then merge clocks. The clock that tracks causality doubles as the "what have you seen" marker, so the exchange is exactly the missing delta — multi-leader replication reduced to its minimum payload, converging toward eventual consistency. The remaining failure is unbounded growth: operations and OR-Set tombstones never leave the log, so metadata climbs forever even after everyone agrees.

An operation that every replica has already seen can never affect a future merge, so it is safe to drop — but only once that is true everywhere. Compute a low-water mark from the vector clocks of all replicas (the minimum counter each replica has acknowledged), and collect any operation and any tombstone that sits at or below it. This bounds metadata without breaking convergence: the kept operations still reconstruct the agreed state, and that agreement is the eventual consistency guarantee — once edits are delivered and stable, all replicas hold equal state with bounded overhead.

class Store:
    def __init__(self):
        self.state = {}                       # key -> value

    def set(self, key, value):
        self.state[key] = value               # local, immediate, offline

    def get(self, key):
        return self.state.get(key)

    def sync(self, peer):
        peer.state.update(self.state)         # whole-state copy: later write
        self.state.update(peer.state)         # clobbers the other's edits

Tradeoffs

Decision	What it buys	What it costs
Multi-leader replication	Writes anywhere, fully offline; no coordinator on the write path	Concurrent conflicts are unavoidable and must be merged; no global order
Vector clocks	Exact causality; concurrency detected, not guessed	Size grows with replica count and must be garbage-collected
CRDTs	Order-independent convergence with no authority	Merge semantics are fixed by the type; metadata (tags, tombstones) rides along
LWW conflict resolution	Always converges, trivial to reason about	Silently drops the losing concurrent edit rather than merging intent
Delta sync over a cursor	O(changes) payload instead of O(state)	Needs per-peer cursor state and correct clock bookkeeping
Compaction at causal stability	Bounded metadata, AP availability	A long-offline replica pins the log; reads may be stale until sync

Scaling it up

The build follows the peer-CRDT path; the other family is worth implementing precisely to feel the tradeoff. Replicache and Linear are server-authoritative: the client applies mutations optimistically against a local store, the server replays them in one canonical order and becomes the source of truth, and the client rebases its pending state onto whatever the server decided. That removes the need for CRDT merge — the server's order is the merge — at the price of requiring the server to be reachable to finalize anything and accepting last-write-wins semantics. Couchbase Lite sits between the two, replicating multi-master with revision trees and a deterministic revision-id winner while exposing conflicts for app-level resolution. Which family fits depends on whether true peer-to-peer convergence without a server is a requirement or an over-engineering.

Text is its own hard problem the toy ducks. A last-write-wins register is wrong for collaborative prose, because two people typing in the same paragraph need their characters interleaved with intention preserved, not one edit discarding the other. Sequence CRDTs — RGA, Yjs's YATA, Automerge's approach — assign stable identifiers to positions so concurrent insertions order deterministically without colliding. Building one is a separate prototype, and it is where most of the real CRDT engineering effort goes.

Vector clocks scale poorly when replicas are ephemeral. A clock is O(replicas), and if every browser tab or short-lived session is its own replica the clocks bloat and never shrink. Production systems assign stable client ids from a server, prune version vectors once clients are retired, or use dotted version vectors to track per-update provenance compactly. The server-reconciliation systems sidestep this entirely by issuing a single monotonic order from the server, which is one of the quieter reasons that model is popular.

Persistence, bootstrap, and partial replication are where an engine meets a real dataset. The local replica lives in IndexedDB or SQLite and must survive reloads; a new client needs an initial bootstrap of possibly-large state before deltas make sense; and most clients should sync only the slice they can see rather than the whole dataset. Linear syncs a workspace subset and lazily loads the rest, Replicache scopes the client view by what the user subscribes to, and both fold per-row authorization into sync so a client never receives rows it may not read.

Schema migration and undead garbage are the long-tail operational pain. The data model evolves while old offline clients still hold the previous shape, so operations have to be migratable on replay across versions. And a tombstone cannot be collected until every replica has acknowledged it, which means a single permanently-offline device pins metadata indefinitely, forcing a policy for evicting replicas that never come back. From here the concrete follow-ons are a sequence-CRDT text-editing prototype that builds the interleaving the LWW register can't, a server-reconciliation prototype that builds the Replicache/Linear optimistic-rebase model as the counterpoint to this one, and a partial-replication and authorization prototype that syncs only an authorized slice with bootstrap and incremental pull. Each extends the same multi-leader, eventual-consistency foundation without re-deriving it.

References

Shapiro, Preguiça, Baquero, Zawirski, Conflict-Free Replicated Data Types (2011), and the INRIA tech report A Comprehensive Study of Convergent and Commutative Replicated Data Types (RR-7506): https://hal.inria.fr/inria-00609399 · https://hal.inria.fr/inria-00555588
Fidge (1988) and Mattern (1989) on vector clocks / logical time in distributed systems — the causality primitive used here.
Kleppmann, McGranaghan, et al., Local-first software: You own your data, in spite of the cloud (Ink & Switch, 2019): https://www.inkandswitch.com/local-first/
Automerge and Yjs — production CRDT libraries, including sequence CRDTs for text: https://automerge.org/ · https://docs.yjs.dev/
Couchbase Lite — replication and conflict resolution with revision trees: https://docs.couchbase.com/couchbase-lite/current/c/conflict.html
Replicache — server reconciliation and incremental pull: https://doc.replicache.dev/concepts/how-it-works
Artman, Scaling the Linear Sync Engine — local store, transaction log, server as source of truth: https://linear.app/blog/scaling-the-linear-sync-engine
Foundations referenced inline: multi-leader replication · vector clocks · CRDTs · eventual consistency