CouchDB architecture: single-leader writes and edge replication

The problem and the practical takeaway

CouchDB replicates documents over HTTP using the _changes feed, but it does not automatically merge JSON documents updated independently on two nodes. When the same document ID receives different updates on separate replicas, CouchDB stores both as conflicting leaf revisions. A read can return either revision until an application picks a winner and deletes the losers. For most eventually consistent sync workloads, the smallest reliable design is a single write leader with one-way or continuous replication to read replicas and edge nodes, plus application-level conflict resolution.

Requirements

• Sync documents between a central node and edge or mobile replicas without requiring a shared schema.
• Detect and resolve conflicting updates to the same document ID.
• Protect replication credentials and document payloads in transit.
• Detect stalled replication and growing conflict counts before they affect reads.

This note assumes CouchDB 3.x. The _rev field is an MVCC token: writes must include the current token, and CouchDB rejects stale revisions with HTTP 409 Conflict.

Smallest suitable design

Route all writes to one leader node. Configure continuous pull replication from edge nodes to the leader, or push from the leader to edge nodes, depending on which side can initiate the connection. Do not enable bidirectional write replication between two nodes that both accept writes for the same document IDs.

Example replication document, submitted to the _replicator database:

{
  "_id": "edge-to-leader-replication",
  "source": "https://EDGE_HOST:5984/mydb",
  "target": "https://LEADER_HOST:5984/mydb",
  "create_target": false,
  "continuous": true
}

Where to run: any host with network access to both CouchDB instances and permission to write to _replicator. Required permissions: server admin, or a database user with write access to _replicator. Meaningful placeholders: EDGE_HOST and LEADER_HOST. Expected check: query _active_tasks and see the replication task move from running to steady checkpoint progress, or completed for non-continuous replication. Risk: "continuous": true causes the replicator to retry indefinitely after errors, which can repeatedly load the source and target; monitor task errors and use backoff where supported.

Equivalent curl request, run from an admin-capable shell:

curl -X PUT "https://LEADER_HOST:5984/_replicator/edge-to-leader-replication" \
  -H "Content-Type: application/json" \
  -u ADMIN_USER:ADMIN_PASSWORD \
  -d '{"_id":"edge-to-leader-replication","source":"https://EDGE_HOST:5984/mydb","target":"https://LEADER_HOST:5984/mydb","create_target":false,"continuous":true}'

Expected response: {"ok":true,"id":"...","rev":"..."}. Re-run only with the current _rev; an obsolete revision returns HTTP 409.

Trust and data boundaries

Replication credentials can read every document in the source database and write to the target. Treat a replication user as having full access to the replicated database. Use TLS for all replication URLs, restrict replication to trusted networks or VPNs, and authenticate with dedicated database users rather than admin credentials. Replication copies documents and attachments but does not enforce the source's validation functions on the target; the target applies its own validators, if any.

Operational checks

1. Inspect _active_tasks for replication entries. A healthy continuous task stays running with increasing checkpoint values; a one-shot task ends at completed.
2. Query the _replicator database to confirm replication documents exist and are not repeatedly erroring.
3. Compare the _changes feed sequence on source and target. A widening gap indicates replication lag.
4. Count documents containing a _conflicts array. A rising count means independent writes are colliding faster than the resolver removes losing revisions.

Diagnostic decision: if _active_tasks shows a replication task in an error state repeatedly, verify TLS certificate validity and network reachability before increasing retry frequency.

Failure modes

• Network partition: replicas diverge and conflicts accumulate until connectivity returns.
• Unresolved conflicts: reads return non-deterministic winners, so application logic may observe stale or unexpected fields.
• Large attachments: one large attachment can stall a replication batch and block checkpoint progress for other documents.
• Full disk: writes halt on the leader or target, and replication fails with HTTP 500 or connection errors.

When to change the design

Change the design if the workload requires strong consistency, multi-document transactions, or frequent conflicting writes to the same IDs. Alternatives include a single leader with synchronous clients, a data store with native consensus, or an application merge service that reconciles writes before they reach CouchDB.

Limitations and verification

CouchDB does not auto-merge JSON. Revision limits and compaction behavior are version-sensitive; confirm defaults for the deployed CouchDB major version before tuning revs_limit. To verify conflict behavior locally, create the same document ID on two replicas, update both independently, replicate, and inspect the target document for a _conflicts array. To verify revision rejection, issue a PUT with an obsolete _rev and expect HTTP 409; repeat with the current _rev and expect success.

Creating or replacing a replication document changes state. To roll it back, send DELETE to /_replicator/{doc_id} with the current _rev. This stops future replication but does not remove documents already copied to the target.