Sync Only What Matters With CouchDB _changes Filters

The problem: syncing without polling every document

You have a CouchDB database and a downstream service—a search index, a cache, or an analytics pipeline—that needs to stay current. Polling all documents on a schedule wastes bandwidth, and pulling the full _changes feed forces your application to ignore most of what arrives. CouchDB's _changes endpoint lets you stream document updates as they happen, and server-side JavaScript filters let the database discard irrelevant changes before they cross the network.

The practical takeaway: open a continuous _changes feed, apply a filter defined in a design document, and save the last sequence ID so the feed can resume after a restart without replaying old events.

How the continuous feed delivers changes

CouchDB assigns every committed write an update sequence number. GET /{db}/_changes returns a JSON stream where each line describes a document change: its ID, revision history, and a deleted flag when a document is removed. Adding feed=continuous keeps the HTTP connection open so CouchDB pushes new changes as they are committed instead of closing after the first batch.

Because the stream is ordered by sequence, you can restart a client later and pass since={last_seq} to receive only changes newer than the last one you processed. This makes the feed resumable, provided you persist that sequence value somewhere durable.

Filtering changes on the server

A filter is a JavaScript function stored in a design document. CouchDB runs it for each change and sends the event to the client only when the function returns true. For example, you can subscribe only to documents in the invoices category or only to non-deleted drafts.

Filters reduce client-side work and network traffic, but they execute on the CouchDB server for every change in the database. Under high write throughput, a complex filter can become a CPU bottleneck. Keep filter logic simple and avoid database lookups inside the function.

Worked example: filtered, resumable sync

Run these commands in a terminal where curl can reach your CouchDB instance, typically http://localhost:5984. You need read access to the target database; in an authenticated deployment, include credentials with -u username:password.

curl -X PUT http://localhost:5984/mydb

Save the filter definition as invoice_filter.json:

{
  "_id": "_design/invoice_sync",
  "filters": {
    "invoices_only": "function(doc, req) { return doc && doc.type === 'invoice'; }"
  }
}

Create the design document:

curl -X PUT http://localhost:5984/mydb/_design/invoice_sync -H "Content-Type: application/json" --data @invoice_filter.json

Start the continuous filtered feed from the beginning:

curl -N http://localhost:5984/mydb/_changes?feed=continuous&filter=invoice_sync/invoices_only&since=0

The -N flag disables output buffering so each JSON change prints as it arrives. Each line looks like this:

{"seq":42,"id":"invoice-1001","changes":[{"rev":"3-8a9f..."}],"deleted":false}

Record the seq value from the last event you process. If the client restarts, resume with that value:

curl -N http://localhost:5984/mydb/_changes?feed=continuous&filter=invoice_sync/invoices_only&since=42

Expected check: after resuming, CouchDB should emit only changes committed after sequence 42. If you see older events, your since value was not persisted correctly.

Trade-offs and operational limits

• Eventual consistency: CouchDB replicates and commits asynchronously. Concurrent writes to the same document can create conflicting revisions, and your sync logic must decide how to resolve them.
• Connection management: A continuous HTTP connection can drop silently. Handle heartbeats, detect disconnects, and reconnect with the last known since value.
• Filter cost: Because filters run server-side on every change, test them under production-like write volume before relying on them for a high-throughput database.

Verify before you depend on it

Start a local CouchDB instance, create a test database, write several documents, and request the filtered continuous feed. Capture last_seq, stop the feed, write more documents, restart with since=last_seq, and confirm only newer changes appear. Review the documentation for your installed CouchDB major version, because _changes parameters and filter semantics can vary between releases.

If your downstream system can tolerate brief delays and you need selective, resumable updates, this pattern is a practical alternative to polling or full-feed ingestion.