MongoDB Change Detection: Choosing Between Change Streams, Polling, and Tailable Cursors

The decision you actually face

When an application must react to MongoDB writes—invalidating caches, updating search indexes, or triggering downstream workflows—the first question is not which driver method to call. It is which change-detection mechanism your deployment topology can support and how much staleness your consumers can tolerate. Pick the wrong mechanism and you either cannot start the watcher, silently miss events, or add avoidable read load.

Decision and hard constraints

Use change streams when you need near-real-time, resumable reaction to inserts, updates, and deletes on a replica set or sharded cluster. Use indexed polling only on a standalone deployment, when latency equal to your poll interval is acceptable, or when you cannot keep a long-lived cursor open. Use tailable cursors only for append-only feeds on capped collections; they are a legacy pattern that sees inserts and nothing else.

Change streams carry non-negotiable prerequisites:

• A replica set or sharded cluster is required. A single-node replica set is the minimum; a standalone mongod cannot open a change stream.
• WiredTiger storage is expected.
• Resumability is bounded by the oplog window. If a consumer stays offline longer than the oplog retains matching entries, the stream cannot resume.
• Update events include the full post-image document only when you set fullDocument: 'updateLookup'. Delete events never include a document.

Confirm your topology first

Before implementing, verify you are not on a standalone. In mongosh, run db.hello() (or the legacy db.isMaster()) as a user with read access. If the response includes setName and a hosts array, you are on a replica set. If those fields are missing, the node is standalone and change streams are unavailable.

Supported options at a glance

Mechanism	Minimum topology	Event coverage	Latency	State to persist	Primary risk
Change streams	Replica set or sharded cluster (single-node replica set minimum)	Insert, update, replace, delete	Near-real-time	Resume token	Oplog rollover raises ChangeStreamHistoryLost (code 246)
Indexed polling	Any, including standalone	Any queryable change	Equals poll interval	Last-seen version or timestamp	Duplicate scans and missed concurrent updates
Tailable cursor	Replica set with capped collection	Inserts only	Near-real-time	Cursor position	No update or delete events; legacy

Trade-offs that matter in production

Change streams move cost to the server through oplog tailing and require you to persist resume-token state. In exchange they avoid duplicate scans and do not miss writes while the oplog entries survive. The operational burden is managing that token and handling the 246 error.

Polling is stateless and simple. You query an indexed lastModified or version field on a schedule. The trade-off is read load proportional to poll frequency, plus the classic ordering and deduplication problem: two updates between polls can be collapsed, and a slow query can skip or repeat documents unless you combine strict ordering with idempotent processing.

Tailable cursors block on a capped collection and return new documents as they arrive. They are unsuitable for anything but append-only feeds because updates and deletes produce no events.

Concrete implementation: a resumable watcher

The following pattern is illustrative; adapt it to your driver and confirm the current API surface against your server and driver release notes.

// Pseudocode — run inside your consumer process
const pipeline = [
  { $match: { operationType: { $in: ['insert', 'update', 'replace', 'delete'] } } }
];

const options = {
  fullDocument: 'updateLookup'
};

// Load the last persisted resume token from <resumeTokenStore>
const resumeToken = loadResumeToken();

const stream = db.collection('<collection>').watch(pipeline, options);

stream.on('change', async (event) => {
  // Process idempotently by documentKey._id
  await applyDownstream(event);

  // Persist the resume token ONLY after the downstream write commits
  saveResumeToken(event._id);
});

stream.on('error', async (err) => {
  if (err.code === 246) {
    // ChangeStreamHistoryLost: oplog window exceeded
    await runFullReconciliation();
  } else {
    throw err;
  }
});

Key points in this pattern:

1. Open one watch() per consumer. Each stream holds a long-lived cursor, which consumes a connection and server memory.
2. Persist the resume token after the downstream side effect commits, not before. If you crash between the database write and the token save, you will replay an event; if you save first and crash before the write, you will lose it.
3. Make downstream processing idempotent by documentKey._id. Replays and polling duplicates both become safe.
4. On ChangeStreamHistoryLost (code 246), fall back to a full reconciliation scan keyed by a version or lastModified field. This is the only safe recovery when the oplog no longer contains the gap.

Validation you can run

On a three-node replica set, verify behavior before trusting the consumer in production.

1. Event fidelity

Perform an insert, an update, and a delete against <database>.<collection>. Confirm the watcher receives exactly one event per operation with the expected operationType and documentKey. For the update, confirm the post-image document arrives because fullDocument is set to updateLookup.

2. Resume behavior

Apply writes, stop the consumer, apply more writes, then restart with resumeAfter set to the last saved token. Confirm the gap replays exactly once and no earlier events repeat.

3. Oplog boundary

Pause the consumer long enough for the oplog to roll over, or trigger a primary step-down. Confirm the client raises ChangeStreamHistoryLost and that your reconciliation fallback executes.

4. Cost comparison

Run the same write workload against a 30-second indexed polling loop and the change stream. Compare end-to-end p99 latency and oplog growth. Polling avoids server-side cursor state but pays in read load and staleness.

Operational guardrails

• Grant the watcher role only changeStream and read privileges on the target database. Do not reuse an administrative account.
• Monitor the oplog window—typically visible via db.serverStatus().metrics.oplog in mongosh—so you know the maximum safe consumer downtime.
• Budget one long-lived cursor per consumer. A fleet of watchers multiplies connection and memory pressure.

Limitations and version sensitivity

Change stream behavior is version-sensitive. Pre- and post-image documents require enabling documentPreImages on the collection and their availability varies by server version. Exact read-concern requirements, driver APIs, and sharded-cluster behavior also differ across releases. Before rollout, confirm your specific server and driver versions against current MongoDB documentation. The constraints above reflect well-established behavior, but the operational details—especially around image documents and sharded clusters—warrant a version check in your environment.