Meteor server MongoDB pool exhaustion under subscription load: oplog vs polling and observer duplication

Direct Answer

The pool exhaustion is almost certainly caused by observer duplication from non-multiplexed publications combined with poll-and-diff query storms when oplog tailing is unavailable. Each unique publication cursor (different selector, sort, options, or fields) creates a separate observe handle that holds a driver connection for its lifetime. Under subscription load, this scales connections linearly with clients rather than logarithmically. The fix priority is: (1) restore oplog access if your MongoDB tier allows it, (2) consolidate publications to enable multiplexing, (3) tune polling interval only if oplog cannot be restored, (4) increase maxPoolSize only as a last resort after verifying the above.

Confirmed Facts vs Likely Explanation

Confirmed (driver behavior)

• Meteor uses the Node MongoDB driver; maxPoolSize is set via connection string (e.g., mongodb://host/db?maxPoolSize=100). Meteor 2.x pins driver 3.6 (default pool=5); Meteor 3.x pins driver 6.x (default pool=100). Check .meteor/versions and package.json.
• Without MONGO_OPLOG_URL, Meteor falls back to poll-and-diff every _pollInterval (default 10s), issuing repeated find queries per observe handle.
• Observe multiplexing key = full cursor description (collection, selector, options, sort, limit, skip, fields). Any difference creates a new handle.
• Each observe handle checks out a connection for its lifetime; pool checkout count ≈ active observe handles.

Likely in Your Case

• Publications differ slightly per client (user-specific selectors, dynamic limits, or sort orders), preventing multiplexing.
• Reconnect storms or hot code pushes temporarily double observers before cleanup.
• Managed MongoDB tier (Atlas M0/M2/M5, Cosmos DB, etc.) does not expose oplog, forcing polling.

Steps to Confirm Pool Is the Bottleneck (Not CPU)

1. Enable driver pool event monitoring at startup:

   const { MongoClient } = require('mongodb');
   const client = new MongoClient(process.env.MONGO_URL);
   client.on('connectionPoolCreated', pool => {
     pool.on('connectionCheckedOut', () => metrics.increment('pool.checked_out'));
     pool.on('connectionCheckedIn', () => metrics.decrement('pool.checked_out'));
     pool.on('connectionPoolClosed', () => metrics.set('pool.checked_out', 0));
   });
   // Expose /metrics or log every 5s

2. Correlate at symptom time: if pool.checked_out ≈ active subscriptions and wait-queue depth grows while Node CPU < 30%, the driver pool is the bottleneck.
3. Verify with MongoDB profiling: db.setProfilingLevel(1, {slowms: 50}); count find queries on published collections per minute. Polling shows repeated identical queries; oplog shows only getMore on oplog cursor.

Measure Observer Duplication Per Publication

1. Instrument observe handle creation by wrapping Mongo.Collection._publishCursor (server-only):

   const origPublishCursor = Mongo.Collection.prototype._publishCursor;
   Mongo.Collection.prototype._publishCursor = function(cursor, sub, name) {
     const key = JSON.stringify({
       collection: this._name,
       selector: cursor.selector,
       options: cursor.options
     });
     console.log('[OBSERVE]', name, 'keyHash:', require('crypto').createHash('md5').update(key).digest('hex').slice(0,8));
     return origPublishCursor.call(this, cursor, sub, name);
   };

2. Count unique keys over a load test. If unique keys ≈ subscription count, multiplexing is failing.
3. Cross-check with cursorId: each observe handle gets a MongoDB cursorId (visible in db.currentOp() or driver cursor events). Distinct cursorIds = distinct observe handles.

Fix Prioritization and Independent Verification

One Missing Diagnostic Detail

What is your MongoDB deployment type and tier? (Atlas M0/M2/M5/M10+, self-hosted replica set, Cosmos DB, DocumentDB, etc.) This determines whether oplog access is possible and changes the entire priority order. If oplog is available, do that first and skip polling tuning. If not, publication consolidation becomes the primary lever.