Behavior of OpenTelemetry Collector batch processor flush during shutdown with limited timeout

Short answer

With the default in-memory pipeline, batches that are still in the batch processor or in-flight to the exporter when the Collector's graceful shutdown deadline expires are dropped. The OTLP exporter at-least-once guarantee applies only to exports that complete acknowledgement before shutdown. Enabling the storage extension can make exporter sending-queue items survive a restart, but it does not automatically guarantee persistence of items still inside the batch processor's internal buffer at the moment of shutdown, and the guarantee is version and configuration dependent.

Confirmed vs likely

Well-established behavior

• On shutdown signal the Collector stops receivers and attempts to drain pipelines. The batch processor triggers a final flush of accumulated items.
• The drain is bounded by the process graceful shutdown deadline. Once that deadline passes the process exits and in-memory queues are lost.
• Exporter retry with exponential backoff and jitter runs only while the process is alive. If shutdown occurs during a backoff wait, the in-flight batch is not retried after exit.

Likely but version-dependent, needs verification

• Exactly how long the batch processor is allowed to flush relative to the overall shutdown deadline is not uniformly documented across Collector versions and distributions.
• Interaction between the shutdown timeout, the batch processor's own flush interval, and the exporter's retry max interval is implementation specific and has changed across releases.
• Storage extension persistence applies to the exporter's sending queue when configured for persistent storage. It does not automatically persist the batch processor's internal memory buffer. Whether items handed from batch to a persistent sending queue are durably written before the shutdown deadline expires depends on write timing and configuration.

Sizing the shutdown timeout

Practical sizing is conservative: the graceful shutdown window should be large enough to cover a worst-case export attempt, i.e., a typical export request latency plus the maximum retry backoff interval plus margin. If the window is shorter, a batch that is waiting in backoff at shutdown time will be lost unless it is already persisted by a storage-backed sending queue.

Without persistent storage, no timeout value prevents loss when the backend is unavailable at shutdown time.

Steps needed for this case

1. Confirm Collector version and distribution. Behavior of batch flush and storage extension has shifted across releases.
2. Check whether a persistent storage extension is attached to the exporter sending queue and whether the queue is enabled. Verify logs for queue drain and timeout exceeded messages during shutdown.
3. Observe shutdown logs and metrics for batch send size and exporter queue size to see if the drain completes before the deadline.

One missing diagnostic detail that changes the recommendation: which Collector version and distribution you are running, and whether persistent storage is configured on the exporter sending queue versus the batch processor. That determines whether queued items can survive the shutdown timeout at all.