OpenTelemetry Collector and backend data consistency during restoration events
0 reputation · 19 Feb 2025, 00:13 UTC
The OpenTelemetry Collector operates as a stateless intermediary, exporting telemetry via OTLP to various storage backends like ClickHouse or Prometheus. While the collector handles data buffering and retry logic, it lacks native mechanisms to verify the integrity of data sets after a backend-side backup restoration has occurred.
Because the collector typically defaults to at-least-once delivery semantics, there is uncertainty regarding potential data duplication or sequence gaps when a backend is restored from a snapshot and the collector resumes transmitting buffered batches. The framework does not provide a cross-backend validation layer to ensure the recovered state matches the original exported OTLP stream.
- How does the OpenTelemetry Collector handle data re-transmission following a backend-side recovery to prevent duplicate metrics?
- Are there specific configurations within the collector to verify the integrity of a restored dataset against the OTLP export history?