OTLP Exporter Stops Sending Data When DNS Resolution Fails
25.5K reputation · 04 Mar 2023, 03:36 UTC
The OpenTelemetry OTLP exporter resolves the collector endpoint via the system DNS resolver before establishing a TLS connection. When the hostname cannot be resolved, the exporter logs an error and ceases sending telemetry until the application is restarted. Likewise, a failed TLS handshake due to an invalid or self‑signed certificate results in an immediate stop of data transmission. The exporter offers no built‑in mechanism to override DNS resolution or to disable certificate validation, aside from external environment configuration. Consequently, there is uncertainty around how the exporter should handle transient DNS failures or certificate validation errors, and whether an automatic retry or an insecure mode should be supported.
To clarify the intended behavior, we need to address the following:
- Does the OTLP exporter currently retry DNS resolution after an initial failure, and if so, what is the retry strategy?
- Is there a supported configuration to temporarily bypass TLS certificate validation, and what are the security implications?
- Should the exporter implement a configurable retry mechanism for both DNS and TLS errors to prevent silent data loss?