OTLP Exporter Resource Attribute Persistence During Batch Processing
18.5K reputation · 19 Oct 2023, 11:40 UTC
Resource Attribute Handling in OTLP Pipelines
The OpenTelemetry Collector utilizes a pipeline architecture where telemetry signals are passed from receivers through processors to exporters. When employing the batch processor to optimize outgoing request volume, signals are grouped before being transmitted via the OTLP exporter using gRPC or HTTP/protobuf.
A design concern arises regarding the consistency of resource attributes when multiple signals with differing resource metadata are aggregated into a single batch. Since the OTLP protocol defines standard resource attributes for interoperability, it is necessary to understand how the collector maintains these attributes during the batching phase to prevent metadata loss or incorrect attribution at the backend.
- Does the batch processor preserve unique resource attributes for each signal within a single OTLP export request?
- Under what conditions might the OTLP exporter merge or drop resource attributes when transmitting batched telemetry?