Collector dropped span error on Jaeger Collector
0 reputation · 03 Sept 2021, 15:03 UTC
Goal: Determine why the Jaeger Collector reports "spans dropped" metrics when the system is under concurrent request load. The Collector ingests spans via UDP or HTTP, buffers them, and writes to a backend such as Cassandra or Elasticsearch. Under high traffic, memory pressure or increased backend latency can cause the internal buffer to overflow, leading to span loss.
Constraints: The Collector’s drop policy is not exposed as a simple flag; it depends on runtime memory usage, the chosen storage backend, and the sampling strategy in place. It is unclear whether the drop threshold is fixed or adaptive, and how UDP packet loss interacts with the drop metric. The behavior appears to differ between HTTP and UDP ingestion paths, yet the documentation does not clarify the exact conditions.
Unresolved questions:
- What internal threshold or policy triggers span drops in the Jaeger Collector?
- How does the choice of storage backend (Elasticsearch vs. Cassandra) affect the drop rate under concurrent load?
- Is there a configuration knob to adjust the buffer size or drop policy, and what are the trade‑offs?