Jaeger in Production: Treat Tracing as a Pipeline, Not a UI Install

You pull the all-in-one Jaeger image, open the Query UI, and watch traces arrive. Everything looks solved. Then the container restarts, the traces are gone, or a production-sized span rate turns the UI into a loading spinner. The problem is not that Jaeger is hard to install; it is that the demo path hides the fact that production tracing is a pipeline decision. The Query UI is a read surface, not the system of record. Before you scale, make ingestion, durable storage, retention, and sampling explicit choices.

All-in-one memory storage is a demo trap

The all-in-one binary bundles the collector, query interface, and storage into one process. In Jaeger v1 this model also included a separate agent; v2 reorganized deployment around a single OpenTelemetry-based binary. Either way, the default development storage is often in-memory or single-node Badger. In-memory storage vanishes on restart. Badger writes to local disk but does not give you a distributed, replicated system of record. Treat both as development or small-scale options and verify what your specific Jaeger version supports before relying on them.

The UI looks healthy because it reads from the same process that just accepted the spans. That tells you nothing about durability, throughput, or retention.

Ingestion: prefer OTLP over legacy Jaeger and Zipkin transports

Jaeger historically accepted spans through Jaeger-native Thrift or Protobuf over HTTP, and it also supported Zipkin ingestion. Today the practical default is the OpenTelemetry Protocol, or OTLP. OTLP is the vendor-neutral wire format used by OpenTelemetry SDKs and the OpenTelemetry Collector. Jaeger v2 is built around OTLP natively; v1 collectors can accept it as well.

Choosing OTLP matters because most new instrumentation targets OpenTelemetry. If you instrument with an OpenTelemetry SDK, the exporter sends OTLP. Do not assume that service will poll a legacy Jaeger remote sampling endpoint; those strategies apply to Jaeger clients, not to OTLP exporters. Confirm whether your environment uses the v1 collector/agent model or the v2 single-binary model before copying port numbers or flags.

Durable storage: Cassandra versus Elasticsearch/OpenSearch

Once you accept that traces must survive restarts and scale, you need an external backend. The two common choices are Cassandra and Elasticsearch/OpenSearch. They optimize for different problems.

Dimension	Cassandra	Elasticsearch / OpenSearch
Write profile	Predictable at high throughput; append-oriented.	Good, but heavier per write due to indexing.
Ad hoc query	Deliberate; you need to know partition keys and schema.	Flexible search across attributes and tags.
Retention mechanics	TTL per row or scheduled cleanup.	Index lifecycle policies automate rollover and deletion.
Operational cost	Capacity planning, repair, and compaction.	JVM heap, shard management, and cluster tuning.

Elasticsearch and OpenSearch are usually easier when humans need to search traces by arbitrary attributes. Cassandra is more predictable when write volume is the dominant constraint. OpenSearch support, index templates, and lifecycle behavior are version-sensitive; verify compatibility with your Jaeger release and storage version before committing.

Sampling and retention are your volume controls

Even with durable storage, raw span volume can overwhelm your budget. Head-based sampling decides at the start of a trace, usually in the SDK or the Collector. It is simple and low-overhead, but it can discard an interesting error before the error happens. Tail-based sampling buffers a complete trace in the Collector and decides afterward, so you can keep errors or slow traces. The cost is memory and queue pressure; an undersized tail-sampling stage will drop traces under load.

Retention is the other dial. How long you keep traces must match what your backend can enforce. High-cardinality attributes—user IDs, full URLs, unbounded IDs—dominate storage cost. Set attribute limits and sampling before you increase retention.

Worked example: an OpenTelemetry Collector pipeline into Jaeger

The following Collector configuration accepts OTLP, batches spans, and exports them to a Jaeger collector over OTLP. Save it as otel-collector-config.yaml and run it from a shell on a host that can reach both your instrumented services and the Jaeger collector. No special OS permissions are required unless you bind privileged ports below 1024.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s

exporters:
  otlp/jaeger:
    endpoint: http://<jaeger-collector-host>:<otlp-port>
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/jaeger]

Start the Collector with otelcol --config otel-collector-config.yaml. Replace <jaeger-collector-host> and <otlp-port> with the values advertised by your Jaeger deployment; ports and receiver names vary by version. Then emit a known trace: a parent span, a child span, and an error span from your instrumented service.

To verify, check three places. First, the Collector logs should show the OTLP receiver registered and the exporter connected. Second, Jaeger Query should display the trace with the correct service name, operation name, attributes, and parent/child relationships intact. Third, query the backend directly—confirm the trace index exists in Elasticsearch/OpenSearch or that the row is present in Cassandra. If the trace appears in the UI but not in the backend, your read path and write path are not aligned.

Trade-offs and limitations to plan for

A Collector tail-sampling stage is not free. It adds a buffered layer with its own failure modes: if memory or queue limits are too low, traces drop silently. Batch processing introduces a small delay, which is usually acceptable but worth knowing during incident response. Elasticsearch/OpenSearch gives you search flexibility but more operational moving parts; Cassandra gives you write predictability at the cost of deliberate ad hoc querying. Finally, do not hard-code ports like 4317 or 4318 without checking your deployed configuration.

A practical rollout checklist

1. Inventory your span rate, peak fan-out, and retention target in traces or spans per second.
2. Choose OTLP ingestion plus a durable backend that matches your query and write profile.
3. Set head-based sampling for steady-state volume; add tail-sampling only if you need error- or latency-aware retention.
4. Set attribute limits to control high-cardinality fields before increasing retention.
5. Validate with a canary trace containing a parent, child, and error span; confirm it in both Jaeger Query and the backend.
6. Run a retention drill: verify old traces are removed by the index lifecycle policy or TTL mechanism.
7. Query the health endpoint exposed by your build instead of assuming a port.

Tracing stops being fragile when you treat it as a pipeline with explicit decisions at each stage. Pick your ingestion protocol, commit to durable storage, control volume with sampling, and prove the path works before you call the rollout done.