Choosing a Jaeger Backend: Cassandra vs Elasticsearch vs PostgreSQL

Decision Context

When deploying Jaeger in a production environment you must pick a storage backend that can handle your trace ingestion rate, query latency, and operational skill set. Jaeger supports three officially documented backends: Cassandra, Elasticsearch, and PostgreSQL. Each offers a different trade‑off between write throughput, read performance, and operational complexity.

Constraints to Consider

• Ingest Rate – Spans per second (SPS) you expect to write.
• Query Latency – Maximum acceptable latency for typical trace lookups.
• Scalability – Ability to add nodes without major re‑configuration.
• Operational Expertise – Team familiarity with the database ecosystem.
• Consistency Needs – Whether you need strong consistency on writes.
• Cost & Resources – Storage cost per GB, compute overhead.

Backend Comparison

Trade‑off Analysis

• Cassandra is ideal when you need high write throughput and can tolerate eventual consistency. It shines for workloads where trace ingestion is the bottleneck and query latency is acceptable if you tune compaction and consistency.
• Elasticsearch offers powerful search and analytics on span attributes. If you need advanced filtering (e.g., by error message or custom tags) and can afford index management overhead, Elasticsearch is the right choice.
• PostgreSQL provides the simplest operational model and strong consistency. For moderate workloads (tens of thousands of traces per second) or when you already run PostgreSQL, it can be a cost‑effective solution.

Concrete Implementation: Jaeger with Cassandra

The following example shows how to launch a minimal Jaeger stack that writes to Cassandra, using Docker Compose. It demonstrates the key environment variables and a basic validation script.

Prerequisites

• Docker ≥ 20.10
• Docker Compose file format 3.8
• Jaeger Docker images v1.46 (or newer)

docker-compose.yml

version: "3.8"
services:
  cassandra:
    image: cassandra:4.0
    container_name: jaeger-cassandra
    environment:
      - CASSANDRA_CLUSTER_NAME=JaegerCluster
      - CASSANDRA_DC=dc1
      - CASSANDRA_RACK=rack1
    ports:
      - "9042:9042"
    volumes:
      - cassandra-data:/var/lib/cassandra

  jaeger-collector:
    image: jaegertracing/jaeger-collector:1.46
    container_name: jaeger-collector
    environment:
      - COLLECTOR_ZIPKIN_HTTP_PORT=9411
      - COLLECTOR_STORAGE_TYPE=cassandra
      - COLLECTOR_CASSANDRA_HOSTS=jaeger-cassandra
      - COLLECTOR_CASSANDRA_KEYSPACE=jaeger
      - COLLECTOR_CASSANDRA_REPLICATION_FACTOR=3
    depends_on:
      - cassandra
    ports:
      - "14267:14267"
      - "14268:14268"

  jaeger-query:
    image: jaegertracing/jaeger-query:1.46
    container_name: jaeger-query
    environment:
      - QUERY_STORAGE_TYPE=cassandra
      - QUERY_CASSANDRA_HOSTS=jaeger-cassandra
      - QUERY_CASSANDRA_KEYSPACE=jaeger
    depends_on:
      - jaeger-collector
    ports:
      - "16686:16686"

volumes:
  cassandra-data:

Key Points in the Config

• COLLECTOR_STORAGE_TYPE=cassandra tells the collector to write to Cassandra.
• Both collector and query services specify the same keyspace (jaeger) and host.
• Replication factor of 3 is recommended for production to avoid single‑point failures.

Running the Stack

# Start the cluster
$ docker compose up -d
# Verify containers are healthy
$ docker compose ps
# Wait a few minutes for Cassandra to finish bootstrap.

Validate Ingest Latency

Use the Jaeger client stress tool to generate a burst of traces. Replace <COLLECTOR_ENDPOINT> with http://localhost:14268/api/traces and adjust the --rate to match your expected SPS.

# Example command
$ jaeger-client-go/stress/jaeger-stress \
    --collector <COLLECTOR_ENDPOINT> \
    --num-traces 200000 \
    --rate 2000 \
    --duration 30s
# The tool prints average write latency and throughput.

Check that the average write latency stays below your SLA (e.g., < 10 ms for high‑traffic services).

Validate Query Latency

After ingest, query a known trace ID using the Jaeger UI or the REST API. The REST call below demonstrates how to fetch a trace and measure response time.

# Replace <TRACE_ID> with an actual ID from the stress run
$ curl -w "\nTime: %{time_total}\n" \
    -H "Accept: application/json" \
    http://localhost:16686/api/traces?traceID=<TRACE_ID>

Typical query latency for Cassandra‑backed Jaeger should be 50–200 ms for a single trace, depending on cluster size and compaction settings.

Pagination Check

Test offset pagination by requesting a large number of traces for a service. The following command fetches 500 traces in two pages.

# Page 1
$ curl http://localhost:16686/api/services/myservice/traces?limit=250
# Page 2 (offset 250)
$ curl http://localhost:16686/api/services/myservice/traces?limit=250&offset=250

Verify that the second page contains traces that start after the last trace ID of the first page.

Operational Caveats

• For Cassandra, compaction strategy matters. Use LeveledCompactionStrategy for read‑heavy workloads.
• Ensure hinted handoff is enabled; otherwise, write latency spikes during node failures.
• For Elasticsearch, keep shard count per index below 10–20 to avoid query latency spikes. Use ilm.policy for automated rollover.
• For PostgreSQL, increase max_connections and use a connection pooler (e.g., PgBouncer) if you hit the default limit.

Conclusion

Choosing a Jaeger backend boils down to the trade‑offs you’re willing to make. If you need the highest write throughput and can manage Cassandra’s operational quirks, go with Cassandra. If advanced search and analytics are critical, Elasticsearch is the natural fit. For modest workloads or teams already comfortable with relational databases, PostgreSQL offers a simpler path.