Choosing Between OpenTelemetry Java Agent and SDK for Production Tracing

Desired Outcome

Decide whether to instrument a Java application with the automatic opentelemetry-javaagent or with explicit SDK code, configure context propagation correctly, and avoid version‑sensitive API pitfalls so that distributed traces appear reliably in your backend.

Prerequisites

• Java 17 or newer (OpenTelemetry 2.x requires Java 11+).
• Access to the application’s JAR or source code.
• Supported OpenTelemetry backend (OTLP collector, Jaeger, Zipkin, etc.) and its current version.
• Knowledge of the application’s service boundaries to set service.name appropriately.
• Optional: Maven or Gradle to add dependencies for SDK instrumentation.

Step 1: Evaluate Scope and Control Needs

Ask two questions:

1. Do you need fine‑grained control over which packages are traced, or is a “trace everything” approach acceptable?
2. Will you be using reactive frameworks (Reactor, RxJava) that require explicit context propagation?

Answers guide you toward the agent (quick, broad) or SDK (precise, explicit).

Step 2: Choose Instrumentation Strategy

Automatic Agent

Pros: Zero code changes, covers most standard libraries automatically. Cons: May generate excessive spans, limited control over export pipelines, and reactive context loss if not configured.

# Run the application with the agent
java \
  -javaagent:/opt/otel/opentelemetry-javaagent-1.28.0.jar \
  -Dotel.resource.attributes=service.name=myservice \
  -jar myapp.jar

Replace 1.28.0 with the agent release that matches your OpenTelemetry SDK version. Mismatched releases can throw NoSuchMethodError at runtime.

Explicit SDK Instrumentation

Pros: Full control over exporters, processors, batching, and package filtering. Cons: Requires code changes and careful pipeline setup.

import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.export.BatchSpanProcessor;
import io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter;

public class OTelSetup {
  public static void init() {
    SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
        .addSpanProcessor(BatchSpanProcessor.builder(
            OtlpGrpcSpanExporter.builder()
                .setEndpoint("http://localhost:4317")
                .build()).build())
        .build();
    OpenTelemetrySdk sdk = OpenTelemetrySdk.builder()
        .setTracerProvider(tracerProvider)
        .setPropagators(ContextPropagators.create(W3CTraceContextPropagator.getInstance()))
        .buildAndRegisterGlobal();
  }
}

Call OTelSetup.init() early in main() or a Spring Boot @PostConstruct method.

Step 3: Configure Context Propagation

Distributed tracing relies on propagators that inject trace context into outbound requests. OpenTelemetry supports multiple standards: W3C TraceContext, b3, and httpbaggage. Consistency across services is critical.

# Agent: set propagators via system property
-Dotel.propagators=tracecontext,baggage

# SDK: set in code as shown earlier
.setPropagators(ContextPropagators.create(
    TextMapPropagator.composite(
        W3CTraceContextPropagator.getInstance(),
        B3Propagator.injector(),
        HttpBaggagePropagator.getInstance()))

For reactive streams, add the otel.reactor or otel.rxjava extensions to preserve context across asynchronous boundaries.

Step 4: Set Up Exporter Pipeline

The agent auto‑configures an OTLP exporter pointing at localhost:4317 by default. Verify that the collector is running and accepts OTLP gRPC. For SDK, you can choose any exporter:

• OTLP (recommended for production)
• Jaeger (UDP or HTTP)
• Console (for debugging)

Example: OTLP exporter to a remote collector.

OtlpGrpcSpanExporter.builder()
    .setEndpoint("https://collector.example.com:4317")
    .build()

Step 5: Verify Traces

After starting the application, open your tracing UI (Jaeger UI, Zipkin UI, or OpenTelemetry Collector’s UI). Look for a service named myservice and a root span matching the first request.

Command‑line verification:

# Agent: check JVM arguments
jcmd $(pgrep -f myapp) VM.command_line | grep opentelemetry

# SDK: use InMemorySpanExporter in unit tests
SpanExporter.reset();
// trigger a span
Tracer tracer = GlobalOpenTelemetry.getTracer("test");
Span span = tracer.spanBuilder("test-span").startSpan();
span.end();
// assert span exists
assert SpanExporter.getFinishedSpans().size() == 1;

Step 6: Recovery and Rollback Options

• Agent mis‑configuration: If spans are missing or too noisy, adjust otel.javaagent.instrumentation.packages to limit instrumentation, e.g., -Dotel.javaagent.instrumentation.packages=com.mycompany.*.
• SDK pipeline failure: If the collector is unreachable, the SDK will buffer spans locally. Check the BatchSpanProcessor logs for retry attempts. To revert to the agent, comment out SDK initialization code and restart.
• Version mismatch: If you upgrade the agent, ensure the SDK (if used) is upgraded in tandem. Verify with mvn dependency:tree | grep opentelemetry.

Limitations and Practical Checks

• The agent cannot exclude specific classes without the packages property; manual filtering is required for fine‑grained control.
• Reactive context propagation is only supported when the corresponding OpenTelemetry extensions are on the classpath; otherwise, spans may drop across async boundaries.
• Exporter compatibility: older exporters (e.g., Jaeger UDP) may be removed in minor releases; always check the collector’s release notes.

Practical check: run otelcol --config otel-collector-config.yaml --log.level debug and watch for “Span received” events to confirm the exporter is functioning.