Service Flow vs Transaction Tracing: Trading Service-Level Overview for Call-Stack Precision Under Custom Instrumentation

Question

In recent Dynatrace versions, when a latency spike appears in an application that uses custom instrumentation, the immediate goal is to isolate whether the delay sits at a service boundary or inside a specific code path. Transaction Tracing provides per-transaction call stacks, SQL statements, and HTTP calls, but it carries higher CPU and memory overhead and may require sampling under high throughput. Service Flow offers an aggregated view of inter-service communication and latency hotspots, yet its completeness depends on Automatic Service Detection, which can misclassify services when custom instrumentation is present and produce incomplete topology graphs.

This creates an unresolved trade-off: accept a potentially partial Service Flow diagram to keep overhead low, or enable targeted Transaction Tracing sessions to capture exact call stacks despite the performance cost. The uncertainty deepens when missing services in the flow cannot be confirmed without cross-referencing application logs or manual service mapping.

Which documented approach should be prioritized first when Automatic Service Detection yields an incomplete Service Flow? How should the choice between Transaction Tracing overhead and Service Flow topology gaps be evaluated for a high-throughput, custom-instrumented service? What verification steps can confirm that the selected view actually captures the true bottleneck without introducing measurement distortion?

Zoha · Answer

Answer
When Automatic Service Detection yields an incomplete Service Flow, the first documented step is to verify and correct the detection itself before deciding to enable Transaction Tracing. Only after the service topology is as complete as possible should you consider a targeted, short‑window Transaction Trace to obtain call‑stack precision.
Likely explanation (under custom instrumentation)
Custom instrumentation can split, rename, or bypass the expected service boundary, causing the detection logic to miss or misclassify services. This leads to gaps or wrong edges in the Service Flow diagram even though the actual code path exists.
Confirmed facts (from Dynatrace behavior)

Service Flow provides an aggregated, low‑overhead view but its completeness depends entirely on Automatic Service Detection.
Transaction Tracing delivers per‑transaction call stacks, SQL, and HTTP details, but it incurs higher CPU/memory overhead and may require sampling under high throughput.
A service‑level hotspot in Service Flow combined with a matching slow segment in a trace is strong evidence of the bottleneck; a topology gap alone is weak evidence.

Steps to evaluate and act

Verify detection: In the Dynatrace UI, check the process group and service naming rules for the custom‑instrumented service. Confirm that the expected entry point (e.g., a specific class/method or HTTP endpoint) creates the correct service name and appears in the detected service list.
Correct detection if needed: Adjust service detection rules (custom service naming, API‑based service detection, or OneAgent SDK calls) to ensure the service boundary is recognized. After any change, re‑baseline and observe the Service Flow for the expected edge and latency hotspot.
Re‑evaluate Service Flow: If the topology now shows the expected edge with a latency hotspot, you can proceed with service‑level optimizations without tracing.
If the edge remains missing or the hotspot is unclear: Enable a targeted Transaction Trace limited to a reproducible request signature (e.g., a specific URL, HTTP method, or custom attribute) and a narrow time window (e.g., 5 minutes).
Monitor overhead: During the trace window, record CPU, memory, and request latency percentiles; compare them to a baseline without tracing to quantify distortion.
Confirm the bottleneck: Match the trace’s slow call/SQL/HTTP segment to the service‑level latency observed in Service Flow. Verify that latency improves after applying the fix to the identified code path.

When to ask for a missing diagnostic detail
If no reproducible request signature or attribute is known to trigger the spike, ask for that detail before enabling tracing; otherwise the trace may capture unrelated traffic and distort the conclusion.

Service Flow vs Transaction Tracing: Trading Service-Level Overview for Call-Stack Precision Under Custom Instrumentation

1 answer

Answer

Likely explanation (under custom instrumentation)

Confirmed facts (from Dynatrace behavior)

Steps to evaluate and act

When to ask for a missing diagnostic detail

Your perspective belongs here.

0 question comments