New Relic APM JVM Heap Growth: Confirming a Leak Without Worsening Memory Pressure

What to Do First?

Before turning on Continuous Profiling on a JVM that is already showing sustained heap growth, follow a lightweight, data‑driven sequence that isolates application‑side leaks from agent‑side retention.

Step 1 – Verify the Leak Indicator

• Check the Heap Used trend over several Full GC cycles. A rising baseline after each GC indicates a leak; a flat or decreasing baseline means the growth is transient or GC‑related.
• Confirm that GC Pause Time is not continually high and that GC Count is not spiking. Excessive pause times can mask a leak but are often a symptom of memory pressure.
• Trigger a manual Full GC (e.g., jcmd GC.run or jcmd GC.heap_dump) and compare the Heap Used before and after. A persistent increase post‑GC is a strong leak sign.

Step 2 – Rule Out Agent‑Side Retention

• Review the agent configuration for limits that keep data in memory:
  • transaction_traces.enabled and transaction_traces.max_samples
  • error_events.enabled and error_events.max_samples
  • custom_events.enabled and custom_events.max_samples
  • Check the newrelic.yml or environment variables for the current agent version’s key names (they differ between 6.x, 7.x, and 8.x).
• Reduce these limits temporarily (e.g., set max_samples to a low value) and observe if the heap trend stabilises. If the growth slows, the agent was a contributor.

Step 3 – Confirm Application‑Side Allocation Hotspots

• Enable Continuous Profiling only for a short, controlled window (e.g., 5–10 minutes). Use the highest sampling interval that still yields useful data—typically 100–200 ms for a 1‑second interval.
• In the New Relic UI, open the Allocation view and look for classes whose allocation rate is steadily increasing over time. Focus on objects that survive GC cycles (i.e., Live counts rising).
• Export the allocation data and cross‑check with a heap dump if available to confirm that the suspect classes are indeed retaining memory.

Choosing a Sampling Interval Under Memory Pressure

When the JVM is already under pressure, the safest approach is to use a *higher* sampling interval (i.e., lower frequency). A 200 ms interval (5 samples per second) is a common compromise: it reduces CPU usage and the amount of metadata the agent keeps in memory, while still exposing long‑term allocation trends. If the heap grows noticeably during the profiling window, tighten the interval gradually—never below 100 ms unless you have verified that the JVM can tolerate the extra overhead.

Agent‑Side Sample Limits to Inspect First

• Transaction trace samples: transaction_traces.max_samples
• Error event samples: error_events.max_samples
• Custom event samples: custom_events.max_samples
• Span event samples (if using distributed tracing): span_events.max_samples

Lowering these limits shortens the retention window for agent‑generated objects, which often consumes a significant portion of the heap in high‑throughput applications.

What We Still Need to Know

Could you share the exact New Relic Java agent version you’re running? The naming of the configuration keys for sample limits varies between major releases, and knowing the version will let us point you to the precise settings.