Transition from 'cache-size' to 'max-memory' in detaspace v2.4.0: impact on memory usage under high concurrency

In detaspace v2.4.0, transitioning from cache-size to max-memory represents a shift from application-level object limiting to JVM-level heap enforcement. While max-memory is designed to cap total heap usage, its effectiveness under high concurrency depends heavily on the garbage collector (GC) ability to reclaim memory faster than the allocation rate.

Impact of Asynchronous GC Hints

Enabling asynchronous garbage collection hints does not change the theoretical heap ceiling defined by max-memory, but it significantly impacts the observed ceiling. If hints are enabled, the system triggers collection cycles more proactively, potentially preventing the heap from hitting the hard limit. Without these hints, the application may spike toward the max-memory limit before a synchronous GC cycle completes, leading to 'stop-the-world' pauses or unnecessary OOM errors.

cache-size vs. max-memory Performance

The measurable difference between the two configurations lies in memory overhead:

• cache-size: Limits only the number of cached entries. Under high concurrency, this often results in unpredictable memory usage because non-cached objects (request-scoped buffers) are not capped.
• max-memory: Caps the total footprint. This provides a safety net for high-concurrency environments but risks OOM if the overhead of managing concurrent requests exceeds the remaining heap space.

Concurrency Thresholds and OOM Conditions

There is always a threshold where max-memory fails to prevent OOM conditions. This occurs when the allocation rate exceeds the reclamation rate. Even with GC hints, if high concurrency creates short-lived objects faster than the GC can sweep them, the heap will exhaust before the next cycle can finalize objects.

Verification Steps

To verify the behavior in your environment, use the following scoped approach:

1. Set max-memory to a specific value (e.g., 4GB) and monitor the Xmx equivalent via jmap -heap <>.
2. Execute a sustained load test using a tool like k6.
3. Analyze GC logs using -Xlog:gc* to check if the heap consistently nears the limit or if OOM occurs despite collection.