Diagnosing and Fixing Java Thread Pool Starvation

Recognizable Conditions

A thread‑pool is starved when every worker thread is busy or blocked, yet new tasks keep arriving. Typical symptoms:

• CPU usage stays < 10 % while request latency climbs.
• executor.getQueue().size() stays high or hits its capacity.
• executor.getActiveCount() equals executor.getMaximumPoolSize() consistently.
• Tasks never finish, but the application does not crash.
• With a bounded queue, RejectedExecutionException appears frequently.

Diagnostic Summary

Observation	Possible Cause
Workers BLOCKED or WAITING	Thread dump shows BLOCKED, TIMED_WAITING, or WAITING states on I/O or locks.
Queue full with AbortPolicy	Submission rate exceeds processing capacity; tasks are rejected immediately.
No logs, no completion	Exceptions are swallowed because the task does not surface errors.
Queue grows linearly	Pool size is too small for the volume of blocking work.

Ordered Diagnostic Checks

1. Capture a thread dump

   Run from the host where the JVM is running. Requires jcmd or jstack execution rights.

   # Replace <PID> with the process ID
   jcmd <PID> Thread.print > thread_dump.txt
   # or
   jstack <PID> > thread_dump.txt
   

   Inspect thread names that match your pool (e.g., pool-1-thread-*). States such as BLOCKED on synchronized blocks or WAITING on Object.wait() indicate contention.

2. Monitor executor metrics

   Expose the following via JMX or periodic logs:

   • getQueue().size() – pending tasks.
   • getActiveCount() – threads currently executing.
   • getCompletedTaskCount() – total finished tasks.
   • Custom RejectedExecutionHandler counter for rejections.

   Example: a simple JMX MBean that prints the metrics every 10 s.

3. Check for swallowed exceptions

   Uncaught exceptions in Runnable or Callable terminate the worker thread silently. Wrap task logic or set a global handler:

   executor.setUncaughtExceptionHandler((t, e) -> {
       logger.error("Thread {} terminated", t.getName(), e);
   });
   

4. Measure workload vs. pool capacity

   Record the submission rate (tasks/sec) and average task duration. Estimate required pool size with Little’s Law: Expected Tasks = Rate × Latency. Adjust maximumPoolSize accordingly.

Concrete Example: Bounded Queue with Caller‑Runs Policy

Suppose you have an HTTP service that submits database queries to a pool. The queries wait on a JDBC connection pool, so they often block for 200 ms.

int core = 10;
int max = 20;
BlockingQueue<Runnable> queue = new ArrayBlockingQueue<>(50);
ThreadPoolExecutor executor = new ThreadPoolExecutor(
        core,
        max,
        60L,
        TimeUnit.SECONDS,
        queue,
        new ThreadPoolExecutor.CallerRunsPolicy());

With CallerRunsPolicy the submitting thread executes the task if the queue is full, preventing silent rejection. Verify by running a load test and observing that executor.getQueue().size() stays below 50 and RejectedExecutionException count is zero.

Fixes Tied to Findings

• Blocking calls – add timeouts or asynchronous I/O to reduce blocking time.
• Unbounded queue – switch to a bounded queue and a safe rejection policy to surface backpressure.
• Undersized pool – increase maximumPoolSize only after confirming that workers are idle and not blocked.
• Swallowed exceptions – enforce try‑catch blocks or a global uncaught‑exception handler.
• Queue modification after construction – avoid calling setQueue after the executor is running; instead, create a new executor.

Escalation Criteria

When the following persist, consider a deeper investigation or involve operations:

• Thread dump shows a deadlock that cannot be resolved by timeouts.
• Rejections continue after right‑sizing the pool and queue.
• SLA or latency targets are breached despite tuning.
• Heap or thread churn indicates a leak in task submission logic.

Practical Verification Checklist

1. Run a controlled load test with the current configuration.
2. Record queue.size(), activeCount(), and completedTaskCount() before, during, and after the test.
3. Capture thread dumps at peak load.
4. Apply one fix at a time and rerun the test.
5. Confirm that queue size stabilizes, rejections drop to zero, and latency returns to baseline.

Following this diagnostic flow ensures that you address the root cause of thread‑pool starvation rather than applying ad‑hoc workarounds.