Tomcat Requests Hang Under Load: Diagnosing maxThreads Exhaustion

The recognizable condition

Your Tomcat instance is alive, CPU is low, memory looks fine — yet requests hang until they time out, and new connections get refused. The access log stops advancing even though traffic is still arriving. This pattern almost always points to Connector thread pool exhaustion: every worker thread is busy (usually blocked on something slow), so the server has no thread left to pick up new work.

The default NIO Connector ships with maxThreads="200" and acceptCount="100". When all 200 threads are occupied, up to 100 additional connections wait in the accept queue; beyond that, the OS starts refusing connections. Because the threads are typically waiting (on a database, an HTTP callout, a lock) rather than computing, CPU stays deceptively low — which is why this gets misdiagnosed as a network problem.

Symptom-to-cause map

Ordered checks

1. Capture thread dumps — twice, five seconds apart

Run on the Tomcat host as the Tomcat user (or root):

jstack $(pgrep -f catalina) > dump1.txt
sleep 5
jstack $(pgrep -f catalina) > dump2.txt

Count threads named http-nio-8080-exec-N and note their state. Threads parked in SocketInputStream.socketRead0 are waiting on a downstream service; threads in BasicDataSource.getConnection or LinkedBlockingQueue.take are waiting on a pool. Comparing the two dumps separates a true deadlock (identical stack frames, locked monitors, no movement) from slow-but-progressing work (stacks change between dumps). A single dump cannot make that distinction.

2. Read the JMX ThreadPool metrics

Via JConsole, jvisualvm, or an existing monitoring agent, inspect the Catalina:type=ThreadPool,name="http-nio-8080" MBean. The smoking gun is currentThreadsBusy pinned at maxThreads while connectionCount keeps climbing. Note that enabling remote JMX requires authentication and SSL configuration — do not expose an unauthenticated JMX port to fix an outage.

3. Check the access log for duration

Ensure your AccessLogValve pattern includes %D (request time in milliseconds) or %F (time to first byte). During the incident window, look for durations climbing toward your proxy or client timeout. If durations are uniformly huge on endpoints that touch the database but normal on static or cached endpoints, the bottleneck is downstream, not Tomcat itself.

4. Inspect downstream pools

Servlet threads most commonly block waiting for a JDBC connection. If you use DBCP (Tomcat's default Resource type), compare numActive against maxTotal (default 8 — far below maxThreads 200, a classic mismatch). If numActive sits at maxTotal while threads queue in getConnection, the database pool is the real ceiling.

Fixes tied to findings

If threads block on the database pool

Size the pool to the workload instead of inflating Tomcat. In context.xml or server.xml:

<Resource name="jdbc/appdb" auth="Container"
          type="javax.sql.DataSource"
          maxTotal="50" maxWaitMillis="5000"
          removeAbandonedOnBorrow="true"
          removeAbandonedTimeout="60"
          logAbandoned="true"
          url="jdbc:postgresql://db-host:5432/appdb" ... />

maxWaitMillis makes threads fail fast instead of queuing forever; removeAbandonedOnBorrow recycles connections leaked by code that never closes them (with logAbandoned printing the stack trace of the culprit). Replace the URL and credentials with your own. Changing this restarts the pool — apply during a maintenance window or to one node at a time behind a load balancer.

If threads are genuinely compute-bound with CPU headroom

Only then raise maxThreads on the Connector in server.xml:

<Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol"
           maxThreads="300" acceptCount="100"
           connectionTimeout="20000" />

Raising maxThreads when threads are blocked downstream makes things worse: you add more waiters competing for the same exhausted resource, increasing memory use and latency without adding throughput.

If slow clients hold threads

Lower connectionTimeout so stalled clients release threads quickly, and confirm your front proxy (nginx, ELB) enforces its own timeouts so Tomcat is not the most patient component in the chain.

Verifying the fix

Reproduce the condition with a load tool (wrk, JMeter) against a staging instance, ideally including one deliberately slow endpoint. Confirm currentThreadsBusy plateaus at maxThreads before the fix and stays below it after. Take paired thread dumps again to verify threads no longer park on the same downstream call, and check catalina.out and the localhost log for "All threads are busy" messages correlating with the incident window. Attribute names and defaults differ across Tomcat 8.5, 9, and 10.1 — verify against the version you actually run before editing configuration.

When to escalate

Hand off to application owners when dumps show contention on application-level locks, unbounded internal queues, or synchronous calls to a slow third-party API — those need code changes, not tuning. Involve infrastructure when you hit OS limits: "Too many open files" in the logs points at file descriptor ulimits, and refused connections with idle threads point at TCP backlog (somaxconn) rather than anything in server.xml. If dumps show a genuine deadlock inside a library, capture the full dumps and open an issue against that component with both snapshots attached.