Tuning the Tomcat Thread Pool: Balancing Throughput and Stability

The Request Queue Bottleneck

When a Java web application slows down under load, the instinct is often to increase the number of available threads. However, blindly increasing thread counts often leads to a "death spiral": higher memory consumption from thread stacks and excessive CPU context switching (the overhead of the OS swapping between active threads), which actually lowers total throughput.

The goal is not to handle every single request simultaneously, but to maintain a steady flow of processing while gracefully handling bursts. This requires a precise configuration of the Tomcat Executor—the component that decouples the network connector from the worker threads that execute your code.

The Mechanics of the Executor

Tomcat uses a thread pool to avoid the overhead of creating a new thread for every single HTTP request. This is managed via the <Executor> element in server.xml. Three primary parameters dictate how the server behaves under pressure:

• minSpareThreads: The baseline number of threads kept alive. This ensures the server can respond immediately to a sudden request without waiting for a thread to initialize.
• maxThreads: The hard ceiling. Once this limit is reached, Tomcat cannot process more concurrent requests.
• acceptCount: The OS-level TCP backlog. When maxThreads are all busy, incoming connections are placed in this queue. If the queue fills up, the OS rejects the connection, resulting in a "Connection Refused" error for the client.

Practical Configuration Example

To implement a shared thread pool across multiple connectors (e.g., HTTP and HTTPS), define the Executor globally in server.xml and reference it by name in the Connector.

<!-- Define the shared thread pool -->
<Executor name="tomcatThreadPool"
          namePrefix="catalina-exec-"
          maxThreads="200"
          minSpareThreads="20" />

<!-- Link the connector to the executor -->
<Connector port="8080"
           protocol="HTTP/1.1"
           executorName="tomcatThreadPool"
           acceptCount="100" />

Execution Context: These changes are made in the conf/server.xml file. You must have filesystem permissions to edit the configuration and administrative access to restart the Tomcat service for changes to take effect.

Verification Steps

1. Baseline Check: Access the Tomcat Manager App to monitor "Current threads busy."
2. Saturation Test: Use a tool like Apache JMeter to send 300 concurrent requests to a slow endpoint (e.g., a 2-second sleep).
3. Observation: You should see "Current threads busy" climb to 200. The remaining 100 requests should sit in the acceptCount queue. Any request beyond 300 should fail immediately with a connection error.

The Trade-off: Throughput vs. Latency

There is a critical tension between maxThreads and acceptCount. A very large acceptCount prevents "Connection Refused" errors, but it introduces "hidden latency." A request might sit in the OS queue for several seconds before a thread even picks it up. By the time the application starts processing, the client may have already timed out.

Furthermore, if your application relies on a downstream database with a connection pool of only 50, setting maxThreads to 500 will not increase throughput. Instead, 450 threads will sit idle, blocked on the database pool, leading to thread starvation where the server is "busy" doing nothing but waiting.

Summary of Tuning Logic

Symptom	Likely Cause	Adjustment
High CPU, low throughput	Too many active threads (Context Switching)	Decrease maxThreads
"Connection Refused" during spikes	Queue full	Increase acceptCount
Slow initial response time	Thread creation overhead	Increase minSpareThreads

To roll back these changes, revert the server.xml to its previous state and restart the service. Always verify your database pool size before increasing Tomcat's thread limits to ensure the bottleneck isn't simply shifted further down the stack.