Tomcat Async Servlets: Releasing Request Threads During Downstream I/O

Problem and Takeaway

When a Java servlet waits for a database query or an external HTTP call, the request thread is blocked. Under sustained load, Tomcat's connector thread pool fills up, and new requests are queued or rejected with 503 errors.

The useful takeaway: since Servlet 3.0 (Tomcat 7+), you can release the request thread back to the pool while downstream I/O completes. You do this by calling request.startAsync(), not by adding a special thread pool to server.xml.

How Async Works in Tomcat

Asynchronous servlet support is enabled by default on Tomcat connectors. You opt in per request: when your servlet calls HttpServletRequest.startAsync(), Tomcat immediately returns the current thread to the connector pool and hands you an AsyncContext object. The response stays open until you call AsyncContext.complete().

The asyncTimeout attribute on the <Connector> element controls how long Tomcat waits before forcibly timing out the async operation. The default is 30000 ms (30 seconds), but you should verify this for your Tomcat version.

Configuring asyncTimeout

No separate "async thread pool" is required in server.xml. If you want to change the wait window, edit the connector definition. Run your editor as the tomcat user or via sudo.

<Connector port="8080"
           protocol="HTTP/1.1"
           connectionTimeout="20000"
           asyncTimeout="60000"
           redirectPort="8443" />

Restart Tomcat after saving. Verify the change by checking the startup logs for parse errors, or by issuing a request that intentionally stalls and confirming the timeout occurs after roughly 60 seconds rather than 30.

A Correct Servlet Example

A common mistake is creating a CompletableFuture without calling startAsync(). The servlet thread is only released when you start async processing. Also note that @Async is a Spring framework annotation; it does not replace the standard servlet API.

@WebServlet(urlPatterns = "/report", asyncSupported = true)
public class ReportServlet extends HttpServlet {
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp)
            throws ServletException, IOException {

        final AsyncContext ctx = req.startAsync();
        ctx.setTimeout(60_000);

        CompletableFuture
            .supplyAsync(() -> fetchReportData())
            .whenComplete((data, ex) -> {
                try {
                    HttpServletResponse asyncResp =
                        (HttpServletResponse) ctx.getResponse();
                    if (ex != null) {
                        asyncResp.sendError(
                            HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
                    } else {
                        asyncResp.setContentType("application/json");
                        asyncResp.getWriter().write(data);
                    }
                } catch (IOException e) {
                    log("Async response failed", e);
                } finally {
                    ctx.complete();
                }
            });
    }
}

Key points: asyncSupported = true is required on the servlet. The whenComplete callback runs on the thread that finishes the future, and ctx.complete() is always called exactly once in the finally block.

Trade-offs and Limitations

Async processing adds real complexity. You must guarantee that complete() is invoked, or the client hangs until asyncTimeout fires. Request-scoped data such as security principals, MDC logging context, and ThreadLocal values do not automatically propagate to the background thread; you must copy them manually.

Async helps only when the bottleneck is I/O wait. If your downstream service is CPU-bound or the work is trivial, the overhead of context switching and manual completion handling may outweigh the benefit.

Practical Verification

To confirm the behavior on your system:

1. Check the Tomcat version by running ${catalina.base}/bin/catalina.sh version (execute permission required). Async support is available in Tomcat 7 and later.
2. Run a load test such as ab -n 200 -c 50 http://localhost:8080/report against a servlet that sleeps for two seconds.
3. While the test runs, capture a thread dump with jstack <tomcat-pid>. In a synchronous servlet, you would see roughly 50 connector worker threads blocked. With async processing, the number of blocked connector threads should be far lower because they were returned to the pool.
4. Check the logs for AsyncTimeoutException or messages from org.apache.catalina.core.AsyncContextImpl indicating timeouts.

If thread counts do not drop, confirm that startAsync() is actually reached and that no filter or wrapper is buffering the response in a way that prevents async dispatch.