Question
Bitbucket Server Tomcat thread pool configuration for concurrent REST API load
Ira Nea
0 reputation · 26 Apr 2020, 10:04 UTC
118.8K views1
Loading…
0 reputation · 26 Apr 2020, 10:04 UTC
Goal: reduce latency spikes observed when many concurrent users request pull‑request lists via the Bitbucket REST API by sizing the underlying Tomcat HTTP thread pool appropriately.
Constraint: the pool’s maxThreads and acceptCount are not configurable through the Bitbucket UI or server.properties; they must be set in server.xml, which requires a restart and can be overwritten during upgrades unless placed in /shared. Additionally, increasing the pool without sufficient heap or CPU can cause out‑of‑memory errors or GC pauses, and there is no built‑in autoscaling mechanism.
Uncertainty: what numeric values provide the best trade‑off for a given hardware profile, how to persist the custom server.xml across upgrades safely, and which JMX metrics indicate the pool is nearing saturation.
What maxThreads and acceptCount values minimize 95th‑percentile latency for a typical 2‑core, 8 GB JVM under 150 concurrent pull‑request list requests?
How can the edited server.xml be placed in /shared to survive upgrades without manual re‑application?
Which JMX attributes (e.g., queue length, active threads) should trigger an alert that the Tomcat thread pool is saturated?
34775 reputation · 26 Apr 2020, 20:25 UTC
For a 2-core, 8 GB JVM environment facing 150 concurrent pull-request list requests, start with maxThreads="200" and acceptCount="100".
In a resource-constrained environment (2 cores), setting maxThreads too high (e.g., 500+) often increases 95th-percentile latency due to excessive CPU context switching and increased GC pressure on the 8 GB heap. A value of 200 provides a buffer above your 150-request baseline without overwhelming the scheduler. acceptCount is set to 100 to act as a safety valve; this is the OS-level queue for connections that cannot be immediately handed to a worker thread. If this queue fills, the server will refuse connections, preventing a total system collapse under extreme spikes.
To prevent your modifications from being overwritten during a Bitbucket Server upgrade, you must move the customized server.xml to the shared home directory. Bitbucket is designed to prioritize configuration files found in the shared directory over those in the installation directory.
bitbucket
tomcatserver.xml in the installation directory (typically /opt/atlassian/bitbucket/conf).server.xml to the conf directory within your Bitbucket Shared Home directory.To determine if the pool is saturated and causing the observed latency spikes, monitor the following JMX attributes within the Tomcat Connector MBean:
maxThreads limit, requests are being queued.currentThreadsBusy equals maxThreads, the acceptCount queue begins to fill. A rising queue length is a primary indicator of saturation.Because optimal sizing depends on the specific latency of your database and disk I/O, verify these settings with a scoped load test. Use a JMX client (such as JConsole or VisualVM) to track currentThreadsBusy while simulating 150 concurrent requests. If you observe high CPU usage (>80%) and increasing GC pauses alongside these spikes, reduce maxThreads to lower the concurrency overhead.
Missing Diagnostic Detail: To refine these numbers, please provide the average response time (service time) of a single pull-request list API call under no-load conditions.
Use comments to ask for clarification. Post a solution as an answer.
5,550 reputation · 26 Apr 2020, 16:42 UTC
On a 2‑core JVM, a pool larger than the effective concurrency (≈80–100 threads) rarely cuts 95th‑percentile latency and can inflate GC pressure. Start with maxThreads=90 and acceptCount=20, then scale in 10‑thread increments while watching latency.
Copy the original server.xml from /opt/atlassian/bitbucket/conf into $BITBUCKET_HOME/shared and edit it there. The installer preserves shared/server.xml across upgrades, but always back it up and test on a staging copy after each major release.
currentThreadsBusy > 70–80 % of maxThreadstaskQueue size during sustained loadrejectedTasks countThese thresholds trigger alerts before latency spikes become visible to users.