Limits of concurrent request handling in R's single‑threaded interpreter

Direct answer

For CPU-bound R services, the practical concurrency ceiling is roughly the inverse of the mean serialized request-handling time. Base R evaluates user code in a single thread, so Plumber endpoints queue inside the interpreter. Adding cores or forking workers does not raise that ceiling for compute-heavy handlers; it only adds overhead.

What is confirmed vs. what is likely

Confirmed behavior: Plumber serializes handler execution by default. Shiny’s promises framework can overlap I/O waits but cannot parallelize CPU-bound R evaluation. On Unix, parallel::mclapply uses fork with copy-on-write; on Windows, fork is unavailable and the function falls back or is unsupported.

Likely explanation: Garbage collection performs stop-the-world pauses whose duration is driven by live heap size and allocation volume. Request paths that parse large payloads or create many intermediate objects increase GC frequency and can dominate tail latency during concurrent bursts. A parallel::cluster backend avoids fork but adds socket serialization and explicit data-export overhead, so it does not automatically reduce tail latency for fast endpoints.

Steps to find your ceiling

1. Benchmark a single request at your target payload size and record mean and p99 latency.
2. Estimate the serialized ceiling as 1 / mean-request-time and compare it with observed throughput under increasing load.
3. Profile allocations during a burst using Rprof or gcinfo(TRUE), then reduce intermediate object creation in the hottest paths.
4. Re-measure p99 latency after each change.

Example profiling commands:

Rprof("burst.prof", interval = 0.01)
# run a representative burst of requests
Rprof(NULL)
summaryRprof("burst.prof")

When workers might help

If endpoints run for seconds or longer, run a controlled A/B test on identical hardware comparing a single Plumber process with mclapply and a cluster backend. Record throughput and p99 at each concurrency level. For short requests, fork and IPC costs usually exceed the serialized handler time, so worker backends rarely improve tail latency.

One missing diagnostic

Are your endpoints primarily CPU-bound or I/O-bound? If they are I/O-bound, asynchronous patterns or external workers may help. If they are CPU-bound, the single-threaded interpreter sets the ceiling and effort should focus on reducing per-request compute and allocations.

Version note: These observations reflect current R 4.x behavior. Exact request-rate thresholds and GC pause durations vary with R version, package versions, payload size, and host memory; verify on your own hardware before setting capacity limits.