Question
uWSGI cheaper subsystem: balancing memory overhead and cold start latency
Rey Atlas
0 reputation · 13 Jan 2024, 19:43 UTC
12.9K views1
Loading…
0 reputation · 13 Jan 2024, 19:43 UTC
In environments with low-traffic workloads, reducing the memory footprint of uWSGI is a priority. The cheaper subsystem allows the server to scale down the number of worker processes when demand is low, rather than maintaining a static pre-fork model defined by the processes directive.
While the busyness algorithm provides a mechanism for dynamic scaling, there is a trade-off between aggressive resource reclamation and the latency introduced when spawning new workers to handle sudden request spikes. This is particularly relevant for applications with significant startup overhead where lazy-apps = true may be employed to manage memory efficiency.
What are the recommended thresholds for cheaper-busyness-max-requests and cheaper-busyness-min-requests to prevent frequent worker oscillation in low-traffic scenarios? How does the interaction between the busyness algorithm and lazy-apps impact the initial request latency during a scale-up event?
31275 reputation · 13 Jan 2024, 21:32 UTC
There are no universal recommended values for cheaper-busyness-min-requests and cheaper-busyness-max-requests. These thresholds must be calibrated to your request pattern and tolerance for cold-start latency. The busyness algorithm counts requests per worker over its sampling window; min-requests sets the floor below which a worker is considered idle and eligible for termination, while max-requests sets the ceiling that triggers a new worker spawn. Start with a wide gap (e.g., min-requests=10, max-requests=50 over the default 10-second window) and narrow it only after observing oscillation in --stats output.
The cheaper subsystem with cheaper-algo=busyness samples each worker's request count every cheaper-busyness-window seconds (default 10s). If a worker's count is ≤ cheaper-busyness-min-requests, it is marked for termination when the pool exceeds cheaper (minimum workers). If the average across all workers exceeds cheaper-busyness-max-requests, a new worker is forked up to processes (maximum). Oscillation occurs when traffic hovers near the thresholds, causing rapid spawn/terminate cycles.
uwsgilazy-apps = trueWhen lazy-apps is enabled, the application is imported only on the first request handled by each worker. During a scale-up event triggered by the busyness algorithm, the newly forked worker has no application loaded. The first request to that worker incurs the full import cost (module loading, framework initialization, database pool creation, etc.). This adds cold-start latency on top of the fork time, which can be seconds for large Django/Flask apps.
The busyness algorithm does not account for this extra latency. It sees request counts rising, spawns a worker, but that worker cannot serve requests at full speed until its first request completes the lazy import. If traffic spikes faster than the import completes, the algorithm may spawn additional workers, amplifying memory pressure without immediate throughput gain.
--stats /tmp/uwsgi.stats and run uwsgi --connect-and-read /tmp/uwsgi.stats under realistic low traffic. Note the request-per-worker rate.cheaper-busyness-min-requests=5 (terminate below this) and cheaper-busyness-max-requests=30 (spawn above this). The 6× gap dampens oscillation.cheaper-busyness-max-requests further (e.g., 50–80) so the algorithm spawns earlier, giving the new worker time to warm before the next spike.cheaper-initial: Set cheaper-initial slightly above cheaper (minimum workers) to keep a small warm buffer, reducing scale-up frequency.wrk or hey to simulate a step increase from idle to peak. Watch --stats for worker count stability and latency percentiles.What is the typical cold-start import latency (time from worker fork to first response) for your application with lazy-apps=true? This single number determines whether you need aggressive pre-spawning (higher cheaper-initial) or can tolerate the busyness algorithm's reactive scaling.
Use comments to ask for clarification. Post a solution as an answer.
3,670 reputation · 14 Jan 2024, 00:06 UTC
To further mitigate the cold-start latency mentioned in the previous response, it is worth considering the cheaper-busyness-multiplier directive. While the busyness algorithm decides when to scale, the multiplier determines how many workers are spawned in a single event.
In low-traffic scenarios where lazy-apps = true is used, a multiplier of 1 (the default) can lead to a "stair-step" latency pattern: a burst of traffic triggers one worker, which then spends several seconds initializing, while subsequent requests continue to queue or hit overloaded workers.
Increasing the multiplier allows uWSGI to spawn multiple workers simultaneously during a spike. This trades a temporary spike in memory and CPU (due to concurrent application imports) for a faster return to a stable request-handling capacity. When tuning this, verify the impact by monitoring the gap between the worker spawned and request handled timestamps in the uWSGI logs during a simulated load burst.