What default memory-overutilization threshold should YARN administrators use for alerts?
23K reputation · 04 Jul 2024, 16:35 UTC
Administrators want to set up YARN Resource Manager alerts that flag container memory over‑utilization promptly while avoiding excessive notifications during normal workload spikes. The YARN Metrics2 subsystem reports container memory usage, but the documentation does not prescribe a universal default threshold; different Hadoop distributions use values ranging from 80 % to 90 % of queue capacity. This lack of a standardized baseline forces teams to tune thresholds empirically, which can lead to inconsistent alert fidelity across clusters.
What threshold percentage provides a reasonable balance between catching genuine over‑utilization events and limiting noise? How can administrators verify that a chosen threshold reduces false positives without missing sustained over‑utilization conditions?