Diagnosing ARM64 Bottlenecks: Frequency, Memory, and Vector Utilization

Recognizing Subsystem Bottlenecks in AArch64 Servers

If your ARM64 server shows poor performance despite low CPU utilization, the issue is rarely simple CPU load; it is usually a bottleneck in memory subsystem, clock frequency, or instruction set utilization. The key takeaway is to move beyond generic CPU usage metrics and check specific hardware performance counters.

Symptom Mapping and Root Causes

Use this table to quickly map observed performance symptoms to potential AArch64 hardware constraints:

Ordered Diagnostic Checks

Execute these checks sequentially to isolate the constraint:

1. Confirm Architecture: Run lscpu --architecture (any user). Expected output confirms aarch64.
2. Check Frequency Throttling: Read /sys/devices/system/cpu/cpufreq/policy*/scaling_cur_freq and scaling_max_freq (root/sysfs access). Verify current frequency is near maximum under load.
3. Monitor Memory Pressure: Run perf stat -e mem_load_retired.l3_miss -a sleep 10 (root). High miss rates indicate memory pressure.
4. Validate TLB Efficiency: Run perf stat -e dTLB-load-misses,iTLB-load-misses -a sleep 10 (root). A ratio over 1% suggests page table overhead.
5. Measure SVE Efficiency: Run perf stat -e arm_sve/container_mig -a sleep 10 (root). Efficiency below 30% points to code optimization needs.

Targeted Fixes Based on Findings

Frequency Throttling

If throttling is observed, check thermal status: cat /sys/class/thermal/thermal_zone*/temp. Fix involves improving cooling or adjusting the governor (e.g., echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor).

Memory Saturation

For high L3 misses, enable huge pages for large allocations: echo 1024 > /proc/sys/vm/nr_hugepages. Always profile access patterns first using perf record -g.

SVE Underutilization

If SVE efficiency is low, ensure compilation targets the feature: use compiler flag -march=armv8-a+sve. Use perf annotate on hot paths to confirm vector instruction usage.

Escalation and Verification

If fixes do not resolve the issue, verify kernel support: check perf list | grep -i sve to ensure PMU events are available for your kernel version. Note that kernels prior to 5.10 may lack full SVE coverage. After any state-changing operation (e.g., setting huge pages), re-run the relevant perf stat command to confirm metric improvement.