Optimizing ARM64 Atomics: When to Use Large System Extensions (LSE)

The Cost of the Load-Linked/Store-Conditional Loop

In standard ARMv8.0-A, atomic operations are implemented using a Load-Linked/Store-Conditional (LL/SC) mechanism. This involves a pair of instructions—typically LDXR (Load Exclusive Register) and STXR (Store Exclusive Register)—that create a loop. If another processor modifies the memory address between the load and the store, the store fails, and the CPU must retry the entire sequence.

Under high contention—where many threads fight for the same memory address—this retry loop causes significant latency and CPU waste. For developers building high-performance concurrent systems on ARM64, the solution is the Large System Extensions (LSE) introduced in ARMv8.1-A. LSE replaces these loops with single, atomic instructions like LDADDAL (Atomic Add) and CASAL (Compare and Swap), moving the synchronization logic into the hardware interconnect rather than the software loop.

Compiler Flags vs. Runtime Dispatch

The primary challenge is deciding how to enable LSE without breaking compatibility with older ARMv8.0 hardware. You have two main paths: static compilation and runtime dispatch.

Static Compilation

If you control your entire fleet and know every CPU is ARMv8.1-A or newer, you can tell the compiler to emit LSE instructions directly. Using -march=armv8-a+lse (or a specific -mcpu like neoverse-n1) allows std::atomic in C++ or __atomic builtins in C to generate the faster LSE mnemonics.

Runtime Dispatch

If your binary must run on both old and new ARM64 chips, you cannot use the +lse flag globally. Doing so will cause an Illegal Instruction crash on ARMv8.0 systems. Instead, you rely on the toolchain's default behavior: the compiler emits calls to a library (like libatomic) or uses the LL/SC fallback. Modern glibc implementations often include runtime dispatching, where the library checks the CPU's capabilities at startup and redirects atomic calls to the most efficient implementation available for that specific core.

Verifying LSE Implementation

To determine if your build is actually utilizing LSE, you can examine the compiled assembly. A simple atomic increment reveals the difference immediately.

Example: Atomic Increment Comparison

// increment.c
#include 
void inc(atomic_int *ptr) { atomic_fetch_add(ptr, 1); }

Run these commands on your build machine:

1. Build without LSE: gcc -O2 increment.c -o inc_std
2. Build with LSE: gcc -O2 -march=armv8-a+lse increment.c -o inc_lse
3. Inspect: objdump -d inc_std | grep -E "ldxr|stxr" and objdump -d inc_lse | grep "ldaddal"

In the inc_std binary, you will see a loop involving ldxr and stxr. In the inc_lse binary, you will see a single ldaddal instruction. This confirms the compiler is generating the hardware-accelerated atomic operation.

Checking Hardware Support at Runtime

Before deploying an LSE-optimized binary, you must verify the target environment. On Linux, the kernel exposes hardware capabilities via getauxval. You can check for LSE support programmatically or via the shell.

Shell Check:

grep -i "lse" /proc/cpuinfo

If the lse flag is absent from the features list, any binary containing LSE instructions will fail. For C developers, the correct way to check this at runtime is using the AT_HWCAP auxiliary vector:

#include <sys/auxv.h>
#include <stdio.h>

int main() { 
    unsigned long hwp = getauxval(AT_HWCAP);
    if (hwp & HWCAP_LSE) { 
        printf("LSE supported\n"); 
    } else { 
        printf("LSE not supported\n"); 
    } 
    return 0; 
}

Trade-offs and Limitations

While LSE reduces the overhead of atomic retries, it is not a magic bullet for all concurrency problems. LSE instructions optimize the mechanism of the update, but they do not solve false sharing. If multiple threads are updating different variables that happen to reside on the same cache line, the hardware interconnect will still struggle with cache coherency traffic, regardless of whether you use LDADDAL or an LL/SC loop.

Additionally, LSE is designed for simple operations (add, swap, compare-and-swap). For complex critical sections involving multiple memory locations, traditional mutexes or spinlocks are still required.

Summary Checklist

• Targeting specific new hardware? Use -march=armv8-a+lse for maximum performance.
• Targeting general ARM64? Stick to the default architecture and let glibc handle runtime dispatch.
• Verifying? Use objdump to look for ldaddal/casal and /proc/cpuinfo to confirm hardware support.