Lock-Free Code on ARM64: Why x86 Intuition Fails and What Replaces It

The x86 assumption that breaks on ARM64

You port a lock-free queue from an x86 build server to an ARM64 cloud instance. The code compiles, the unit tests pass, and then under production load you see corrupted payloads or a consumer thread spinning forever. The logic has not changed; the memory model has. On x86, a plain store followed by a plain store to a different address is observed in program order by other cores. On aarch64, that guarantee does not exist.

Weak ordering is the default, not an edge case

ARM64 is a weakly ordered architecture. Independent loads and stores may be observed out of order, and a store can become visible before an earlier store to a different address. The familiar x86 pattern—write the payload, then set a flag—relies on store-store ordering that x86-TSO provides but aarch64 does not. A compiler barrier such as asm volatile("" ::: "memory") stops the compiler from reordering, but it does not stop the CPU. You need an architectural ordering constraint.

Acquire/release, not barriers, for handoff

The ARMv8.0-A instruction set provides load-acquire (LDAR) and store-release (STLR). A release store ensures all prior memory operations are visible before the store itself; an acquire load ensures all subsequent memory operations happen after the load. In C++ use std::atomic with memory_order_release and memory_order_acquire. In Rust use Ordering::Release and Ordering::Acquire. Let the toolchain emit LDAR/STLR rather than reaching for inline assembly or volatile. volatile forces an access but provides neither ordering nor atomicity.

Full barriers (DMB, DSB) serialize more aggressively and can dominate hot paths. Prefer acquire/release pairs for producer-consumer handoff; reserve heavier barriers for cases where you truly need system-wide ordering.

Worked example: single-producer/single-consumer ring buffer

The classic failure mode is a ring buffer where the producer writes a payload and then publishes an index. On x86 the plain stores often “happen to work.” On aarch64 the consumer can see the new index before the payload is visible, reading stale data.

#include <atomic>
#include <cstddef>

template<typename T, std::size_t N>
class SpscQueue {
    static_assert((N & (N - 1)) == 0, "N must be a power of two");
    struct alignas(64) PaddedAtomic {
        std::atomic<std::size_t> v{0};
        char pad[64 - sizeof(std::atomic<std::size_t>)];
    };
    PaddedAtomic prod_;
    PaddedAtomic cons_;
    T buf_[N];
public:
    void push(const T& item) {
        std::size_t p = prod_.v.load(std::memory_order_relaxed);
        std::size_t c = cons_.v.load(std::memory_order_acquire);
        std::size_t next = (p + 1) & (N - 1);
        // In production, block or drop when next == c.
        buf_[p] = item;
        prod_.v.store(next, std::memory_order_release);
    }
    bool pop(T& out) {
        std::size_t c = cons_.v.load(std::memory_order_relaxed);
        std::size_t p = prod_.v.load(std::memory_order_acquire);
        if (c == p) return false;
        out = buf_[c];
        cons_.v.store((c + 1) & (N - 1), std::memory_order_release);
        return true;
    }
};

The producer copies the payload into buf_[p], then stores the head index with release. The consumer loads the head with acquire before reading the slot. That acquire/release pair is what prevents reading a stale payload on aarch64. The alignas(64) padding keeps the head and tail indices on separate cache lines; the actual line size is implementation-defined, so read CTR_EL0.DminLine if you need to confirm it.

The ARMv8.1 LSE portability trap

ARMv8.1-A introduced Large System Extensions (LSE): single-instruction atomics such as LDADD, LDCLR, LDEOR, LDSET, SWP, CAS, and CASP. These replace the older LDXR/STXR load-linked/store-conditional retry loops, reducing contention and code size. Toolchains emit them when you target armv8.1-a or later, for example with -march=armv8.1-a or -mcpu=<core>+lse.

The trap is that LSE is not universal. Targeting +lse on an ARMv8.0 core such as Cortex-A53, A57, or A72—including Raspberry Pi 3 and 4—causes SIGILL. Before shipping an LSE build, confirm the deployment target actually supports it.

Runtime detection and build strategy

On Linux, check the auxiliary vector for the atomics feature bit:

#include <sys/auxv.h>
#include <asm/hwcap.h>

if (getauxval(AT_HWCAP) & HWCAP_ATOMICS) {
    // LSE available
}

You can also inspect lscpu output for the architecture version. If the feature is absent, fall back to an LL/SC build or use runtime dispatch via an indirect function resolver. Do not assume “ARM64” implies LSE; SVE, PAC/BTI, and cache geometry also vary by core and OS enablement.

Verifying the result

Disassemble the synchronization path and confirm the instructions you expect. Run this on the build host or target where the binary was compiled:

# Replace queue.o with your object file
objdump -d queue.o | grep -E 'ldar|stlr|ldadd|cas'

You should see LDAR/STLR (or LDADD/CAS if LSE is enabled) on the handoff, with no plain STR/LDR crossing the ordering you rely on. If you see only plain loads and stores, the compiler has not emitted the ordering constraints.

For model-level checking, write MP/SB/IRIW litmus tests and run them through the herd tool with the ARMv8 .cat model. Treat those results as model-level: emulators and simulators may not reproduce every ordering corner, so confirm timing-sensitive behavior on real hardware—Raspberry Pi 5, Ampere Altra, or Graviton—under taskset pinning. Stress the ring buffer with long runs, varying thread counts, and CPU frequency changes. Where supported, run under ThreadSanitizer to catch data races the architectural model may not flag.

Limitations and trade-offs

• memory_order_relaxed still gives atomic, single-copy-atomic aligned accesses, but no ordering. Using relaxed for handoff flags is the most common portability bug when moving x86 lock-free code to aarch64. Reserve it for counters where order does not matter.
• Mixing atomic and non-atomic accesses to the same object is undefined behavior, even if the non-atomic access is volatile.
• False sharing remains a performance issue: cache line size is implementation-defined. Padding to 64 bytes is common but not guaranteed; query CTR_EL0.DminLine if you need exact geometry.
• DMB/DSB are not a general fix. They serialize execution and can dominate hot paths. “Fixing” races by sprinkling barriers everywhere hides design bugs and hurts scalability.

Actionable checklist

1. Replace every “write data, then set flag” handoff with memory_order_release on the publisher and memory_order_acquire on the consumer.
2. Compile with -S or objdump -d and verify LDAR/STLR appear on the synchronization path.
3. If you want LSE atomics, target armv8.1-a or later, and add a runtime getauxval(AT_HWCAP) check with an LL/SC fallback.
4. Pad shared indices to the cache line size and keep producer and consumer state on separate lines.
5. Run litmus tests through herd, then repeat stress tests on real ARM64 hardware under CPU pinning.